Experienced proofreaders and editors are familiar with the challenges of ensuring the highest quality content. They aim for flawless accuracy, style, and readability, always intent on capturing the authentic voice of the author or brand. So, they keep a keen eye out for the latest natural language tools.
Getting To Know ChatGPT
Most editors, proofreaders, and content creators agree that tools like Open AI’s ChatGPT can produce remarkably fluent content for everything from basic emails to highly detailed reports. And when it comes to improving text, ChatGPT seems particularly effective at fixing grammar errors, along with most punctuation. Tests by Tencent Lab at the Chinese University of Hong Kong found that ChatGPT performed best in grammatical correctness, even outperforming Grammarly. At the same time, however, the tool tended to stray from the original “expression” of the source sentence toward overcorrection.
ChatGPT tells users its capabilities for follow-up corrections upfront – as well as its limitations in that it may occasionally generate incorrect information, produce unhelpful instructions, biased content, or even incorrect or nonsensical answers.
Nevertheless, ChatGPT’s tremendous acumen for pattern recognition makes it a powerful tool for swift and sweeping improvements, while allowing users to home in on training it to achieve better outcomes. Eagle-eyed editors can prompt the tool to fine-tune everything from targeting the right audience to keywords and specific styles to comma preferences. However, collaboration is key to ensuring accuracy.
An Experiment With ChatGPT
If anything, ChatGPT’s programming seems committed to continuous improvement. To test the tool’s approach to semantic meaning, intent, and nuance, we tried a simple experiment using text from a Wikipedia page on the topic of “content (media)”:
Prompt: “Please proofread and edit to improve the language and style of this content. Please use AP style guide. List all changes at the end with reasons for each change:”
ChatGPT did a good job with grammar and flow, but it missed some nuances, starting from the first sentence:
Original sentence: “Content is the information contained within communication media.”
Edited sentence: “Content is the information conveyed through various communication media.”
ChatGPT cited “clarity and conciseness” as reasons for rewording the sentence. Astute human editors would question the word change, regardless of the apparent logic. Upon closer inspection, the original term contained is more accurate and meaningful than the replacement term conveyed – since media in the context of communication media is defined as a means for both storing and delivering information. ChatGPT missed the contextual meaning of content as something contained in this instance.
The tool went on to cite “concision and clarity” two more times in the first paragraph, taking liberties with possible meanings in a tradeoff for more streamlined content. It also falsely claimed that it had added an extra sentence. A staunch editor could continue arguing with the ChatGPT tool over semantics!
Testing ChatGPT
The results of our little experiment matched the conclusion of the Tencent Lab study above: ChatGPT performs strongly in relation to grammar but tends to overcorrect to the point of pushing semantical boundaries. In addition:
- ChatGPT doesn’t currently offer a direct interface for tracking changes, as with Microsoft Word. Although you can prompt ChatGPT to list all edits or changes, along with the reasons for the changes, cross-referencing can be tedious, and explanations short or vague. Human tracked changes are easier to follow, with more in-depth explanations and clarifications.
- As the old adage goes “garbage in, garbage out”: quality output from ChatGPT depends on the quality of your prompt. At the same time, no matter how good your prompt is, ChatGPT could spit out unreliable information or inject bias.
- Because ChatGPT sounds authoritative, it can be easy to assume it always knows what it’s talking about. Remember, it’s a tool operating on assumptions and patterns that may not necessarily identify facts or the correct meaning.
As with all editing, pay close attention to proper context when editing with AI tools.
Semantic Play of ChatGPT
ChatGPT’s penchant for semantic play is nicely illustrated in another experiment, where a researcher asked ChatGPT to improve the famous song lyrics of “Stan” by Eminem:
Original song intro:
Dear Slim, I wrote but you still ain’t calling
I left my cell, my pager, and my home phone at the bottom
I sent two letters back in autumn, you must not’ve got ’em
There probably was a problem at the post office or something
ChatGPT’s improvements:
Dear Slim, I’ve written you multiple times
But still, no reply, I feel left behind
I sent letters through autumn rain and snow
Maybe they got lost, I don’t really know
ChatGPT’s reasons for improvements:
“The intro was reworded to make it sound more emotional and desperate. The original intro lacked emotion, and the changes made it more impactful and personal.”
This experiment demonstrates ChatGPT’s endless capacity for both fabricating nonsense and grammatical wordplay. While it may stretch boundaries or miss nuances, its evolving capabilities around patterns make it a valuable tool for editors to watch and work with.
Some More AI-Editing Takeaways
Quality, authentic content is the goal – especially for business leaders who function as “thought leaders” with cutting-edge brands.
- Whether you use human editors only or work with a natural language processing tool like ChatGPT, ensure you check content for originality and quality with a content checker.
- Always check for factual accuracy, sources, and potential biases – especially with AI tools. Ask ChatGPT, or whichever tool you use, where it has sourced information from. You may have to do some homework to dig beyond the information the tool can provide.
Even with the most advanced tools, there is ultimately no substitute for human editing. Seasoned editors understand niches with a sense of brand or authorial voice. They never stop questioning context to capture meaning. And at the same time, they’re wise enough to use tools to improve their work wherever possible.
Ready to Supercharge Your Content?
The combination of Proofed’s traditional editing expertise along with evolving natural programming tools like ChatGPT makes for a supercharged combination! Let us help you scale up with cutting-edge quality. Learn more about our editorial services for AI-generated content and schedule a call with our team to find out how we Proofed can serve your content needs.
