For this article, we investigate how Hemingway Editor, an editing app, compares with a human editor in improving the overall quality of a writing sample and removing any mistakes. Keep reading to see how well each performed and where they fell short.
Side-by-Side Comparison
While Hemingway Editor had several positive features, it failed to accomplish the goal: to improve the writing sample and remove mistakes in grammar, punctuation, and spelling.
The human editor outperformed Hemingway Editor because they actually made changes to the writing sample, removed all writing mistakes, and greatly improved its overall quality and readability.
Pros
| Hemingway Editor | Human Editor | |
|---|---|---|
| ✔ Easy to use at home | ✔ Fixes all writing issues, such as grammar, spelling, and punctuation mistakes | |
| ✔ Free | ✔ Checks each sentence for clarity, meaning, and flow, making changes as needed | |
| ✔ Color codes a text based on type of writing issue | ✔ Reorganizes paragraphs as needed to improve the flow of ideas and overall readability of a text | |
| ✔ Provides suggestions for complex words and phrases | ✔ Leaves feedback with suggestions or explanations for edits | |
| ✔ Pinpoints exact words, phrases, and sentences that need revision based on a readability scale | ✔ Ensures coherence and concision throughout the text | |
| ✔ Gives users a readability grade with a suggested grade for all texts | ✔ Provides both Clean and Track Changes copies of the document |
Cons
| Hemingway Editor | Human Editor | |
|---|---|---|
| ✘ Does not highlight all writing mistakes, such as grammar, spelling, or punctuation | ✘ Costs money | |
| ✘ Does not provide suggestions for sentences highlighted as hard or very hard to read | ✘ Longer turnaround time | |
| ✘ Does not take into consideration other common writing mistakes, such as clarity and flow | ||
| ✘ Makes incorrect suggestions at times because it cannot detect context | ||
| ✘ User needs decent editing and proofreading skills to make changes | ||
| ✘ Readability scale and grade are not 100% clear or consistent | ||
| ✘ No Track Changes copy of a text |
How the Hemingway Editor Works
Hemingway Editor is an AI writing assistant that highlights words, phrases, and sentences in a text, showing you what areas need improvement. It looks at five common writing errors:
- Adverbs (highlighted in blue)
- Passive voice (highlighted in green)
- Complex words with simpler alternatives (highlighted in purple)
- Sentences that are hard to read (highlighted in yellow)
- Sentences that are very hard to read (highlighted in red)
Each passage is given a “readability grade,” which ranges from 5 to 15. The readability grade indicates the reading level required to fully understand the given text. The Hemingway Editor suggests that users aim to achieve a readability level of 9 for most texts.
The point of the readability grades is to point out words, sentences, or paragraphs that may be too lengthy or filled with jargon or that may be confusing or tedious for your readers – all of which are undesirable qualities for any text.
Writing Sample and Methodology
The writing sample used for this experiment was a passage from a post-doctorate student’s thesis (used with permission). The author’s first language is not English, but they have studied English at a high level throughout their higher education. The writing sample was 605 words in length.
We submitted the sample to an experienced editor at Proofed to be proofread and edited with the aim to remove any language mistakes and improve its clarity, concision, and coherence.
The same sample was put through Hemingway Editor to highlight words, sentences, and paragraphs that needed attention.
The results were assessed based on whether the writing quality was improved and writing mistakes were corrected.
Hemingway Editor Results
After copying and pasting the writing sample into the Hemingway Editor, it immediately received a readability grade of 13, which is 4 grades above the recommended grade of 9.
As shown in the image, here’s the readability grade breakdown:
- 9 adverbs. Aim for 5 or fewer.
- 9 uses of passive voice. Cut to 7 or fewer.
- 5 phrases have simpler alternatives.
- 5 of 33 sentences are hard to read.
- 14 of 33 sentences are very hard to read.
Incorrect Suggestions and Other Mistakes
The first mistake Hemingway Editor made is an incorrect suggestion for a “phrase with a simpler alternative,” (highlighted in purple). In this case, the highlighted word is part of a title being cited in the text, which Hemingway Editor was unable to discern.
Although Hemingway Editor did provide suggestions for alternative words, which you can see in the image below, it was not appropriate for the context.
Other mistakes and errors it didn’t highlight include:
- Missing articles
- Missing or incorrect punctuation
- No suggestion for the definition of an abbreviation
- Abbreviations introduced incorrectly
- Incorrect verb tense
- Incorrect word order
- Incorrect word usage
Score After Human Intervention Based on Highlights
After making changes based on Hemingway Editor’s suggested number of adverbs and use of the passive voice (with no changes made to complex words or phrases because two of the highlights were incorrect suggestions) the grade remained the same: 13.
After making further changes, focusing on sentences that were identified as hard or very hard to read, Hemingway Editor still gave the writing sample a readability grade of 13. And this time, the individual targets changed.
After the first revision, Hemingway Editor indicated that a total of five adverbs was okay but that changed to four or fewer, and the target of seven uses of passive voice was changed to six or fewer.
Although one sentence previously highlighted as “very hard to read” was changed to “hard to read,” another previously unhighlighted sentence was highlighted as “hard to read.”
There was no explanation on the site for these changes.
Hemingway Editor Pros and Cons
Overall, Hemingway Editor did a good job of highlighting problem sentences that some readers might have difficulty following. However, it didn’t highlight many grammar, spelling, and punctuation mistakes in the text that are typical of someone who speaks English as an additional language. Furthermore, it didn’t provide any suggestions for changes to the problem sentences.
Essentially, Hemingway Editor didn’t improve the writing sample in any way but merely highlighted sentences and areas that could be difficult to read for some people.
| Pros | Cons | |
|---|---|---|
| ✔ Easy to use at home | ✘ Does not highlight all writing mistakes, such as grammar, spelling, or punctuation | |
| ✔ Free | ✘ Does not provide suggestions for sentences highlighted as hard or very hard to read | |
| ✔ Color coded the text based on type of writing issue | ✘ Does not take into consideration other common writing mistakes, such as clarity and flow | |
| ✔ Provided suggestions for complex words and phrases | ✘ Makes incorrect suggestions at times because it cannot detect contex | |
| ✔ Pinpointed words, phrases, and sentences that needed revision based on its readability scale | ✘ User must have decent editing and proofreading skills to make changes | |
| ✔ Gave a readability grade with a suggested grade for each element | ✘ Readability scale and grade is not 100% clear or consistent while making edits to a text | |
| ✘ No Track Changes copy of a text |
Human Editor Results
The same writing sample was submitted to an editor at Proofed for editing and proofreading services. After about a day, the writing sample was returned with a Clean copy and a Track Changes copy, showing what changes were made along with comments on areas that were unclear and needed further attention, or explaining why an edit was made.
The editor at Proofed not only eliminated the mistakes that Hemingway Editor missed, but they also improved the overall quality and readability of the text by fixing long, complex sentences and reorganizing and breaking up long paragraphs.
They also left comments on areas that needed further attention from the writer, such as word choice ambiguity, and reduced the writing sample word count from 602 to 536.
Let’s take a closer look:
In this section, the editor at Proofed was able to edit and rewrite sentences as needed to improve the clarity and flow of the paragraph. While Hemingway Editor highlighted the same sentences in this section as “hard to read” or “very hard to read,” the editor at Proofed actually made the necessary changes.
Human Editor Pros and Cons
The editor at Proofed corrected all mistakes in the writing sample. They focused on clarity, cohesion, and concision in the text to make sure the main ideas and points were clear and easy to understand. Additionally, the editor ensured that each paragraph flowed in a smooth manner and made changes to accomplish that goal.
| Pros | Cons | |
|---|---|---|
| ✔ Fixed all writing issues, such as grammar, spelling, and punctuation mistakes | ✘ Cost money | |
| ✔ Checked each sentence for clarity, meaning, and flow and made changes as needed | ✘ Took about a day | |
| ✔ Reorganized paragraphs as needed to improve the flow of ideas and overall readability of the text | ||
| ✔ Left feedback with suggestions or explanations for edits | ||
| ✔ Ensured coherence and concision throughout the text | ||
| ✔ Provided both a clean and a Track Changes copy of the document |
Conclusions
Based upon this test, AI writing assistants such as Hemingway Editor, which promise to help you proofread and edit your writing at home, fail to follow through on quality and accuracy in their edits.
The biggest issue we found with Hemingway Editor is that it did not provide suggestions for all the problem areas it highlighted. Additionally, the readability grade lacked consistency, changing grades and requirements without providing any reasons.
The editor at Proofed was able to make edits to the writing sample accurately and consistently, leaving comments with feedback or suggestions for further action from the writer. The editor not only removed writing mistakes, such as grammar, punctuation, and spelling issues, but improved the flow, coherence, and clarity of the writing sample by adding a bulleted list, rewriting problem sentences, and reorganizing paragraphs.
Tired of AI writing assistants failing to provide consistent and reliable editing and proofreading for your business? Schedule a call with us today to learn more about how our experienced, qualified editors can help.
