AI detectors promise to identify machine-written text. Universities use them for academic integrity; publishers use them for quality control. The marketing claims high accuracy. Our testing tells a more complicated story.
How We Tested
We ran 120 samples through five popular detectors: human-written essays, AI-generated text, AI text heavily edited by humans, and human text translated through AI tools. The results were sobering: false positive rates on human writing ranged from concerning to alarming, with non-native English writing flagged most often.
The False Positive Problem
The most troubling finding: detectors disproportionately flag writing by non-native English speakers, whose simpler, more formulaic prose patterns resemble AI output statistically. Real students face real accusations based on these tools. Several universities have quietly stopped using detectors for disciplinary decisions — a wise move the data supports.
What Detectors Can (and Cannot) Do
Detectors are decent at one narrow task: identifying unedited raw AI output. The moment a human edits meaningfully — restructuring, adding personal examples, changing rhythm — accuracy collapses. They are screening tools at best, not evidence. No detector output should ever be the sole basis for an accusation.
The Better Approach
For educators: design assessments AI cannot fake — in-class writing, oral defence, process documentation. For everyone else: stop treating detection as a technical problem with a technical fix. The question is not "was AI used?" but "does the work demonstrate understanding?" That question requires human judgment, and always will.
What to Do If You Are Falsely Flagged
If an AI detector flags your genuine work, do not panic — but do act methodically. First, request the full report, not just the verdict: which passages were flagged and with what confidence? Second, gather your process evidence: drafts, notes, revision history, browser history showing research. Process documentation is the strongest defence because it proves human authorship far better than any detector disproves it. Third, appeal in writing, calmly citing the documented false-positive rates of these tools — several academic studies now exist you can reference.
For institutions, the lesson is structural: no high-stakes decision should rest on detector output alone. The responsible policy combines process-based assessment (drafts, reflections, in-class components) with human review of any flagged work. Students can protect themselves preemptively by keeping version histories — Google Docs and Word both track revisions automatically. In a world of unreliable detection, your documented process is your proof of authorship.
Keep exploring: if you work with AI writing regularly, see our in-depth Claude review for the assistant writers trust most, and read what schools should actually teach about AI for a broader take on AI in education.
