Why AI text detectors cannot be reliable

02.08.2026  

Updated in August 2026. A new section explains Claude’s AI-content watermarking technology and how it differs from conventional AI detectors.

AI text detectors cannot reliably establish whether a person or a language model wrote something. They assess statistical similarity to known examples, but cannot see how a text was produced. Even a high ‘AI probability’ score is therefore only a reason to investigate further, not proof of authorship, plagiarism or misconduct.

This distinction matters in education, journalism and media literacy. Treating an automated assessment as a verdict can cast suspicion on honest pupils, lecturers, journalists and writers whose work is concise, structured, written in a second language or shaped by established professional conventions.

What an AI text detector actually measures

Early detectors often relied on the predictability of words and variation in a text’s rhythm. The assumption was that a model selected more predictable wording while people wrote less predictably. The problem is that plain, academic, technical or carefully edited human prose can also be highly predictable.

Modern systems generally use trained classifiers. They turn a text into a mathematical representation, compare it with training data and return a probability estimate. That estimate does not explain where a particular document came from. It shows only how closely the text resembles examples the system has encountered before.

Why a high score is not proof

1. Human and machine styles are converging

Language models learn from human writing, while people increasingly read, edit and imitate AI-assisted texts. The boundary between the two styles is becoming less clear. A formally correct introduction, symmetrical structure or neutral tone can come from either a person or a model.

2. Paraphrasing changes the result

Research shows that repeated paraphrasing can substantially reduce detector performance. Sadasivan and colleagues describe practical attacks on detection systems as well as theoretical limitations: as the distributions of human and machine texts grow more similar, the highest achievable classification accuracy falls. More recent experiments with adaptive paraphrasing also demonstrate that such attacks can transfer between detectors.

3. Language and style can introduce bias

Detectors depend on the composition of their training data. Authors writing in languages other than English, using standard professional terminology or deliberately simplifying their language may face more false positives. Applying a single threshold mechanically is particularly risky in Ukraine’s multilingual educational and media environments.

4. Models change faster than detectors

A classifier is trained on particular generators and model versions. A language-model update, changed settings or a new editing method can produce texts unlike the detector’s training samples. An excellent laboratory result does not guarantee the same performance on real material.

5. Real work often involves mixed authorship

A person may develop an idea and its arguments, ask AI to structure a draft, then check facts, rewrite paragraphs and add their own conclusions. Conversely, an original human text may undergo automated language correction. A binary ‘human or AI’ answer does not describe this process.

What OpenAI’s experience showed

OpenAI introduced its own AI text classifier in January 2023, then withdrew it in July because of low accuracy. According to the company, it correctly identified only 26% of AI-written texts in a test set as likely AI-written and incorrectly classified 9% of human texts as AI-written. OpenAI specifically warned against using the tool as a basis for decisions, noting its poor performance on short and non-English texts and the possibility of evasion through editing. The account remains in OpenAI’s official announcement.

How to assess writing fairly in education and the media

Rather than making an accusation on the basis of one number, examine the combined evidence and the working process:

  • Do not impose sanctions solely on the basis of a detector result.
  • Review drafts, version histories, notes, sources and the reasoning behind revisions.
  • Ask the author to explain the main arguments, choice of facts and route to the conclusion.
  • Distinguish AI use, plagiarism, factual error and deliberate manipulation: these are different issues.
  • Check claims against primary sources rather than assessing style alone.
  • Maintain a clear policy on permitted tools, disclosure and human responsibility.
  • Use a detector only as a weak supporting indicator requiring independent verification.

AUP’s experience: critical thinking matters more than an automated verdict

The Academy of Ukrainian Press consistently combines media literacy with practical study of artificial intelligence. At its Media Literacy 2.0: Using AI for Media Literacy and Education webinar, participants explored the tools’ capabilities alongside disinformation risks, ethics and responsible model use.

AUP’s article Artificial Intelligence Can Make Things Up – and That Is the Problem emphasises checking facts and sources and maintaining critical distance. Its webinar on quality journalism in the age of AI covers model hallucinations, manipulation, and legal and ethical issues.

This experience suggests a practical principle: AI can help structure material, make comparisons and explore options, but cannot replace human judgement. Likewise, a detector can draw attention to a text, but must not substitute for evidence, editorial scrutiny or a fair process.

Questions and answers

Can a detector prove that AI wrote a text?

No. It assesses similarity to known machine-generated texts and can make errors in either direction.

Does a high percentage indicate plagiarism?

No. Plagiarism checking looks for matches with sources; an AI detector analyses statistical properties of style. These are different checks.

How can work be assessed fairly?

Combine analysis of sources, the creation history, drafts, the author’s oral explanation and fact-checking. No single automated measure should be decisive.

Should detectors be abandoned altogether?

Not necessarily. They can serve as preliminary signals or research tools if users understand the error rate, disclose limitations and avoid categorical conclusions.

Conclusion

An ‘even better’ detector alone will not solve unreliable identification of AI-written texts. Education and the media need transparent rules, a culture of citation, scrutiny of working processes and critical thinking. Responsibility must remain with people, both when producing material and when assessing it.

Update: Claude begins watermarking generated content

After this article appeared, Anthropic announced a new AI-content watermarking technology. We have added this section to explain how it differs from conventional detectors.

While most services rely on AI text detectors, Anthropic has taken a different approach: marking content as it is produced.

In August 2026, the company said future Claude models would automatically produce an invisible, machine-readable watermark in text. This is neither hidden characters nor an added passage: a statistical pattern emerges through word selection during generation. Images and certain other supported files would use C2PA, a standard that adds digital provenance metadata. The measure is being introduced to meet EU AI Act requirements but will apply globally, not just within the EU. Anthropic explains the details in its official Claude watermark announcement.

The distinction between these approaches matters:

  • An AI detector tries to infer whether AI wrote a text by analysing style, vocabulary and statistical regularities.
  • A watermark is a specific statistical pattern created by the model during generation, which can be checked technically without analysing stylistic ‘signs of AI’.

Watermarking could therefore offer a more reliable way of identifying a particular model’s involvement than contemporary AI detectors, which often produce false positives.

Anthropic nevertheless stresses that this is not a universal solution. The watermark works only for content produced by Claude models supporting the feature. It does not determine whether a human or another model wrote a text, performs less well on short, factual and lightly edited passages, and can be weakened by substantial rewriting. C2PA metadata can also be lost when files undergo further processing.

For education, this suggests a new direction: rather than increasingly elaborate attempts to detect AI use, the market is moving towards tools that transparently mark content at the point of creation. If other major language-model developers adopt a similar approach, the discussion could shift from ‘Did AI write this?’ to ‘Was AI use honestly disclosed?’

Original article and sources. This is AUP’s independent analytical compilation, drawing on Hamza Boulahia’s Why We Can’t Have a Reliable AI Text Detector, published by Towards AI on 27 July 2026, and the primary sources linked above. The Ukrainian article is not a translation and does not reproduce the source article’s structure or wording.

The Ukrainian article was processed using AI tools for structure and language adaptation. Its final text was checked and edited by a human.

Translation note: This is a literary translation into British English, produced and proofread using a language model. Read the Ukrainian original.

"AUP-info" multimedia online media 
(identifier in the Register of Entities in the Media Sector: R40-00988).
envelopetagclockmagnifiercrossmenuchevron-uparrow-leftarrow-right