Section: Technology & AI Author: V Reading Length: ~27 min Sources and further reading: 22 items Topics: AI, education, academic integrity, text detection, accountability SEO / Working Title: How do you recognise text written by artificial intelligence?
The question is: how do you recognise text written by artificial intelligence? It sounds simple only until we notice that we are not comparing one thing. A statistical score is not origin. A watermark is not a detector of arbitrary chatbot output. A low FPR in the laboratory is not a low FPR for a non-native writer. Turnitin Similarity is not Turnitin AI writing. Only when these four things are kept apart can we read what classifiers can do as of September 2026, what shared tests say about them, and why the public policies of Charles University and CTU do not treat a detector as proof of origin.
1. The school, the vendor and the laboratory are not measuring the same thing
Imagine a committee at a university. A teacher brings a text that seems too smooth. The student says they worked alone and only had the language polished. The academic office wants a clear rule, because cases are rising and every decision must survive an appeal. The vendor shows a percentage on a dashboard. The laboratory shows an accuracy table on a shared set. Charles University writes in a public recommendation that no technically reliable way to detect generative AI exists [12]. CTU, in a 2023 methodological instruction, forbids the use of AI tools to detect student work because of false positives and false negatives [13].
At first glance this is a dispute about whether “we can spot AI”. In fact each party is describing a different object. The vendor measures a flag according to its own model and its own threshold. The laboratory measures performance on a particular set of texts, in a particular language, against particular attacks and with a particular calibration. The university decides whether a number may enter a sanction against a particular person. The student does not bear the consequences of the tool’s average accuracy. They bear the consequences of a conclusion about one text, in one proceeding, with one chance to defend themselves.
That is why everyone can be partly right and the shared decision still wrong. Turnitin is right that its tool returns a numerical score and that in May 2023 it processed a large operational volume of submissions [10]. Researchers are right that detectors can be measured and that some signals carry information. Charles University and CTU are right that a disciplinary verdict is not a laboratory graph. The moment a dashboard percentage is translated into the sentence “the student cheated”, the object of measurement has changed. And that is where the most expensive error is born.
The detector measures a statistic. It does not decide who wrote the text.
— Jiný Kontext
2. One score has at least five layers
One number at the end of an analysis looks simple. It is produced, however, in a chain of layers. The first is the signal. It may be perplexity, a classifier, a watermark, a comparison with stored output, or metadata. Each signal answers a different question and each fails in a different way. Perplexity punishes language that is unusual or, conversely, too predictable. A classifier learns a boundary between sets. A watermark looks for a statistical trace that the generator had to insert in advance. Metadata help only when they exist and have not been stripped from the text.
The second layer is the threshold. A tool can be useful at one setting and dangerous at another. A school does not only need to know that a classifier has a handsome graph. It needs to know the false alarm for an individual, because punishment falls on a person, not on an average. RAID therefore calibrates the true positive rate at a false positive rate of 5%. That is more precise than an unchecked default slider, but even then it is no licence for a disciplinary automaton [6].
The third layer is the population of texts. An English essay is not a Czech seminar paper. Code is not the introduction to a thesis. The text of a native writer is not the text of someone writing in a second language. A translated paragraph is not evidence that the author changed. A change of domain can change the ranking of tools, because the model is not recognising an author. It is recognising similarity to what it learned as human and as machine.
The fourth layer is revision. A text may be paraphrased, translated, edited or mixed with human work. This article does not describe how to evade school rules. It only describes the fact that revision changes the statistic. If the signal changes and authorship does not, the score cannot be a seal of origin.
The fifth layer is the decision. The same result may be an indicative prompt for a conversation, but it must not be standalone evidence of fraud. This layering is not a detail for methodologists. It determines whether the number belongs in an educational conversation, in a revision of the rules, or in a proceeding that can harm a student.
3. A similarity statistic is not a seal of origin
The classifier did not see a person at the keyboard. It does not know whether the student wrote the first draft, used a model for an outline, had the English corrected, translated the text back and forth, or took a finished output. It sees a text and compares its properties with what, in its set, it treats as human and as machine. That can produce suspicion. It cannot, by itself, produce authorship.
This is not a play on words. The origin of a text is a historical question. Who did what, when, to what extent and with what degree of independence. A statistical score is a question of distribution. How much the text resembles one class against another. There is a bridge between these questions, but it is not automatic. The bridge is knowledge of the process, the assignment, interim versions, consultations, a declaration, a defence, and the chance to explain the work.
DetectGPT is a useful example. It rests on the idea that machine-generated text can have a different probability curvature from human text [15]. That is an elegant statistical hypothesis. It is not a record of a file’s origin. As soon as the model, the language, the domain or the revision of the text changes, the properties of the signal change too. Sometimes it helps. Sometimes it breaks. The conclusion “this resembles model output” is not the same as “this person used AI improperly”.
A school therefore needs to distinguish three sentences. First: the tool flagged the text. Second: the text has features that warrant a conversation. Third: the student broke the rules. Software can say the first sentence. The second requires a teacher. The third requires a process.
4. A watermark is not a detector of an arbitrary chatbot
A watermark in text sounds like a solution. The generator would, while writing, choose some words with a small statistical shift, and the detector would later check whether the shift remained in the text. Kirchenbauer and colleagues described a scheme that, according to the 2023 paper, allows detection from a short span, reported in the laboratory setting as “as few as” 25 tokens [3]. That is an important research result. It is not a universal detector of text from the internet.
A watermark works only when the generator actually used the particular scheme, when the detector knows the required configuration, and when the text has not passed through a change that breaks the trace. If someone feeds a detector ordinary output from a public chatbot for which no publicly verifiable watermark has been deployed and published for checking, they are not looking for a seal. They are looking for statistical similarity. That is a different tool.
As of September 2026, public materials do not support the conclusion that the main consumer chat services give an ordinary user a verifiable text watermark of this scheme. After withdrawing its classifier, OpenAI talks about origin in other modalities and about broader questions of provenance, but it has not returned a public, reliable text classifier that would by itself settle the origin of a seminar paper [1].
It does not follow that watermarks have no value. It follows that their value is different. They can be useful in a closed system that controls both generation and verification. In a public and mixed environment, where text travels through editors, translations, copying and human revision, a watermark does not become a court stamp.
5. A low FPR in the laboratory is not a low FPR for a non-native writer
A false positive rate looks like a technical figure. In a school, however, it is the probability that an innocent text will be marked as suspicious. And that probability is not a property of the tool by itself. It is a property of the tool on a particular population of texts.
Liang and colleagues in 2023 examined 91 human TOEFL essays and 88 essays by eighth-grade pupils in the United States. Across seven detectors, including GPTZero, they measured an average false positive rate of 61.22% on the TOEFL essays. Eighteen of the 91 TOEFL essays, or 19.78%, were flagged by all seven detectors. At least one detector flagged 89 of the 91 essays, or 97.80% [8]. That is a strong number, but it has precise limits. It measures human English TOEFL essays by non-native writers, against tools as they stood on 15 March 2023. It does not measure Czech seminar papers in 2026.
It is important even so. It shows that the population of writers is part of the measurement. A narrower vocabulary, a simpler sentence structure or a learned school style can look “model-like” without a model having written the text. To punish that signal is to punish style, language background or genre. That is exactly the kind of error an institution must not hide behind the word objectivity.
It does not follow that every detector today makes the same error to the same degree. It follows that it is not enough to take an FPR from an English or a vendor test and apply it to Czech students. Anyone who wants to use the number in a proceeding must show its validity for the given language, genre, length and population. Without that, a low FPR in a presentation is only a distant figure.
6. Turnitin Similarity is not Turnitin AI writing
Teachers know the check for similarity with sources. That has its own problem, but at least it measures another and intelligible thing: whether a text matches another text. Turnitin Similarity is not the same as Turnitin AI writing. A match with a source may show copying, a poor citation or a common phrase. An AI writing score asks whether part of the text looks machine-written. Mixing these results in a single sanction is an error of object.
The example is simple. A student may cite a source honestly and still have high similarity, because the bibliography, statutory wording or the assignment itself is the same. Another student may paraphrase so that similarity falls, while authorship remains a problem. A third may write alone, but in a genre that is formal, repetitive and school-polished. Each case needs a different question.
Turnitin itself, in its materials, distinguishes thresholds and limits. For the AI writing model it states a minimum length of 300 words and describes how it treats a low range of marking [11]. In an operational text from 2023 it speaks of a document-level false positive rate of less than 1% for documents above 20% AI writing on an internal test of 800,000 academic texts from before ChatGPT, and of a sentence-level false positive rate of around 4% [10]. These are vendor numbers, with a defined threshold and the vendor’s own set. They are not the evidentiary rules of a Czech school.
The right administrative sentence is therefore not “Turnitin showed a percentage, the case is clear”. It is: “The tool flagged the text in one task; we must establish what it measured, whether the length, language and threshold fit, and whether there is independent procedural support.” That is slower. But a school that punishes faster than it distinguishes is not punishing accuracy. It is punishing convenience.
7. Fourteen tools on fifty-four documents are not an operational guarantee
Weber-Wulff and colleagues tested, in the winter and spring of 2023, a total of 14 detectors, of which 12 were public, plus Turnitin and PlagiarismCheck. The set had 9 documents in 6 classes, hence 54 documents, in English. In a strict binary cut, no tool reached more than 80% accuracy; the best, Turnitin, had 76%. In the semi-binary cut it had 79%, and in the logarithmic scoring 81% [5].
This study has limits. The set is small. The language is English. The tools have changed since 2023. It is not a verdict that a particular version of a particular tool in 2026 can do nothing useful. It is, however, a readable demonstration of something else: how easily a detector behaves differently by class of text and by method of scoring.
A small shared method has the advantage of showing the mechanics. Accuracy is not one thing. A binary result punishes a different failure from a softer classification. Translated texts, human-edited texts and mixed texts are not mere noise. They are precisely the situations in which schools live. A student need not submit clean model output. They may have their own draft, language editing, a translation, a consultation and citations. It is in these in-between states that a disciplinary decision is hardest.
That is why fourteen tools on fifty-four documents are not an operational guarantee. They are a warning against translating a laboratory table into a proceeding about one person. Anyone who wants to use a detector responsibly must know what error looks like on their own texts. Not on a text from an advertisement.
8. RAID shows a drop when sampling changes. It is not a Czech thesis
RAID is a larger and harder test. Dugan and colleagues in ACL 2024 described a shared benchmark for the robust evaluation of machine-generated text detectors. The dataset contains more than 6 million generations, 11 models, 8 domains, 11 adversarial attacks and 4 decoding strategies; it evaluates 8 open and 4 closed detectors [6]. The authors track the true positive rate at a false positive rate of 5%. That matters, because without calibration it is possible to look good on a metric that a school cannot accept.
RAID also shows fragility. A repetition penalty, according to the paper, reduces accuracy by as much as 32 percentage points [6]. At naive thresholds a false positive rate of up to 1.7% appears for closed-source tools; open metrics are often substantially higher [6]. These numbers have precise limits. They belong to the given benchmark, models, domains and settings. They are not a direct measurement of a Czech diploma thesis.
That is why they are useful. They do not say: “never use any detector”. They say: “a change of sampling, domain and revision changes the result enough that a default threshold is not proof.” RAID-extra also contains Czech and German news and code [7]. That shows that a Czech-language stress test exists in another domain. It does not mean that anyone has validated disciplinary use on Czech seminar papers and final theses.
A public benchmark is a map. It is not a judgment. It helps to see which signals deserve attention, where tools fall and how they behave when conditions change. A decision about a student must stand somewhere else.
9. A withdrawn classifier is evidence of a limit. It is not evidence that a new tool has none
OpenAI published an AI Text Classifier in January 2023. In July of the same year it withdrew the tool for low accuracy [1]. On an English challenge set it reported that the tool correctly labelled 26% of AI texts as “likely AI-written” and falsely labelled 9% of human texts [1]. The threshold was set carefully so that there would not be too many false positives. The price was clear: the tool missed many AI texts.
This example is often used too quickly. One side turns it into proof that detection as a field has no value. The other says that the old tool is already the past and need not be discussed. Neither sentence is enough. It is an audit of one tool from one company, one English set and one period. It is not a measurement of Czech, of code, of short texts or of mixed authorship.
At the same time it is evidence of a limit that cannot be swept away. When even a large provider withdraws a public classifier because it is not accurate enough, a school cannot claim, without further support, that a different percentage from a different service is enough for guilt. A new tool may be better. It must, however, show that on the relevant population and at a threshold that matches the consequences.
The useful question is not whether detectors are improving. They certainly change. It is whether their error remains acceptable for a particular decision. For an indicative warning the line may be different from a disciplinary proceeding. For a conversation with a student it may be different from cancelling a piece of work. The word “suspicion” and the word “proof” must not share the same slider.
10. Paraphrase breaks the statistic. This text is not a school how-to
Sadasivan and colleagues, in Can AI-Generated Text be Reliably Detected?, show that paraphrasing can reduce the performance of detectors and of a watermark [2]. On watermarked output in an experiment on about 100 XSum passages, detector accuracy after a PEGASUS paraphrase fell from 97% to 80% and perplexity rose by 3.5 [2]. For DetectGPT on 200 XSum passages, generated by GPT-2 Medium and paraphrased by a T5 model, AUROC fell from 96.5% to 59.8% [2]. Under recursive DIPPER paraphrasing the dossier reports a drop from 82% to 18% AUROC [2][16].
These numbers are not a recipe. They do not tell a student what to do. They do not tell a school that every revised text is suspicious. They say that a statistical signal can be changed without a simple change in historical origin. A text may pass through a machine, a person, a translation and an edit. Authorship does not become binary merely because a tool returns a binary flag.
An AUROC of 50% corresponds to chance. When the result after revision approaches random discrimination or falls below it, it is wrong to rest a sanction on the idea that the score holds stably. It is worse still to think that a low score automatically clears and a high score automatically convicts. Both errors have the same root: they mistake statistical similarity for origin.
That is why paraphrase in an article about detection should be mentioned with care. Not as a guide to evasion. As a test of how far a claim can be pushed. If a small change of wording changes the result substantially, the result is not a seal. It is a signal with a limit.
11. A declaration of use is not detection
Some schools respond to generative AI with a mandatory declaration. That is a different thing from detection. A declaration does not say that we can reconstruct every use of a model after the fact. It says that the author must describe their work and bear responsibility for how the tool was used.
The Faculty of Humanities of Charles University, in Dean’s Measure No. 3/2026, introduces a mandatory declaration on the use of generative AI in a final thesis [14]. CTU, in its materials, works with a declaration on artificial intelligence in final theses [20][21]. This is not a black box. It is a procedural element. It forces the author to distinguish research, an outline, language editing, the generation of text, and their own responsibility.
The AI Act in Article 50 and the related European Commission FAQ address transparency duties for certain kinds of outputs and texts in the public interest [22]. They do not thereby settle the origin of every seminar paper. This is not a university detector. It is a legal frame for labelling and transparency in a different decision space.
A declaration can therefore be reasonable. It is not a substitute for proof. It helps to set rules, to improve the conversation between supervisor and student, and to say in advance what is allowed. If a breach is handled after the fact, it is not enough to compare the declaration with a tool’s percentage. The work must be read, the process followed, and questions asked that the author of the work should be able to defend.
12. Charles University does not take a detector as proof. CTU forbids it for detection
Public Czech policies are more sober on this point than many debates. Charles University, in its recommendation on final theses, states that no technically reliable way to detect generative AI exists [12]. CTU, in Methodological Instruction No. 5/2023, states that AI tools should not be used to detect student solutions, precisely because of false positives and false negatives [13]. Later CTU materials work more with a declaration of AI use than with promoting a detector to the rank of proof [21].
This does not mean that schools are giving up on academic integrity. It means that they are not looking for it in one black box. A supervisor may know the development of the text, the consultations, the literature, the formulation of the problem and the student’s ability to defend the method. A committee may examine whether the student understands their own claims. Rules may distinguish permitted language help from an unauthorised taking of output.
The National Library of Technology, in the context of AI tools for research, notes that detectors are indicative [19]. That is the precise word. Indicative does not mean useless. It means that the result points toward a further question. It does not replace the answer.
In a disciplinary setting this restraint matters in human terms as well. A false alarm is not only an error in a table. It is an accusation of dishonesty against a person. And such an accusation needs more than a similarity statistic.
13. A flag above twenty percent is not a share of fraud
Turnitin stated in May 2023 that in the 7 weeks after the detector preview it processed 38.5 million submissions. Of these, 9.6% had a score above 20% AI writing and 3.5% had a score of 80 to 100% AI writing [10]. These numbers often tempt a simple sentence: that many students cheat. That would be a bad translation.
The unit measured is submissions in the Turnitin system, including assignments where AI may have been allowed or where use of a tool was meant to be declared. A score above 20% is a flag according to the tool, not a conclusion that rules were broken. A document with a high score may be a case for checking. It is not, by itself, a case file.
The vendor also works with its own internal test of 800,000 academic texts from before ChatGPT and, for documents above 20% AI writing, reports a document-level false positive rate of less than 1% [10]. This number too has a boundary. It holds for the vendor’s method, a particular threshold and a particular set. At low values Turnitin itself qualified the results, and the documentation states limits for shorter texts [11].
A simple rule follows: a flag is not a share of fraud. It is an administrative signal. If a school uses it for a conversation, it must keep it apart from guilt. If it uses it for a sanction, it must have further support. Otherwise orientation becomes a verdict.
14. The best decision is the one whose false alarm you can bear
Of a tool we often ask how often it catches AI text. Of an institution it is equally important how often it harms human text. The scheme below is not a ranking of tools. It is a map of errors that a school must name in advance.
| Type of failure | What it looks like | How to test | What limits the harm |
|---|---|---|---|
| Score as origin | GPTZero or Turnitin is translated into the sentence “guilty” | Two detectors on a known human text; watch for disagreement | Detector outside punishment, a declaration and a defence |
| Laboratory FPR on another population | A TOEFL essay is marked as AI | Liang: the writer’s population is part of the measurement [8] | Do not read an English FPR as a Czech one; protect non-native writers |
| Similarity as AI writing | A match with a source is read as machine authorship | Separate the tools, the outputs and the conclusions | Two columns, two questions, two rules |
| Default threshold | “99% accuracy” without knowing the threshold and the set | RAID: TPR at FPR 5% [6] | Calibration and a ban on automatic punishment |
| A short text as a thesis | A paragraph is scored in the same way as a long paper | Check the 300-word minimum at Turnitin [11] | Do not judge below the tool’s threshold |
Illustrative scheme: The detector stands in the first column, the population of texts in the second and the school’s decision in the third. The arrow from the first column to the third does not run directly. It runs through verification, a defence and the rules.
The best decision is therefore not the one that closes a case as fast as possible. It is the decision whose error can be explained, corrected and borne in human terms. For low-risk use a detector may help to rank texts for manual review. For a sanction the threshold must be stricter, because a false alarm is not only a technical loss. It is a blow to reputation.
15. A text whose origin you know will say more than a vendor’s percentage
A practical test need not be complicated. It is not meant to prove that someone cheated. It is meant to show whether the institution understands its own tool. Take a text whose origin you know: a paragraph of your own written before ChatGPT arrived, an archived text by a teacher, an old student paper with a clear date. Run it through two detectors. If the results differ, that is not a brand dispute. It is a warning that a score is not origin.
How to test a detector on a text whose origin you know
The same known human text can be sent through translation and back. Weber-Wulff and colleagues had, in their set, a class of machine-translated texts [5]. Translation changes the statistic, but it does not change the person who wrote the original text. If a flag appears after translation, the school sees the difference between a change of surface and a change of authorship.
The third test is length. Both OpenAI and Turnitin tied their limits to reliability on a text that is long enough; Turnitin states a minimum of 300 words [1][11]. A short paragraph, an abstract or an answer in a test cannot be read as a diploma thesis. It is a different unit of measurement.
Three tests before a sanction
The first test is disagreement between two tools on a known human text. The second is a change of score after translating a known human text. The third is length below the tool’s threshold. None of the three says who cheated. All three say whether the score can carry the weight the institution is placing on it.
Before disciplinary use
Before a sanction must come questions that are not only about the tool. Are there interim versions? Can the student explain the literature? Does the style fit previous work, without turning that into a pseudoscience? Was the use of AI in the assignment allowed, limited or forbidden? Was the rule clear before submission? If the answer to these questions is missing, a percentage will not supply it.
Disagreement between two tools on a known text is not a brand dispute. It is evidence that a score is not origin.
— Jiný Kontext
16. In Czech, do not extrapolate an English FPR
Czech is not merely English with diacritics. It has different inflection, word order, phrases, genre habits and school style. If a detector is trained and tested mainly on English texts, its numbers cannot simply be transferred to a Czech seminar paper. RAID-extra shows that a Czech news domain exists in the extended test [7]. That is useful. It is not a validation of Czech university theses.
Liang’s study measures non-native English [8]. Weber-Wulff measures English documents in a small set [5]. The OpenAI Classifier measured an English challenge set [1]. Turnitin reports its own operational and internal test numbers [10][11]. Each of these sources has value. None of them says: “Here is a reliable false positive rate for Czech diploma theses at Charles University and CTU in 2025 to 2026.”
What would weaken the main claim? A public, pre-registered test on Czech seminar and diploma texts with a calibrated false positive rate of at most 1% and a stated true positive rate at the same time. A test that survives translation and ordinary human editing. A deployed, publicly verifiable watermark on the main chatbots. Or a new policy of Czech universities that would expressly promote the detector to the rank of proof. As of the date of the search, the dossier has no such support.
Until it exists, it is more precise to say less. A detector may be an indicative aid. It is not the author of the text. It is not a witness. It is not an evidentiary automaton.
17. The hidden cost is a false alarm that the punished person cannot bear
Public debate often counts the harm of a missed AI text. A student cheated and the school did not catch them. That harm exists. Academic integrity is not decoration. The second harm is less visible: the false alarm. The student wrote the text themselves, or used a tool in a permitted way, and the system marked them as suspicious. The teacher begins to look differently. A committee wants an explanation. Reputation is damaged before the error is repaired.
That is the hidden bill of a convenient percentage. Software returns a number in a second. The institution then has to carry the consequence for weeks or months. If it has no process that catches a false alarm, the detector is not cheap. It only shifts the cost onto the person with less power.
It does not follow that schools should give up. It follows that they should name more precisely what they are punishing. What may be forbidden is an undeclared taking of output, a failure to follow rules for citing a tool, an inability to defend the work, or a misleading declaration. All of these are questions that can be checked by people and by process. A detector may give a prompt. It does not have the last word.
The question is therefore not how to recognise text written by artificial intelligence with one number. It is: what, from a similarity statistic, may you trust, for which population of texts, at which threshold, and which error, for this person, can you bear? If the answer is “we do not know”, that is uncomfortable. But it is more precise than a certainty the tool never measured.
Related texts in this series
- Deepfake: How do you recognise a fake video or photograph? — an image detector has the same problem as a text detector: the laboratory is not operations.
- Is a photograph proof? — the authenticity of a file is not the completeness of an event.