Skip to content

Teaching and learning

Designing assessment when AI detection is unreliable

Current evidence offers little support for relying on detection tools. A more productive question may be what an assessment task actually provides evidence of, and how it can be designed accordingly.

Published 11 August 2026Updated 5 October 2026Reading time 10 min
ELT Pub logo
Admin
ELT Pub editorial team

1. Introduction

Much of the early discussion of generative AI in higher education focused on detection. This focus was understandable, but there are now good reasons to doubt that detection can bear the weight placed on it. This article therefore considers the question that follows: how might assessment be designed when the authorship of a submitted text cannot be reliably verified?

2. Limitations of detection

Three problems are considered here. The third is arguably decisive.

Accuracy is limited in ways that matter in practice. Detectors report probabilistic scores rather than findings of fact. Accuracy figures reported by vendors are typically derived from controlled comparisons that may not resemble real submissions, in which students edit, paraphrase, combine their own writing with generated text, or use tools intended to evade detection. In an independent evaluation of fourteen tools, Weber-Wulff et al. (2023) concluded that available detectors were neither accurate nor reliable. OpenAI withdrew its own classifier in July 2023, citing its low rate of accuracy (OpenAI, 2023).

False positives do not appear to be evenly distributed. Liang et al. (2023) found that several widely used detectors misclassified a majority of essays written by non-native English writers as AI-generated. A plausible explanation is that detectors sensitive to limited lexical variety and predictable syntax are more likely to flag carefully constructed writing produced in an additional language. Similar concerns have been raised about students who write in formulaic registers, including those taught to them by their own institutions.

For those teaching in international programmes, this concern alone may be sufficient reason for caution. A tool whose errors concentrate among students who are already more vulnerable to integrity allegations cannot be regarded as neutral, whatever its aggregate accuracy.

The procedural costs are high. Suppose, for illustration, that a detector were 98 per cent accurate, which would exceed what current evidence supports. Across 400 submissions, this would still produce approximately eight false accusations. Each would involve a formal process, considerable distress for the student, and a defence that is difficult to construct, since proving authorship of a text is often close to impossible. Such cases may also damage the trust on which much teaching depends.

The asymmetry of errors. An undetected instance of AI use may result in one inflated grade. A false accusation can affect a student's academic standing, in some cases their visa status, and their trust in the institution. Because these errors are not comparable in their consequences, policies that treat them as equivalent are likely to be poorly calibrated.

3. A more productive question

If the authorship of a submitted artefact cannot be verified, one response is to redesign assessment so that authorship of the artefact is no longer the sole object of assessment.

This is less radical than it may appear. It returns to a question that sound assessment design has long asked: what is this task evidence of? A take-home essay was arguably never clear evidence of unaided writing, given the availability of proofreaders, writing centres, study groups, and commercial essay services. Generative AI did not create this problem so much as remove the friction that had allowed a weak assumption to persist.

4. Five redesign strategies that do not rely on detection

4.1 Assess the process as well as the product

Evidence can be collected at intervals: a proposal, an annotated bibliography with the student's own notes, a draft, and a revision memo explaining what changed and why. Both the trajectory and the final product can then be assessed.

This approach may be effective because generative AI is considerably better at producing a plausible finished artefact than a plausible history of one, including false starts, abandoned framings, and sources that proved less useful than expected. It may also improve teaching, since feedback can be given at the draft stage rather than on a finished piece that will not be revised.

Cost: additional marking. This can be reduced by keeping intermediate stages low-stakes, lightly commented, or peer-reviewed.

4.2 Anchor tasks in material a model cannot access

Tasks can require engagement with material available only in the local teaching context, for example:

A model may write competently about a general topic, but it has no access to what a particular class concluded in a given session or what occurred in a student's classroom. Requirements of this kind may also tend to produce more specific and better-grounded writing, independently of the technology.

4.3 Make AI use explicit and assessable

Rather than prohibiting tools that many students are likely to use, instructors can require students to document their use and assess the quality of that use. A short appendix might record what was asked, what the tool produced, what was retained, and what was rejected and why.

This reframes an integrity problem as a learning objective. Evaluating model output, including identifying confident errors, missing counter-arguments, and fabricated citations, is an increasingly necessary skill and can be assessed directly. Students who are unable to critique the output will tend to reveal this, which provides useful diagnostic information.

This approach also reduces the incentive to conceal use. Concealment is arguably what makes the current situation most corrosive, and students who are required to document their use have less reason to hide it.

4.4 Add an oral component

A brief conversation about a submitted piece of work can go a long way towards resolving questions of authorship. It need not resemble an interrogation; a short viva might ask why the work was framed in a particular way, what was discarded, which source changed the student's view, and what they would do differently.

Students who wrote a piece are generally able to discuss the decisions behind it, whereas those who commissioned it, from a person or a model, often find this difficult. An oral component also assesses capacities that a written artefact does not capture.

Cost: time, which increases with cohort size. Options include sampling rather than examining every student, using small groups, observing structured peer discussions, or reserving oral components for the highest-weighted assessment.

4.5 Use supervised conditions where they are genuinely required

Some claims, such as professional certification, threshold competence, or licensure, require conditions in which the work is unambiguously the student's own. Invigilated assessment remains appropriate for these purposes, but it is best used deliberately rather than by default.

Applying it universally carries costs. Timed, handwritten examinations primarily assess recall and performance under pressure, and provide limited evidence of the extended, revised, source-based reasoning that much of a degree aims to develop. A wholesale return to closed-book examinations in response to AI may therefore sacrifice considerable validity for a modest gain in certainty.

5. Communicating policy

Whatever approach is adopted, it is advisable to state expectations at the level of each task rather than only in a programme handbook. Students frequently report that rules on AI use are unclear, and inconsistency between modules appears to contribute substantially to this. Three sentences on the assignment brief may suffice:

For this assessment you may use generative AI for [specified purposes]. You may not use it for [specified purposes]. If you use it, include the documentation appendix described in the rubric.

Specific statements are more useful than general ones. "Use AI responsibly" offers little practical guidance, whereas "You may use AI to check grammar and to generate practice questions; you may not use it to draft the analysis section" is clear and actionable.

6. The costs of redesign

Each of the strategies above requires more staff time than a single end-of-term essay marked once. This is a genuine constraint in a sector under considerable pressure, and it should be acknowledged rather than dismissed with an appeal to good practice.

Two implications follow. First, redesign can be selective: focusing on the one or two most heavily weighted assessments in a module may be more feasible than changing all of them. Second, it may be necessary to make clear to colleagues and administrators that assessment security under current conditions carries a cost, and that policies requiring integrity without corresponding resources effectively transfer that cost to individual staff.

7. Conclusion

Much current concern rests on an assumption that merits examination: that a student who uses a model to produce an essay has avoided learning that the essay would otherwise have generated.

In some cases, this assumption is likely to be correct. It is also worth considering, however, how much learning some essays generated before generative AI, particularly those written hastily from a few sources, on questions in which the student had little investment, and returned with feedback after the module had ended.

Generative AI has made such assessments difficult to sustain, although their educational value may already have been limited. The redesigns outlined above can be justified as better assessment on their own terms, being more diagnostic, more specific, and more closely aligned with the reasoning that programmes aim to develop, and they are also more robust to a technology that seems likely to remain. This appears to be a more achievable goal than prevailing in an escalating contest with detection tools.

References

Found an error in this article? Corrections are made openly and are welcome — see the editorial policy or write to ngoconglem@gmail.com.