Traditionally, speaking assessment has meant a teacher in a room with a student, a rubric, and a clock. But that model has limitations of accuracy, reliability and efficiency which the Dynamic Speaking Test is designed to address. The table below sets out how the two approaches differ.
| Feature |
Teacher-led assessment
|
Dynamic Speaking Test
|
|---|---|---|
| Speed | 15–20 minutes per student, plus marking time | 100-student cohort tested and graded in 30 minutes |
| Consistency | Subject to rater fatigue and unconscious bias | Same criteria applied to every test |
| Accuracy | Inter-rater reliability varies between raters and over time | Same level of accuracy as the panel of expert markers, with an inter-rater reliability above 90% |
| CEFR alignment | Manual mapping to descriptors | Automated CEFR reporting (A1–C2) |
| Data per learner | Typically a single CEFR level or pass/fail | Overall CEFR level, a numeric score for placement within the level, and a sub-skill profile across fluency, pronunciation, grammar, vocabulary and task achievement |
| Scalability | Requires more trained staff as cohorts grow | Scales without additional staff |
| Cost | Staff hours and venue costs | Per-test credit; no staff hours or venue costs |
In teacher-led speaking tests, a learner’s grade can depend to some extent on which examiner they meet. Human raters are susceptible to inter-rater variability – differences in strictness, the order in which responses are heard, or natural fatigue towards the end of a long day of testing. DST’s AI treats every test taker exactly the same. By applying a standardised model to every audio input, the system ensures that a CEFR level awarded in one session means the same as one awarded in another – a level playing field for everyone who sits the test.
The reliability of DST is not theoretical; it has been independently measured. In a 2024 calibration study conducted in Germany, a panel of experienced human examiners cross-evaluated a sample of learner responses alongside the AI. Results show the same level of accuracy as the panel of expert markers, with an inter-rater reliability above 90%. In other words, the AI’s judgement sits inside the band of normal expert variation: it disagrees with human experts no more than human experts disagree among themselves.
A common concern with automated testing is whether AI can be tricked by pre-prepared scripts. DST addresses this through Task Achievement analysis. The AI doesn’t just measure fluency and grammar; it evaluates whether the response actually answers the prompt. By analysing the semantic relevance of what’s said, it distinguishes between a learner who has memorised a text and one who can genuinely navigate a communicative challenge – solving a problem, developing an argument, responding to the unexpected. This is also true to the CEFR’s can do approach: language ability is described not by what a learner knows in the abstract, but by what they can actually do in a real communicative situation.
Where a human examiner might award a test taker a single grade – for example, A2 – DST breaks down each response into sub-skills, producing a detailed profile of the learner’s speaking ability. The spidergram shows how the same A2 learner might score strongly on fluency and vocabulary while needing to work on their pronunciation. This means teachers and L&D managers can see exactly where to focus instruction, and learners themselves can see not just where they stand, but what’s behind the grade.
