Test reliability

The test is highly reliable and can accurately grade within ±0.54 IELTS point of a human grader.

Between March-June 2022, faculty from a major South American university conducted a high stakes pilot of the Speechace speaking test. 110 students took the test successfully and the Speechace speaking test provided scores with the following distributions:

Later, an ACTFL certified evaluator and instructor at the university conducted a manual evaluation of each of the student’s recordings, which were then compared against the automated score from the Speechace speaking test. The results of the comparison are showcased in the chart below:

From the above chart, we observed an average correlation of 0.80 between teacher score and Speechace algorithmic scores with an R2 of 0.65. Furthermore, we observed a mean difference of ±0.54 IELTS points between teacher score and Speechace scores.