Data forensics for exams: a case study
Since the establishment of Xquiry, our data forensics services have been deployed by various exam institutions. One might ask what those analyses showed these institutions about exam fraud, and what actions the results have initiated. In this article we discuss a short case study of an analysis for one of our clients, and the measures they took based on our findings.
Exam institution and analysis
The exams of this particular institution are administered nationwide, in groups at training locations and on an individual level at exam centres. There are many different exams within the line that candidates can apply for, resulting in a total count of multiple tens of thousands of candidates a year.
Data forensics analysis over a period of three months showed a notable pattern. One trainer in particular appeared to have trained a significant count of candidates detected for possible fraud. When corrected for the total of candidates the trainer trained in the sample, the notably higher detection rate remained. And, not only were candidates delivered by this trainer more frequently detected for one of the exam; the trend continued over multiple exams over the line.
Digging deeper into this pattern a specific pool of proctors appeared to continuously supervise the detected candidates at this location. And ultimately there appeared to be one event in particular, where many of the candidates present were detected for possible fraud.
Due to these results the exam institution started an investigation into the causes of the pattern found. The lists of candidates of suspicious exam events were studied, along with the process reports filed by the proctor for each event. For the highly detected event an unusual incident was reported: the proctor had left the exam room to escort a candidate to the lavatory. This left the question whether the other candidates had been left alone in the exam room, without a proctor.
A consult with the trainer revealed a different scenario; against all regulation of the exam institution, a trainer had been present during the event and thus been left alone with the candidates during the exam. Moreover, the trainer revealed that this was not a standalone occurrence, but regular practice. But then why is it prohibited? The absence of the trainer or teacher is key to independent examination.
With or without ill intentions a trainer (with more knowledge of the candidate and his or her learning progress) can be tempted to assist the candidates to find the correct answers on the test, unlike a well instructed proctor.
So, what happened next? The exam institution first evaluated their own communications of the regulations surrounding independent examination and surveillance. It appeared that certain regulations were unclear and unknown to many parties involved in the examination of the candidates.
These regulations, especially those surrounding the presence of a trainer in the exam room during the exam, were stressed by the exam institutions. A measure with positive effect, as the trainer concerned did not reappear as such in the data forensics analysis of the next period.
Xquiry data forensics can not only detected possible fraud, but may also reveal structural problems and solutions for fraud prevention with the entire exam institution, in this case the communication of regulations. Are you interested to know what Xquiry would have to offer for your exams? Contact one of our experts at firstname.lastname@example.org or +31 33 489 29 60.