It’s hard to tell if students really know the answer or are guessing, which is why one university is scrapping multiple-choice exams. Shutterstock
Multiple-choice questions have become one of the mainstays of assessment at universities across Australia and the world. But is this the best approach? Or even valid?
Central Queensland University has weighed up the advantages and limitations of multiple choice and has decided to abolish them from all exams. This is for two reasons: firstly, because of the potential impact of guessing; and secondly, because of the lack of authenticity in the method of answering the question.
Do they know the answers, or are they guessing?
The multiple-choice test format has its origins in the work of educational psychologist Frederick Kelly and was first used on a wide scale in US Army intelligence testing during the first world war. Since that time, multiple-choice questions have been adopted across many areas of education and testing.
In universities, they are often employed with large enrolment classes. Their principal benefit is their efficiency. Once set, they can be machine-marked, giving a fast turnaround time for results.
This is a major boon for hard-pressed teaching staff, who have tight deadlines. However, it is important to consider the limitations of the multiple-choice format.
The first limitation relates to their mathematical approach: a correct choice for each question is typically given a single mark, with either a zero mark or a fractional negative mark for the other options. However, that correct choice may be the result of either a) knowing or b) guessing, and there is no easy way to separate the two.
Exams in most subjects at university have a pass mark of 50%. To achieve such a mark in a multiple-choice exam with 100 questions and four options (one correct answer and three “distractors”) can be achieved by knowing 33 answers and guessing the remaining 67 questions (on average, scoring a “correct guess” in a quarter of these, 17).
And that’s only the “on average” position. Once you factor in the randomness of guessing and the influence of binomial probabilities, we can only be 95% confident that the student knows, and hasn’t guessed, around one-quarter of their answers.
This gives rise to something best termed “spurious accuracy”, where a student who has scored 49% will fail and another on 50% will pass, but that may simply mean that the second student made a lucky guess on one more question than the first. Is that really the way we want to decide whether students have passed or failed a subject at university, when there is such uncertainly about the validity of the mark?
You can’t choose options A to D in real life
Some of the mathematical and statistical issues with multiple-choice questions can be reduced by raising the minimum mark required for a “pass”. For example, with a requirement of 80% to pass and with appropriate negative marking, the issue of random guessing can be reduced considerably. However, a more fundamental obstacle to their use cannot be so readily overcome; the format of answering multiple-choice questions is not one that many people will meet in the real world, except perhaps if they’re a quiz show contestant on TV, or if they’re completing a marketing survey.
Source: Does the student a) know the answer, or are they b) guessing?
Leave A Comment