This is my first ever blog post. Ra-ra and all that, let’s get to the subject matter.
Yes, its going to be one of those repositories for all those thoughts I probably vastly overvalue when I first conceive of them. But as I cannot be objective and they may actually serve some purpose, I might as well pop them on-line.
Topic of today? UK exam scores. Why? I just read some other blog on the subject: http://www.badscience.net/2007/08/calling-all-science-teachers/ and have some feelings on the matter.
I am not particularly qualified to comment on the education system, so I beg of you don’t listen to my ‘opinion’, but rather follow my logic…
Many people have suggested, and most recently in the public eye, Dr Goldacre in his excellent book “bad science”, that exam standards may be dropping in the UK.
I’d like to analyse this statement for the general case (i.e. any population of which a subset write an annual exam in which the questions do not repeat). Let’s try to frame the question of their ease in a less emotive logical statement…
Let’s say we have data that show the pass rate is gradually moving up year-on-year.
This must mean that one or more of the following is true:
i) the population is getting genetically smarter
ii) the population is increasing well prepared by its environment (parents, teachers, peers, the internet, etc.)
iii) the subset of people in doing the test has changed
iv) the questions are becoming better correlated to what people know
v) or last, the test questions are getting gradually ‘easier’ (or the marking more generous)
There may be more, but don’t want the extreme complexity to cloud my (eventual) point.
Now each of these statements is hard to prove without more data – and the only data we seem to have is the test scores (although we do have the tests themselves which may prove useful).
It may well be that people are getting smarter – but we might use some science to tackle that – for example you could argue that evolution cannot work this fast (and I personally doubt that nerdyness is particularly good survival and seduction tool).
But the environment is certainly changing, the subset doing the tests may be drifting, schooling techniques are being constantly refined and the correlation between what’s interesting (celebrities, MMR vaccines) and what’s examined is also hard to pin down.
I would say there is more than enough vagueness to ensure that no-one, no matter how well qualified, could answer the question “are we dumbing down” with any conviction.
However, there is a “but”.
The examiners can set the difficulty of each fresh test to be whatever they want (in theory). They could make it easy and let everyone get A’s, or they could make them so hard that only the brightest “X” percent get an A. Yet what we see, year on year, are slight improvements.
There are (at least) two hypotheses as to what the examiners are doing:
a) they are aiming to make the questions identical in difficulty to the last year, despite the full knowledge that this strategy has, to date, resulted in a gradual trend toward better marks.
b) they are deliberately aiming to get just slightly better results than last year due to some “incentive”
As the examiners for all the subjects are probably a fairly independently minded bunch and as there is no evidence for it, there are good reasons to doubt the latter hypothesis. Occam’s razor would surely favour the former, though we can’t be sure.
So where does that leave us?
We can’t suddenly make the tests harder, thus lowering the number of A grades to what they were years back – that would be unfair, and would mean that future young people will actually have to know more and work harder than their colleagues from the present time to get an A.
Why not simply rank the scores, then place predetermined fractions into each grade? This, incidentally, is what (I believe) was done when I went to school, which was essential as we had several different regional exam boards with different exams, so rankings rather than absolute results were felt more comparable. Isn’t this how IQ tests scores work? This would mean, incidentally, that by definition, exam/IQ scores for a population simply couldn’t increase with time.
Perhaps most attractive is the option to leave things as they are in the UK, and ignore the media circus. After all, what does their opinion matter?