I smell a rat.
I’ve been watching news reports for the last couple of days making reference to a bias in the re-assessment of grades. All of them were very emotional. Few of them contained much in the way of evidence. With so many people being downgraded it would be easy to pick out problematic examples. I always get suspicious when news reports don’t have evidence.
So I went and looked on line. Guess what – there was a statistical analysis of the process available and easy to read. Why wasn’t it front place in reports?
Okay, this could get a bit technical, so I’ll simplify what I found.
First – year after year there is a bias at the top grades against the poorest areas. Every year. The final assessed results this year follow the same bias, not more or less – the same. So the argument that the process of adjusting the CAG (teacher grades) has resulted in a bias toward the rich is no more valid this year than any year. I’ll come back to that later.
The second thing that the analysis showed was that the teacher assessed grades were better than you would typically expect. The report doesn’t point fingers, it just notes this is natural. A teacher with a B/C student is most likely to opt for a B this year. I understand that. This has compressed the grades upward, which is one of the things the algorithm was designed to correct.
But there is another more important thing in the report – the report shows that the teacher assessed results have a larger than normal bias toward rich areas. That was the second thing the algorithm fixed. It removed the excessive rich bias and replaced it with the normal rich bias. So this means the downgrading affected the grammars and fee paying schools more than the poor schools. Hmm. I guess it wasn’t hard for the former public school boys to choose the teacher assessed grades as the best option then, it helps their former educational establishments.
So what will a return to teacher assessed grades mean? Look at the top first. Changing grades will not create new university places. What it is most likely to do is introduce a wealth bias. That’s right – it will replace people from poor backgrounds by people from rich backgrounds simply on the basis of wealth.
The other side is the pass/fail issue. Getting a pass is really important for some less cerebral careers. This is a problem, because the algorithm looks like it was very heavy handed. Normally you expect around 1 in 50 would get a U, the teacher assessed grades were closer to 1 in 500. It might have been sensible to avoid “fixing” this.
How does this vary from normal? Well, actually it is the same process that is normally used, the only difference is the teacher assessment. Distributions are adjusted every year, but it seems the big difference this year is that the rich bias has been dealt with. The whole idea of the grading system is to produce a sensible profile of students. If everybody gets an A then admission to university would need to use different criteria for entry (maybe which school people came from). Moving from the algorithm based results to the teacher assessed results will have the effect of moving pupils from rich areas up the ladder compared to pupils from poor areas. I’ve seen so many of my friends jumping on the Tory bashing bandwagon without thinking about the end result.
However there is something that comes out of this which I think is much more important. The statistical analysis shows that teachers in rich areas are over generous with assessments of pupils. This might be the first time this evidence has been so clear. So how about the normal bias toward rich areas? How much of A level and GCSE results come from teacher assessment? Given that we now know that teacher assessment is biased toward rich pupils is it maybe time to change that process?
Bottom line for me is that, in my opinion, there is a problem with the system every and this year has just highlighted it. Sadly fixing this year for the rich pupils will probably be the end of it – after all, if this was just about the pupils from poor families there would be no way there would be so much press coverage.