The way that progress is measured in schools is fundamentally flawed

The way that progress is measured in schools is fundamentally flawed

As a former senior HMI, I fully understand the importance placed on national league tables and the role they play in assisting parents, carers and others to assess the relative success of a school.

However, that’s why the data used must be accurate and take into consideration the characteristics of each pupil in the cohort. We’ve just published our fantastic 2018 Annual Report, but I struggle to accept that it is a truly fair reflection of our academies’ successes.

The introduction of the Progress 8 measure a few years was an attempt by the Department of Education to quantify more fairly how much progress students make during their time at a secondary school. Now while I am fully supportive of that objective, unfortunately, I am becoming more and more disillusioned with Progress 8 because I believe it is fundamentally flawed and biased towards schools with the most affluent intake.

Despite our academies having the second lowest attainment on entry of any multi-academy trust in the country, our performance is strong, particularly for disadvantaged students. But this is not reflected fully in the Progress 8 analysis. You might say, as Mandy Rice Davies so famously did, that I would say that, but the evidence shows it is true.

Compelling new research from the University of Bristol reveals if a contextual element is applied to the DfE’s performance data some schools would rise more than 500 places in the national league tables while others would fall by a similar amount.

Dr George Leckie and Professor Harvey Goldstein at the University’s School of Education, extended the Government’s Progress 8 measure to account not just for pupil prior attainment in English and mathematics, but student ethnicity, gender, age within year group, special educational needs and eligibility for free school meals.

The “adjusted Progress 8” sees the Co-op Academies’ rankings in the national school league tables change considerably. The academies included in the research saw performance rise for three of them dramatically while one fell slightly. One of the academies rose by 24 percentile points to the 90th percentile!

But this is not just about a school’s position in the league tables. It is my belief that some very good school leaders don’t get the credit they deserve and a few, even more worryingly, senior leaders have lost their jobs based on the DfE’s narrow analysis. That is why it is imperative that the data is as accurate as possible and takes into account every aspect of a school’s performance by applying a contextual element.

During my time with Ofsted, I was a member of a very small group that developed a new school inspection framework that incorporated contextual value added. It was, in my opinion, more sophisticated than the blunt Progress 8 measure we use today and more reflective of a school’s academic success but it was withdrawn by the DfE when the coalition government took over because it was felt that it didn’t challenge pupils sufficiently well to reach the national average. An aim that is as flawed as the Progress 8 measure itself!

At the Co-op Academies Trust we are considering asking the University of Bristol to undertake a contextual value added analysis for all of our academies each year with the intention of publicising this alongside the government’s blunt measure.

I would urge other schools and trusts to do the same so that parents and carers, and, indeed, staff and governors, get a more accurate picture of a school’s performance.