October 10, 2014

A change is coming

Floor standards – they used to be floor targets – have been around for many years. Ten years ago the target for secondary schools was 20% of pupils getting 5 or more passes at grade C or above – it didn’t include a requirement for English or maths then either. How things have changed – the current standard is 40% getting 5 A*-C passes and that has to include English and maths (5ACEM).

Back in 2004 the regime introduced by David Blunkett was in force – any school below 20% 5A*C for 3 years in succession would be closed. At that time FFT were producing value-added (VA) analyses (we did, and still do, both value-added and contextual value-added) for a range of indicators. I was invited to a meeting at DfE because they were interested in an analysis of VA for 5A*C. 
Interestingly, the analysis showed that – for schools below 20% - around 1 in 5 of them had value-added scores in the top quartile. Reaction to this was, thankfully, not ‘goes against policy so ignore it’ but was ‘that is interesting  ... tell us more’. The civil servants concerned discussed the analysis with David Miliband (at that time a minister in DfE) and the outcome was a change to policy – no longer would schools below 20% be automatically closed – some would (those were attainment, based on intake, should have been way above 20%) but others would be supported in various ways.
What impact did this have? Well, some of the schools made rapid progress and have continued to do so since then. Why? A common message from some of the head teachers was that they were able to say to staff “we are doing well in terms of pupil progress – how can we do more?” instead of “I’ve just come back from a meeting where we were told how awful we were”.
The other thing that came out of those discussions was the need to look beyond threshold (5A*C) measures. Reports which FFT provided for DfE included – at our insistence – analysis of overall points score as well as 5A*C. This meant that schools focussing too much on the C/D boundary could be distinguished from schools who were doing well with all of their intake.
Well, it may have taken 10 years, but we finally have – in Attainment 8 and Progress 8 – headline accountability measures which focus beyond the C/D boundary. The threshold measures are not disappearing (C+ in English and Maths, English Baccalaureate) but, critically, Progress 8 is to be used for the new floor standard.
Progress 8 comes into effect for all schools in 2016 but there is an option to opt-in a year early. Schools have until June 2015 to decide. So, key questions will be:
• What difference will the change be from 5ACEM to Progress 8?
• Will it change the number of schools below the floor?
• How will it affect my school – and should we opt in a year early in 2015?
What we need to bear in mind is that there are other changes coming into effect that will also have an impact:
• In 2014 decisions made by the Wolf review come into effect. In broad terms these mean that no subject counts as more than one GCSE (some vocational subjects used to count as more than one), a smaller range of ‘high quality’ subjects count and a maximum of two vocational subjects can be included.
• In 2014 rules regarding early entry come into effect. These mean that, for the purposes of performance tables, DfE will count the first entry. In the past this was the highest grade achieved.
• From 2017 onwards, ‘reformed’ GCSEs graded on a 1-9 scale will be part of KS4 outcomes and, by 2020, all GCSEs (and presumably other subjects) will be assessed on this scale.
Let’s start by looking at the potential impact of the 2014 changes. All analyses are based on outcomes in 2013 – this blog will be updated or extended when 2014 data becomes available.


So, applying Wolf criteria would (in 2013) have almost doubled the number of schools below the floor and applying both Wolf and first entry would have had a massive impact – with around a quarter of schools below the floor standard.
By the way, the figure of 189 shown for ‘2013 rules’ is higher than the 154 published by DfE (click here). This is because our calculation includes schools which are now closed. Also, the full definition of floor standard means that schools which are above the national median for expected progress in English and maths are excluded from the DfE figures.
So, are we likely to see over 800 schools below the floor standard when 2014 outcomes are published? No, we think that increases on this scale are very unlikely. Why?
• Schools will have changed their curriculum with a focus upon increasing the number of GCSE subjects entered and on ensuring that vocational qualifications are, where appropriate for pupils, those listed as ‘high quality’ subjects following the Wolf review.
• Schools will have adapted to the ‘first entry’ rules – and these do not fully apply until 2015 onwards by which time schools will have adapted further.
• Some schools will be excluded because their pupils make expected progress above the national median.
So, it looks likely that the number of schools below the floor will increase in 2014 but it’s impossible to tell by how much – there are too many variables to make even an educated guess.
What then of the possible impact of Progress 8? The floor here is calculated in a very different way. Instead of being an arbitrary standard (40% of pupils above a threshold) it is a relative measure:
• Each pupil’s attainment in a group of 8 subjects (click here) is compared with the average for pupils nationally with the same prior (key stage 2) attainment.
• The difference is divided by 10 (8 subjects but English and maths are doubled).
• If the school’s Progress 8 score is more than half a grade below then it will be below the floor standard.
• The ‘escape clause’ provided by expected progress in English and maths no longer applies.
Let’s look at what would have happened in 2013 if Progress 8 had been applied – and how it compares with the earlier figures.

The Progress 8 calculation is done using Wolf rules because this is what will apply when Progress 8 becomes the new headline accountability measure in 2016. One difference, though, is that Progress 8 allows up to 3 high quality vocational subjects – 1 more than allowed under the 5A*CEM calculations for 2014 and 2015.
The number of schools who would have been below the floor standard in a mythical world where Progress 8 was applied to 2013 data is 325 – so similar to Wolf without early entry rules.
What’s more interesting though are changes in the schools who would have been below floor standard. The chart below shows, for maintained secondary schools in 2013:
• The % of pupils attaining 5A%C (including English and mathematics).
• What the school’s Progress 8 score would have been.
• Both calculations apply ‘Wolf’ rules but do not apply ‘first entry’ rules.

In this chart, schools are shown in 4 groups:

In total, 500 schools are below the floor on either one or both measures – but only 180 of these (just over one-third) are below on both:
• Some schools with low 5A*C (EM) figures are above the Progress 8 floor
• Some schools with high 5A*C (EM) figures are below the Progress 8 floor
In the main, this is because the 5A*C (EM) floor is a measure of both the attainment of a school’s intake and the progress made by pupils in that school – and, in reality, it only looks at the performance of pupils around the C/D boundary. Progress 8 focuses only on the progress made by pupils and is equally affected by the performance of pupils with high, middle and low attainment upon entry to the school.
An initial look at variations across regions and types of school suggests that the impact is spread fairly evenly. Schools where the curriculum has remained more GCSE-based tend, in the main, to do a bit better.
Areas such as the North-West, North-Est and Yorkshire & Humberside may see more of an increase in schools below the floor than other areas when 2014 data is published. This might also be the case for sponsored Academies. 
Again, however, a degree of caution is needed. Trends over the period from 2011 to 2013 show that schools have been moving towards a more GCSE-based curriculum and we expect this to continue – so the patterns might be quite different by the time we get to 2016!
When thinking about Progress 8, schools will need to think beyond the overall Attainment 8 and Progress 8 scores. A school’s Progress 8 score is, in the main, affected by two things:
• How many subjects (out of the 8 allowed) has each pupil entered?
• What grades has the pupil attained in the subjects which count?
We have analysed this by splitting Progress 8 into two parts:
• Quantity - Each pupil’s number of entries - subjects counted in Progress 8.
• Quality - Each pupil’s average attainment in the subject which counted.
Analyses using this approach can how quantity and quality compare to national averages based upon the school’s intake (attainment of pupils at key stage 2). They will enable schools to better understand their own strengths and weaknesses, to plan for improvement and to decide whether or not to opt for Progress 8 in 2015. 
Using this approach we can make a guess at what could happen in 2016. If, by then, schools had changed their curriculum and entry choices so that most pupils were entered for 8 subjects that counted in Progress 8 then – even if levels of attainment were the same as in 2013 – the number of schools below the floor would fall from 325 (the calculation earlier) to 189.
It is important, therefore, for a school to be aware of:
• Its’ own strengths and areas for improvement.
• How the national picture will change as other schools change their curriculum and entry patterns.
We will be providing schools with reports based on this approach at a series of events, which we are running in partnership with ASCL, (click here) and will, subsequently, provide further reports via FFT Aspire.
Much to ponder. What will reality turn out like when DfE publish 2014 data? What impact will this have on primary schools when DfE announce details of new tests and floor standards?