Relative Difficulty of Examinations in Different Subjects - Review

With the release of GCSE and A level this August, some attention inevitably turns to the supposed relative easiness of exams. It is, therefore, useful to look at a report released by a group of researchers at the University of Durham in July 2008, entitled: Relative difficulty of examinations in different subjects.

The report, commissioned by SCORE, a group concerned with science and education convened by the Royal Society and other interested bodies, addresses a number of key research aims:

• To discover how (if at all) the relative difficulty of different examinations can be defined, how well different statistical methods agree, how consistent differences are over time, and how much estimates of relative difficulty vary for different subgroups.
• To investigate what evidence existing studies present on relative difficulties of UK examinations in different subjects and to what extent these existing results converge.
• To investigate whether STEM (science, technology, engineering and mathematics) subjects are generally more difficult than others.
• To look at the practical implications and policy options of any differences in difficulty.

 

The research group, led by Dr Robert Coe, compared data on nearly a million GCSE and A-Level candidates and reviewed 28 different studies of cross-subject comparison conducted in the UK since 1970. Various methodological approaches to determine relative difficulty of subjects were discussed then five different methods were applied to the data.

Overall, there emerged substantial differences in the average grades achieved by the same (or comparable) candidates in examinations in different subjects, at both GCSE and A-Level. Although the choice of which statistical method to use made some difference to this, the inter-method differences were always much smaller than the overall differences in difficulty of different subjects by any method.

The statistical differences revealed, according to the researchers, that at A-Level, the STEM subjects are not just more difficult on average than the non-sciences, 'they are without exception among the hardest of all A-levels'. The other sciences are also generally more difficult than the non-sciences, though the difference is less extreme. Thus, on average, subjects such as physics, chemistry and biology at A-Level were a whole grade harder than drama, sociology or media studies, and three-quarters of a grade harder than English, Religious Education, or Business Studies.

At GCSE, the STEM subjects were seen to be a little more difficult on average than the non-sciences, though the difference was less than at A-level.

What does this mean for A-levels and GCSEs?

The SCORE report suggests three possible policy implications:

1. Do Nothing. Interpreting statistical differences as 'difficulty' can be problematic - e.g. difference in achievement could be caused by several factors other than genuine difference in standard of the two subjects, e.g. motivation of candidates, the quality of teaching, the intrinsic interestingness of the subjects. The current situation is largely acceptable. Therefore, the best course of action is to take none.
   
2. Make all the subjects the same standard. The standards of grades in different subjects could be equated to make them statistically comparable. However, the SCOPE report suggests this is unattractive as some subjects would become far too hard, with others becoming too easy and comparability over time being lost.
   
3. Change the way grades are used. Existing statistical differences would continue but a 'fair conversion rate' would be applied so some grades are acknowledged as worth more than others. The researchers suggest this is the best way forward.

 

What does this mean for RE?

The report illustrates a clear divide between the science subjects and the arts subjects. The STEM subjects sit at the harder end of the graph, in both A Level and GCSE, and the newer arts subjects, such as Film Studies, Photography and Media fall on the easier end. Religious Education appears with the 'traditional' arts subjects, such as English Literature, English Language, fairly close to the middle of the graph.

Thus if grades, as the researchers argue, are to be viewed as having different values depending on the subject, a B in physics being roughly equivalent to an A in Media Studies for example, then an A in RE should be viewed as equivalent to an A in the other traditional arts subjects. While there is a difference of three quarters of a grade between RE and the STEM subjects it is important to note that statistically it came out as comparable with arts subjects which are usually viewed as academic - English Literature, History etc.. This illustrates that RE is not a soft option as is occasionally suggested and is an academic subject in its own right. Furthermore, the comparability with the academic arts subjects shows that RE examinations are achieving an appropriate level of difficulty.

James Robson

Autumn 2008