Measuring conceptual understanding in mathematics

A major challenge for mathematics education research is how to measure pupils’ conceptual understanding with acceptable validity and reliability. Furthermore, opinion is often divided on whether abstract or concrete representations are better at supporting pupils’ conceptual understanding.

This study developed a measure of conceptual understanding using the Comparative Judgement (CJ) approach, and explored how it could be applied in practice.

CJ is a way to assess open-ended and creative mathematical work, which has been found to produce reliable and valid outcomes. It involves no mark schemes and no marking. Instead, two pieces of student work are presented on a screen and the assessor decides which is ‘better’. When many such pairings are shown to many assessors, the decision data can be statistically modelled to generate a score for each student.

Key findings:

• CJ can be used to evaluate students’ conceptual understanding, and to evaluate the relative effectiveness of different teaching approaches. This finding will enable researchers to evaluate the effectiveness of educational resources and approaches more quickly and validly than has been possible to date. This in turn will provide policy-makers and teachers with better evidence about the relative effectiveness of educational interventions.
• In terms of the abstract vs. contextualised representations debate, the study found different evidence for different types of mathematics. When comparing two technology-specific approaches for learning algebra, the researchers found that the abstract approach they tested was more effective than a contextualised approach. However, for differential calculus, the abstract approach and contextualised approach that they tested were found to be equally effective.
• Therefore, the role of abstraction and contextualisation when teaching mathematics is nuanced. Effectiveness depends on the concept being taught, the approach used, and perhaps the age and prior achievement of learners. Importantly, the CJ approach enabled the researchers to overcome the measurement problem that has limited the findings of previous research.

 This project follows on from previous work, including a study by the same researchers, and a recent pilot study demonstrating how CJ could be used to measure learners’ understanding of mathematical concepts.