I have started reading Graham Nuthall ‘The Hidden Lives of Learners’ on my journey into work on the tube. I wanted to reflect for clarity of thought on where I am up to at chapter 2 ‘myths and misunderstandings about assessment’.
For me this chapter has made me think about assessment holistically across the curriculum that I lead. More specifically in the short term about low stakes knowledge testing that we use need to be refined. We assess knowledge, understanding and skills using a variety of methods. To assess the level of knowledge and in particular understanding definitions of key terms and concepts to increase increase students subject specifc (geographical) volcabulary we use the socrative app on the ipad.
Chapter 2 of Nuthall has made me reflect if I have put enough thought into the creation of these knowledge tests. There is a thought about how I want these knowledge tests to be used. Do I use them to evaluate my teaching? Do I use them as a point of reference for students to be able to see what they know and don’t know? Probably both. My feeling is that with these tests being ‘cumulative’ where essentially 10 questions are added to the same ‘quiz’ each half term that students and myself should see an increase in the performance of students on this test and then students can re-learn the concepts they still don’t fully understand. These tests become more useful tool for students to learn new terms and hopefully they see the relevence too.
The problem I see with my current testing is through question design. Perhaps i’ve been a bit careless in the desgin of knowledge based multiple choice questions. I have to be really careful if tests really demonstrate what students know and don’t know. There may be students who try hard and really want to do well, they may be students who don’t care. Looking at test results for one class there were unexpected results such as a 100% for a ‘weaker’ and some more able achieving 60%, furthermore the 100% result dropped to 40% on the second cumulative test – these suggest a lack of validity. The aim should be for all students to want to succeed and to have fully valid questions.
I also have to be really careful with question design to ensure that the question shows that they fully understand the concept being tested rather than achieving the correct outcome through chance. Page 46-48 of Nuthall is about students answering tests through two systems of searching memory and deduction. Direct recall is rare, so questions also need to enable students to be challenged to deduce the correct answer using their wider background knowledge. I.e. not making it too easy to deduce the correct answer. There is also the problem of students who achieve 100% each time, therefore suggesting that questions could be further challenging for those students.
When planning, the important concepts and key words of the topic that need to be learned are decided first and taught within the unit or project (backward planning). These are assessed through the knowledge test. The knowledge test therefore does tell me if students have fully grasped the concept or not. Perhaps knowledge tests should also be used more frequently within a unit so that re-teaching can be carried out and with the aim of all students achieving a high percentage on a end of unit cumulative knowledge test. The idea of being cumulative is so that learning is spaced to allow concepts to enter long term memort.
So perhaps as a reflection, more challenging and thought out questions based on the ideas of deduction need to be created. There could be a need to do the following structure within a unit/project to assess knowledge and understanding of concepts.
1) Pre- test – use it as a means of planning – e.g. use of SOLO taxonomy and personalistion – do some students need to be taught the concepts they already know?
2) Test concepts being taught within lessons more regularly – spaced practice. Doug Lemov’s ‘no opt out’ here also works to encourage students to memorise concepts.
3) Test multiple concepts and key words in a cumulative end of unit knowledge test – spaced practice over a longer period. Socrative app can give students feedback on performance and the teacher to show teacher if strategies have been successful if students care and question design in thorough.