Uncontrolled Assessments

Despite the title, a lot of this also relates to AS and A2 – Physics, at least. (They don’t trust me with squishy stuff or cooking.) It seemed a waste to spend time answering the call for evidence from Ofqual without blogging it too. Especially because it made me realise how messed up the whole ISA situation is. Maybe it’s much better in OCR and Edexcel; ‘my’ kids do AQA.

I’m not going through all my answers, just those that seemed particularly relevant, complimentary or critical. I’ll start off, because I’m a teacher, with the comment that when the question below is on an official Ofqual document, it seems odd that Gove wants students to lose marks for poor grammar:

“Please explain you answer.”

Weaknesses

There is no way to assess hands-on practical ability by the current method – the students are specifically told that their results are irrelevant. The students’ problem-solving abilities are tested by the practical itself, but this is not measured in any way.

In my opinion, there is very little difference, if any, to previous versions of controlled assessment – the results are no more ‘authentic’ than before. Teachers are still able to see the question papers in advance, and for AQA, section 1 is always effectively identical to the sample paper – which we can share, with the markscheme, with students. Unsurprisingly students end up rote-learning their answers. To be manageable in a real classroom all students end up doing a similar practical anyway, despite claims they would have more freedom. The ‘research’ section is useless, and students have frequently shown that the only helpful result when searching for a context is the teacher notes supplied by AQA. This means that the variety in support – which an AQA advisor agreed was ‘against the spirit but still allowed’ – shows that teacher efforts are still a big issue.

Workload/Management

The type of task has been badly managed by AQA, and is longwinded. Students must:

  1. research a hypothesis
  2. design an experiment and results table
  3. complete an exam paper on the design process
  4. complete the practical (modified if necessary)
  5. produce a graph or chart of results
  6. complete a second exam paper on their analysis.

Completing this process, at least for the first time, takes a minimum of 7 lessons in school (researchx2, paper1, practicalx2, graph, paper2) and probably more for each ISA. Students missing lessons due to illness or other commitments obviously makes the process more complex, a burden that inevitably falls on the teacher. In our timetable, students spend at least three weeks on one investigation, often only spending 1-2 hours on practical work. This is not sustainable.

It takes a while and there is a huge amount of separate pages they have to be provided with, especially for the final paper (their research notes, their results and graph, a group set of results, two pages of case studies). Each section must be marked BY THE TEACHERS and kept securely, then moderated.

The specific tasks set show that AQA have not tested out the likely experiments, nor checked online for research sources that are reasonably pitched for GCSE students. The results are frequently nonsensical with the equipment available in a school lab, meaning no useful pattern for the students to discuss. The practical tasks are very straightforward, about a Year7 level.

The marking of the work is a long process and standardisation is difficult. I would conservatively estimate each ISA takes at least 20 minutes per student (paper 1, table, graph, paper 2). For a class of 30 this is ten hours of marking for each ISA – and they are expected to complete at least two per year. We have currently been trying to complete them for year 9 and 10 classes. This means up to thirty hours of marking per colleague.

Conclusion

The tasks are badly designed and not fit for purpose – the skills could be assessed more effectively and more uniformly. It feels like the tasks have been tweaked and changed so many times that they are effectively designed to fail at their claimed purpose.

The workload is unmanageable in my subject. Exam boards would, I suspect, have spent a lot more time thinking about this if they were responsible for paying those marking it. I don’t know how the cost of sitting the module has changed compared to externally marked ones, but of course schools don’t have to pay overtime either. This means less time to prepare lessons and teach the rest of our classes. Several exams per ISA also makes it very difficult in terms of SEN entitlement, with extended time and TA support being difficult to schedule with a whole year group doing assessments in a range of subjects, taking up more and more time.

I know colleagues who have missed work due to finishing marking, or because of being ill after ridiculously long hours to meet deadlines.

It could be done so much better. Students gain very little in the way of real practical skills, and the majority of the questions in the exam are about things that could be in regular exams unrelated to the ‘experiments’ they have done.

Students are certainly stretched, in that some questions are so obscure – or the markscheme so erratic – that only a few achieve the highest marks. In some cases the only way a student could write an answer that gains all the available marks is if they had been coached by their teacher. They struggle to apply what they have learned outside of the ISA environment, because it is so unlike the practicals and investigations they do at other times.

Scientific content is at a low level in most cases, but the maths demand can be quite high. This penalises students who are capable of explaining trends and patterns but struggle with numbers.

 Constructive Feedback

Of course, as a teacher I feel the need to suggest a way to improve. I think it’s worthwhile that the ISA starts with a research task, but it needs to be standardised. How about a different structure:

  1. Produce a booklet of sources, some more useful than others, that students use to plan a practical in exam conditions.
  2. All students nationally do the same thing, from the same materials.
  3. A standard practical with a paper, using standardised experiment, like the old A-level set-up.

These exams could be sealed beforehand then externally marked, anonymously, removing at a stroke many of the major issues with internally-marked work. Several samples would need to be produced, but this would avoid the demand for more and more assessed pieces – time consuming for students and staff – in the hope of incremental improvements. The two main criticisms of the current system – that it fails to assess the skills fairly for all candidates, and that it takes too much time – are removed at a stroke.

Two other posts worth reading on this are by @hrogerson on OCR-B and by @DrRachael, also on AQA.

Maybe you have other suggestions?

Advertisements

4 thoughts on “Uncontrolled Assessments”

  1. I am very jealous that the AQA practical activities are at Year 7 level, I feel the OCR B ones are difficult, and deliberately made difficult. However, I agree that the practicals have not been tested, for OCR B either. It worried me that the exam board have to come up with new practicals every year and the tasks will become more obscure and difficult to compete or understand.

    I went to a session about controlled assessments, by OCR. And the poor wording of the exam papers was inflicted by Ofqual as the exam boards wanted the exam questions to be broken down or worded differently.

    Ofqual have a lot of questions to answer regarding these science controlled assessments, on top of how late the new specifications were delivered to us and now the issues in geography.

  2. Interesting concerns you both raise – I am feeding your messages where I think they may make a difference.

    1. Thank you! There are two issues that are perhaps worth flagging up. One is that the requirements from Ofqual are presumably part of the complication, and it will be interesting to see how Gove’s ideas of a terminal exam affect coursework, controlled assessments etc.

      The second is that due to the harsher grading of exam papers due to central decisions, there will be even more pressure on science departments – and therefore on science teachers – to help their students get the best possible marks.

  3. I totally agree with the two main criticisms you drew in your conclusion. In my humble opinion, as a person who has spent more than 10 years in education, the current ISA system fails to assess the skills of candidates fairly. This stems from the fact that firstly, the application of candidate preparation varies so widely whereby some schools seldom give candidates enough advice about the background knowledge required while others give candidates all the information necessary except the actual ISA paper to take home. Also centres application of the mark scheme vary, some centres opting to harshly mark students so as not to be seen lenient by AQA moderators. This ISA practical lets our students down, it is too much of an administrative burden on teachers. The solution would be to have an externally marked centralised system which is objective and transparent to candidates. Failing that practicals in science should be scrapped similar to that in mathematics coursework, instead candidates should have defined competencies in practicals they need to demonstrate which can assessed locally by teachers and if they fall short they should receive extra-lessons to build them up. This would create a much fairer system for candidates, as well as to reduce administrative burden on teachers at the same time.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s