A New Exam Board?

We’ve seen a lot of problems with exams recently – just look at the problems last summer with mistakes in a wide range of exam papers. Today I’ve found that AQA have spent so little time checking that suitable research sources are online that the only good Google results are their own teacher notes, and a primary science investigative cartoon. On top of this, a new specification inevitably means a lack of practice material which means students and teachers don’t really know what to expect.  If you have to explain why this is unfair to non-teachers, perhaps this analogy might help; we wouldn’t expect to have a driving test on the road having only practised in car parks, would we?

I have an idea.

In fact, I have two ideas, neither of which is mine. If we take the ‘backward design’ principle (originated by Wiggins and McTighe, introduced to me by Robin Millar’s work) and combine it with a ‘curated crowdsourced’ model, maybe there’s a way to do a better job. 

Backward Design

My apologies to Robin and other experts if I miss the subtleties – I’m just a classroom teacher with delusions of writing grandeur. Instead of beginning a syllabus with the content that we want to teach, backward design asks what we want students to be able to do at the end – how will they be tested? How will we know if the course was successful or not (or more precisely, how successfully the student has completed it)? If we create assessment tasks that will allow us to differentiate between students – ideally including, but not limited to written exams – then we can develop a list of what students should learn, which gives us a list of possible learning/teaching activities. As Robin and others point out, ‘teaching to the test’ is only a problem if the test is not fit for purpose. If we produce a realistic, useful test then being prepared for it is a positive thing. 

Crowdsourcing

So who better to contribute possible questions than teachers? Imagine a Google form set up by a new exam board; let’s call it CCEB. Exemplar material, based on accepted good practice, shows how to lay out mathematical working. Questions are entered, with a markscheme. Dropdown boxes allow those entering the question to define marks available, and from key words describing the area(s) of science being tested. Active teachers, retired staff, academics – even students – all can contribute. The contributions are freely given on the basis that the results will be freely available as far as practical, probably via Creative Commons licensing.

Curated

When a certain threshold is reached – which if every science teacher in the UK supplies a single question, won’t take long – the submissions are sorted by category and checked by CCEB staff. Because they are being proofread rather than written, it will be quicker and easier. If you have some of the original contributors – determined by random allocation – paid for a day’s work, they can be pre-moderated as well. Mathematical questions can be kept in the same form but with different numbers substituted. A large pool of questions is now complete, ready for the exam, which can be balanced between topics. There will be enough questions, all produced at the same time, for several specimen papers to be made available. With a large enough pool, you could even make all the questions open source, like those for the theory element of the UK driving test.

One Day 

It’s feasible that in the future, with enough questions available, every student could get a different but equivalent exam, as described in John Barnes’ book Orbital Resonance.

In the meantime, maybe we as science educators can get involved with setting better exams than the ones we complain about. The exam boards could ask for submissions in this way now. The cynic in me thinks that this would make it much harder for them to justify their existence. Maybe they would like to prove me wrong.

Advertisements

Demonstration or Class Practical?

It’s always a tricky one, isn’t it? Do we show them the experiment, knowing that a half-dozen or so will be messing around at the back or comparing nail varnish with their friends? Or do we let them loose with glassware and clamp stands, waiting for the crashing noises or the blank looks to begin?

Okay, I’m exaggerating. A bit. But for most of us, it’s probably taking a bit of time to think about the kinds of practicals we do, and why. Are we focused enough on what the students will learn from it? Or are we doing a particular practical or demonstration because it’s in the scheme of work, or because we’ve always done it?

I’ve used among other sources David Sang and Alom Shaha’s workshop at the ASE Conference and materials from Getting Practical and SCORE Education to produce a checklist (downloadable below, simply click on the image). The focus is about the benefits of a demonstration or a class practical. It’s an easy way to think about what can be added to an activity, or ways to tweak it to improve outcomes. Simply sharing with the students what they might be trying to gain from a practical is worthwhile – although in some cases as a plenary to avoid spoiling a surprise or insight. Simply take a moment to read through the lists, and see if you can justify the activity in terms of learning. If you’re not sure, what could you change?

There’s loads of good ideas online – the National Stem Centre eLibrary is of course one place to look – and it’s often possible to convert a practical into a demo or vice versa. For example, I demonstrate heat transfer in fluids using the two chimneys apparatus and a convection square, plus hot and cold water with food colouring in gas jars, which I first saw in ‘Nina and the Neurons’. By the third demo the kids can predict and ex0plain what’s going to happen quite well. I then give them coloured ice cubes to float in water, and to predict, explain, describe and explain again (PEOE) what they see. Bonus points for a commentary that uses key ideas such as ‘density change’.

I hope this kind of reminder is useful, for experienced teachers as well as those more recently joining the profession. Feedback would be very much appreciated, as this is my 100th post and I’ve had less than 1 comment for each on average…

 

Demonstrations (#aseconf 2/3)

I managed to make it to the 2012 ASE Conference for just one day, the Saturday. My plan is to blog it in three chunks for the sessions I attended, in order. We’ll see how it goes. These will be edited versions of my Evernote summaries of the sessions and my commentary (in italics), although I’ll link to other resources I’ve since found that I think are relevant. Apologies if I mix up any names or misquote any of the people involved. I really enjoyed the sessions and the social side, but will cover this in more detail in the third post.

Presented by David Sang (among many other roles, editor of the Practical Physics site) and Alom Shaha (teacher, film producer etc)

In an electricity and magnetism public lecture, Oersted noticed compass movement during public demo – real public science.

I’m now thinking about reenacting this for the students, perhaps as a plenary after more ‘interactive’ work.

  • Use webcam to make it visible.
  • Mark north/south without magnet, make sure kids see change, note alteration.
  • Show range of effect, compare strength of wire and earth magnetic fields.
  • Equal strength when at 45degrees.

A demo has many possible purposes, but should always – like everything we do in a lab or classroom – lead to a better understanding of some of the ideas. It can be used as a stimulus for them to do investigative work. While explaining the demo, we can give differentiated possibilities. A useful mantra should be ‘hands-on, minds-on’.

For any demonstration, there are some things to consider:

  1. Visibility/clarity
  2. Preparation and practice, e.g. clamps and where you stand
  3. Prepare for failure, be ready with explanations
  4. Ensure kids focus on important aspects – what are we changing, what is happening
  5. Involve students in practical (holding equipment, readings, recording data)
  6. Contextualise (history, application, consequences, possibilities for the future)
  7. Predict, (explain), observe, explain. (I already used this myself but now I’ve added a prompt poster to my wall)
  8. Q&A are a standard way to check ‘takeaway’ understanding (why not ask students to ask their classmates a question?)
  9. Extend (possibly via Q&A)
  10. Give correct explanation, try not to give misconceptions (although this doesn’t mean you shouldn’t use appropriate levels of model)
  11. Good opportunity to repeat the demo, perhaps with more involvement or explanation from students (giving commentary?)
  12. Summarise (giving a summary as part of a L2L split plenary would work well)
  13. Safety – nobody died.

For this one in particular (link between magnetism and electricity) can show same principle with generators, generator handles, cheap wind up torches. A wind up torch vs cell/switch/bulb would nicely demonstrate different energy changes (classic misconception is that closing switch is KE) in energy circus.

Why not do demos?

  • Safety (rarely for most – see guidance e.g. CLEAPPS)
  • Unreliable
  • Technician time/materials cost
  • Prefer to ‘learn by doing’? (NB see evidence for/against this)

All worth considering, but use them as prompts to improve quality rather than going straight for a video.

Why do as class practical?

  • Small groups can be fun/hands on
  • Practical skills
  • Know/appreciate problems eg ‘messy data’
  • Make (and justify) plans
  • Experience non daytoday phenomena

Best reason to do demo (from Alom)

We can promote ‘awe and wonder’ by showing them something they could not have observed (or perhaps appreciated in isolation) – this is worthwhile. (eg induction with lenz law in copper tube – sleight of hand helps!). This will often involve an unexpected result, perhaps because we set up the situation with an unnoticed or unappreciated ‘tweak’ or ‘cheat’.

Alom: Nobody goes into science because the science was like ‘magic’ – but because they wanted to figure out magic. Emotional engagement is a good thing, and kids link enjoyment to both teacher and subject. This improves performance, recruitment and retention.

My plan is to turn the choices – reasons to use a demonstration vs a class practical – into a checklist or flowchart for a later blog post. If you’ve any particular ideas, I’d love to incorporate them so why not comment below?

Further reading

#SciTeachJC: Subject Knowledge

The paper for Week 9 of SciTeachJC was Johannes Met­zler and Ludger Woess­mann “The Impact of Teacher Sub­ject Knowl­edge on Stu­dent Achievement: Evi­dence from Within-Teacher Within-Student Vari­a­tion” IZA Dis­cus­sion Paper Num­ber 4999 (2010) (.PDF link)

The main conclusions of the paper were that higher teacher expertise in their subject resulted in a higher level of achievement for their students at primary level. There was an effort to account for confounding factors, partly because the same teacher taught both maths and reading to the student tested. This provides an immediate limitation as far as secondary teaching is concerned, as it might be reasonably suggested that there is a bigger overlap in knowledge between any two science specialists than between, for example, science and language specialists.

Perhaps predictably, the discussion had two main themes; the need for an ‘appropriate’ level of subject knowledge, and how pedagogy also has a huge impact on a  teacher’s effectiveness. To place this in context, @declanf and others suggested that 0.1 standard deviation is a small improvement compared to other factors.

@uncletungsten: I say that 0.1 sd advtg. is next to nothing. Even when 1 sd of subject knowledge could mean 3-5 years of subject specialization.

It was suggested that there is a balance between specialist knowledge – giving a teacher confidence to teach effectively – and having recently struggled, thereby having empathy for a students’ likely problems. @morphosaurus was one of several participants who felt that as a non-specialist, she taught some areas of the specification better than those who had studied it in more detail. How much of this is due to enthusiasm, and how much to better pedagogy, is of course hard to measure. I wonder if colleagues are more likely to use innovative methods with content they are experienced with, or with more recently studied material? @Bio_Joe pointed out that being able to tell a student that yes, we struggled too is very powerful.

@Arakwai: I agree! Gives the teacher a better appreciation & understanding of misconceptions & difficulties students may have.

We agreed that expertise and enthusiasm would often be strongly correlated, and that as long as correct information is taught, that personal interest is often what enthuses students. @Lethandrel and others agreed that a basic level of subject knowledge is necessary before someone can be considered a ‘specialist’. The issue here, as uual, is KS3. Should we be teaching within specialism there to improve confidence and avoid misconceptions?

@mariamush It’s my experience and knowledge beyond spec that enables me to teach Chem successfully, couldn’t offer the same in phys and bio

Most of us pointed out that with a limited amount of time and money, continuing subject knowledge development is possibly challenging. It is, hoever, necessary, when both scientific understanding (Higgs boson anyone?) and the greater emphasis on scientific method have changed since our original qualifications. We talked about how swapping ideas with colleagues, in and out of specialism, can be a big help. Book and documentary recommendations can keep the costs down.

@cardiffscience: Quite RT @teachitsobeing “one page ahead of the class”? Curriculum changes rapidly- anyone’s degree really embed HSW?

@teachingofsci: possibly – you don’t get much better than Jones, ridley, dawkins, @edyong209 and attenborough for evolution!

@RobertDavies2 so revision on top of planning, evaluation, reports, book marking… to name a few? #toomuch

This comes back to an important question; in most cases is there enough variation in teacher subject expertise for it to be worth worrying about? Yes, there will be variation – but cost (both financial and time) is high if effect is small. (@teachitso pointed out that Hattie puts subject matter knowledge 125th in his rank of effect sizes)  Who should pay these costs? Will Heads of Department consider it worthwhile (for general CPD rather than troubleshooting identified individuals) when there are courses on exam specifications?

@AnthHard: If the Sci Learning Centres put on some subject knowledge CPD, would there be much response?

@SciCommStudios:  one of my hats is Uni of Surrey Outreach – we are looking at putting on some Chem CPD…and have been wondering about interest

@Bio_Joe: If SK needs a top up I recommend the 7day free courses run by Goldsmiths (I did genetics one it’s amazing)

Worthwhile as these aims are – and I would comment now that we are considering the opinions of a self-selected group of teachers, not the profession as a whole – are we making the same mistake as Gove, Jamie Oliver and many others by focussing on subject knowledge when we should recognise that we are teachers first, and scientists second?

@danidelle23: i think we agree with each other to a point. knowing how to use your knowledge is probably the hardest part

@DrDav: Think knowing how to teach a topic can be more important that knowing the topic. How to identify and deal with misconcepts.

@morphosaurus defined pedagogy as “Ability to break down concepts for students to understand, and have resources that explained things well helped.” Having these resources to hand, and having needed to break a topic down, might explain why some of us felt that being a non-specialist was not necessarily a big disadvantage when working with younger or weaker students. Avoiding misconceptions is of course a major concern – you might reasonably equate this to the medical precept, “First, do no harm”

Our priority should perhaps be how to teach specialist knowledge, rather than having the knowledge ourselves. In the same way that teachers need to be able to model and teach thinking skills, we need to express ideas so that students can understand them. #asechat, subject specific teachmeets and similar ideas are perhaps a good way to share good ideas about what matters most.

In conclusion: we should neither over, nor underestimate the importance of a good level of subject knowledge. We’d like to see more research on the relative importance of truly specialist subject knowledge (degree level or higher, with continuing ‘refreshers’) in secondary education, compared to other factors.

Further reading

@alomshaha shared a link to a post TwentyFirstFloor blog on ‘what makes an expert teacher.

@Bio_Joe linked to an abstract of a 1998 paper that concluded subject knowledge is one of several factors.

@AnthHard recommended @teachitso’s summary on ‘who should teach’ considering Hattie & the Finland enigma.

Learning Toolkit

After a Twitter discussion I realised I’d never followed up my Learning Journey blogpost with the printable material I was working on, so here it is. These form the basis of a display (with examples, ideally using students’ own work) of a ‘toolkit’ which can help pupils to be more independant. It links well to L2L concepts (see this page from @teachitso for a quick justification), for obvious reasons, and you might like to use individual pages or the whole thing. If you’re in a school which allows mobile devices, why not add a QR code to those which could do with more explanation?

My vague hope is that these ideas will turn into a separate website at some point in the future – a version of this blog, aimed at students to use independantly of teachers. Thoughts, comments, suggestions?

I’ll update this post at some point in the future, but I should really get on with (a)work and (b)ASE conference write-ups. Let me know if any of this is useful.

learningtoolkit as pdf.

From Good to Outstanding (#aseconf 1/3)

I managed to make it to the 2012 ASE Conference for just one day, the Saturday. My plan is to blog it in three chunks for the sessions I attended, in order. We’ll see how it goes. These will be edited versions of my Evernote summaries of the sessions and my commentary (in italics), although I’ll link to other resources I’ve since found that I think are relevant. Apologies if I mix up any names or misquote any of the people involved. I really enjoyed the sessions and the social side, but will cover this in more detail in the third post.
 
From Good to Outstanding (T124)
 
In theory the slides for this are on the ASE website. I can’t find them so I can’t link, even if they are available to nonmembers. Grr.
 
What does ‘outstanding’ look like?

OfSted have a video explaining what they are looking for.

  • Over time, look for evidence of themes not just snapshot
  • Main focus will be on checking amount of progress by students, a way to measure impact of teachers.
  • Data also checked, record sharing and tracking, pupil and parent discusions.
  • Minimum aim is ‘good’, not ‘satisfactory‘.

It’s really interesting that we start with how we will be judged – cf Robin Millar’s ‘backward design’ concept.

Nick O’Brian

  • An outstanding teacher covers all areas, one bit at a time.
  • This is (or should be) a corporate, not just an individual aim.
  • Can be easy to focus on one area, often linked to School Improvement Plan.
  • Being more rounded means supporting each other.
Best way to improve is to share outstanding practice in a department, tap into ‘local’ skills.
 
I’m looking forward to restarting lunchtime ‘skills’ sessions in my workplace. These run informally at lucnhtime, where a few peers meet (discreetly) to swap ideas and suggestions. the aim is to bring to each session one difficulty and one resource or approach to ‘show off’. 

From HOD’s point of view, improving results may be focus. In most useful cases (ie not exam ‘strategy’) this will be via improved teaching. A good way to help with this is to cross reference performance management targets. This allows a department to set up mentors/mentees, so everyone makes the most of each other’s strengths. This could also work online.

He recommends Pimp Your Lesson and Ginnis’ Teachers’ Toolkit. In his setting this has led to a ‘focus of the week’.
 
Link this with lunch sessions? It sounds like the aim is to get preproom as supportive as twitter – imagine if you could share the same enthusiasm in there as we see on #asechat or at #SciTeachJC.
 
If department time is not focussed on teaching and learning, bin it. Requires brave approach and sadly works best coming from the top down. I would love to think that this could realistically include exams.
 
Refer back to test scores, using them as formative assessment for you as well as students. If kids underachieve, reflect on this and seek advice from collagues whose students have done particularly well. What have they done differently to you?

Identify and tackle groups that are not achieving:

  1. Classroom intervention (over time)
  2. Set or group (sessions, etc)
  3. Back to individuals, but led by department
Open doors, learning walks should have a specific focus. They are not just for SMT!
 
After reflection, I’m now going to give this a try in combination with the ‘focus of the week’ idea, perhaps modified to a different idea every fortnight. Each time I’ll choose an aspect I want to work on and see if I can observe colleagues for this. We don’t do enough peer observation in my workplace but things will only change if I try things out myself; we are fairly conservative. It might work best if I observe first, then try out myself.
 
He talked about cards providing quick checks for reflecting on oneaspect of a lesson (we have equivalents for the students, L2L cards which they use to prompt contributions to a split plenary). Use paired observations, self-moderating:

  • What went well?
  • Even better if?
I really like this language and will start to use it with the students.
 
Only do things in lessons that will allow/encourage kids to progress.
Always ask yourself: How will Monday be different?
 
Laura Monroe

Explained KS3/4 in Northern Ireland. Sounds much more open than in England, although a few ‘must cover’ areas, eg careers.

  • Do they know about careers in science?
  • What surprises them?
  • Links to syllabus
  • Minimum qualifications
Have kids research for marketplace, then give indirect questions. For some it will be obvious who to ask, some more tricky. Each group gets different questions to avoid collaboration.

Emphasis on women in biology, displays in corridor, people with Nobel prizes, local, inc current/recent. Use key words so kids can link to own learning. Include staff at school, hope to add ex-pupils in future. Steal this idea!

Have kids research then sum up an assigned scientist in a tweet. Have kids set success criteria before doing homework. Alternatively write obituary. Anything like this (careers, literacy, online skills) ticks a lot of outstanding boxes, esp if you start with a hook demonstrating local/contemporary relevance.

Write CVs for anything that shows adaptation – organisms, cells, organelles (eg red/white blood cells) – then kids compete. Judged by peers who therefore need to know both sides. e.g. 8 students, 4 present CVs for white blood cells while other 4 (red blood cells) mark their information, then swap. Worth stealing – pass on to Biology colleagues, add to KS3/4 SOWs.

Essays – write a paragraph, then spend lesson improving with markschemes, glossaries, peer checking, then redraft at end. Very good way to show AfL, and progress in lesson, would work well for split plenary. This isn’t too different from what I do now but would be a good way to practice use of checklists.

Friday afternoon [Subject] Resource pack, fun but relevant AS/A2 packages with markschemes. £90 makes this a department not a personal purchase. Learning without realising it, is best way. Q&A cards can be created and used in a similar way. An online version is ‘Ript’, free software. I’ve found several things that could be this, but there are loads of online flashcard systems available.

Complete Q loops by discussion and make an actual physical loop – much more interactive, especially for the first time. If you use a stopwatch, you can challenge students to beat their previous time. Because they get different cards each time, they gain familiarity quickly.

James O’Neill

Hates paper based activities, but can be useful to boost kids, if you can stop all the cards getting lost.

National Strategies still very useful, but you’ll have to look for them in archives. Especially worth checking out the Ped-Pack. Unfortunately the original resources were all really badly organised; teachfind or similar is a good ‘doorway’. I’m not so sure about this but it could be my misbehaving computer/internet connection.

Tea stained paper and magnets practical shows the field. Can’t find a link to this at the moment.

Emphasize to students that when filling in APP sheet it will be hit and miss, each activity will have a level and will not always meet target over 2/3 years. Conditional formatting in excel can automate feedback by pinpointing themes.

Use of red/orange/green cards for immediate feedback – can assign kids as troubleshooters, go over, stop lesson.

Taboo and wordslap activities are quick and easy to use. Put together powerpoint?

“Who doesn’t know?” – if they stay quiet, they are accepting that they should have an answer.

Use Blooms for objectives in lessons, check out digital version which includes podcasting/tweeting etc.

Ideas I want to try out in the next week/fortnight

  • Linking a ‘focus of the fortnight’ with my own observations of colleagues and trying out one new approach at a time.
  • Having students write CVs – I’m going to try it for power stations (year 10 revision for exam)
  • “Who doesn’t know?” as a way to involve quieter kids.

Getting Easy Marks (AQA P2 June 2011)

Ah, the indescribable joy of marking mock papers. It’s not all bad, admittedly – some of my students have done rather well. But as is my usual habit, one of the activities they’ll be using after the holiday is to look for the easy errors. These are, as I tell them, the marks that pretty much everyone doing higher tier should get most of the time. It’s a mixture of straightforward recall and simple methods, and it can be useful both for able students (who make careless errors just like everyone else) and those who are hoping to get a B grade rather than a borderline C (who can be reassured with these improvements).

Getting Easy Marks June2011 as ppt; click on the image above for the A4 pdf version.

How you approach this depends on individual style, but I like to have students add asterisks to their completed revision checklists (as seen in an earlier post) to highlight ‘fixable’ issues. I then have them taking turns teaching each other, as most have one or two questions they did very well on and which they can explain to their peers. For trickier questions, I often discuss examples of weak, reasonable and excellent answers, sometimes having them consider a flowchart of successful approaches. Rehearsing model answers can be particularly effective, for example in the regular ‘padding material in an impact situation’:

  1. Any kind of padding/crumple-zone/helmet/etc increases impact time…
  2. …so the rate of momentum change is less (using delta mv/t)…
  3. …which means the average force is reduced.

On the June 2011 paper this was on Question 7, but any similar question can be answered using this kind of generic answer. It would have to be better than the student in my class who suggested that an insulating material stops injury because of electric shock after electrons have been rubbed off by friction as the motorcyclist slides over the road surface. Really, I despair sometimes.

Students could look for revision material online to match their specific weaknesses (some links are described here) or write questions to test each other on particular areas. As well as doing more practice papers in exam conditions, I’ve blogged before about varied methods for using papers. It is perhaps a little late to start producing new mind maps and so on, but a short maths exercise in class will show students how quickly 20 minutes per day adds up to give a fair chance of gaining a half-dozen or so marks – and potentially several grades.

Last year’s version of this post (with the June 2010 paper similarly dissected) is here, along with my resources for students on the difference between foundation and higher tier. Hope some of it is useful; I’d love to hear how you’ve used the materials, if at all.