Writing Questions

I came across an interesting problem recently. How should you mark a question that asks students to put a list of numbers in order?

Fawn Nguyen wrote a blog post about the initial problem. She gave the following as examples of solutions students may give and asked people to mark as they see fit.

I took the question at face value and had a look at what other people had said. Then I thought the following:

Assume that the person is arranging the numbers one by one so they look for the number they think is the smallest and they put that first. Then they pick a number that they think is bigger than that one but smaller than all the others. If this is the case then we really should be checking these pair wise, if there is one number that they have no idea where it goes, we shouldn't be penalising all the other positions if they put it randomly in the wrong place. So I think we could:

- Check each pair to see whether they are correctly ordered.
- Give a mark for each correct pair.

Going by what my gut feeling is these seem pretty good rankings.

Here are some scaled scores from other teachers against my scores: 

In all honesty I probably wouldn't give 7 points for a question like this and also the student would almost definitely get some credit even if they guessed. To account for that I could take those scores and subtract 4 from them. Which makes me look a little mean:

But then someone asked:

"What if someone exactly reverses the order, do they get a zero, or something else since this seems like a problem reading directions?"

Now I worry that if the question was in any way ambiguous, how can we know that we are checking knowledge of the thing that we are hoping to find out about? (Or are we coming up with an arbitrary number to go into a spreadsheet so that we can rank children and give managers a stick to beat teachers with...)

And reminded me of this controversial question:

I always take test questions structured like this as a dare. CANNOT?! We'll see about that. CAN TOO.

In other words, I hate this question design. pic.twitter.com/oNSWKaXyuL

— Tracy Johnston Zager (@TracyZager) January 25, 2018

So I think I would rewrite the question. 

What is it that we want to know?

The original question was about integers and fractions so perhaps we want to know some of the following things:

• Does this person know how to compare fractions with the same denominator?
• Does this person know how to compare fractions with the same numerator?
• Does this person know when a fraction is greater or less than 1?
• Can this person identify when 2 fractions are equal?
• Does this person know how fractions fit onto a number line?
• Does this person know what increasing/decreasing/ascending/descending means?

Here are a sequence of questions that I think might highlight whether there were misconceptions better than the original question. To be fair it may take longer to test and longer to mark but I think it would tease out the misconceptions better and the questions are less ambiguous:

These questions aren't perfect but I think we as teachers need to reflect on why we are asking particular questions and what we want to get out of the results. If we want to check for specific knowledge then we need to ask quite specific questions that will help us as teachers to understand where our learners are in their journey. 

If we want to spark debate or give context or develop understanding then we probably want a different type of question altogether. You might like to try this question about chocolate from NRICH or this question from Open Middle about comparing fractions on a number line.