Answer Classes Points specification.

[kohlhase]

I am currently working on answer classes for the KRMT exam. This gives me the opportunity to refine the discussion we had before; in particular the question of whether answer classes are exclusive or can be combined.

From my KRMT experience, we have two kinds of specifications we want there:

• Answer Classes: these fully describe the answer we see. They are exclusive.
• Answer Traits: these are partial descriptions of observable aspects of an answer. They can be combined.

As a consequence I think we need two differentformalizations.

A typical answer class is the answer ABD for a "DFS in a tree" question, a typical answer trait is something like forgotten a semicolon.

In sTeX we could just use the [pts=] key to indicate an answer class and introduce a new key [update=] key for answer traits the values of this are update specifications, e.g. -.5 or +1.

These keys would have to be mutually exclusive (maybe better have differen macros after all?) and VoLLKorn needs to do the appropriate thing.

[kohlhase]

Actually, we do not need different keys, just allow the value space of the [pts=] to be

• absolute values, e.g. [pts=3]
• updates, e.g. [pts=-.5] or [pts=+1].

VoLLKorn needs to do the sensible thing though.

We could also think about multiplicative updates, e.g. [pts=*.5]

[kohlhase]

One of the phenomenon I notice here is that answer traits and classes differ in the treatment of feedback.

• in answer classes, you have to give a feedback that is fully separate from the answer class specification
• in the answer traits, the feedback is typically almost the same as the trait specification (and can in the future maybe be derived from it given enough care in formulating the specification)

[lambdaTotoro]

We could also think about multiplicative updates, e.g. [pts=*.5]

Just as a comment on this, I think it would be smartest to remain commutative, so either + or * for answer traits.

[kohlhase]

We could also think about multiplicative updates, e.g. [pts=*.5]

Just as a comment on this, I think it would be smartest to remain commutative, so either + or * for answer traits.

That is of course right, and probably also associative :-)

[Jazzpirate]

it would remain commutative and associative anyway - pts=-.5 would be an abbreviation for pts=+(-.5); analogously for /. I don't think we want to allow for arithmetic expressions in the pts option, so that would be perfectly fine

[kohlhase]

Actually, we do not need different keys, just allow the value space of the [pts=] to be

the best thing about this is that we do not have to change the sTeX

[lambdaTotoro]

it would remain commutative and associative anyway

No. If you have an answer class that gives 3 points and two traits that add 0.5 and one that multiplies with 0.5, you have different results depending on the order you add the traits.

(3 + 0.5 + 0.5) 0.5 = 2 (3 0.5) + 0.5 + 0.5 = 2.5

It would only remain commutative as long as you have only + or only *, which was my point.

[Jazzpirate]

Ah, you're right, and that makes me realize there is a bigger issue that needs discussing: I implicitly assumed that the algorithm would start with the total points of the (sub)problem and answer classes subtract from that (or divide, or multiply), such that the final points are the result of successively modifying the total number of points - i.e. answer classes that do have pts describe mistakes. The alternative is starting with 0 and adding points for correct things - i.e. answer classes that do have pts describe successfully done things. The two are basically mutually incompatible, but both occur in practice

[kohlhase]

Yes, you are right there. I guess we will have to see how things work out with the KRMT exam, whether the subtractive approach (works OK). I would think the subtractive approach is natural, since it starts with the max-points specified in the {s,sub}problem environment

[Jazzpirate]

I mean, that would fit my grading style pretty well, but others may approach it differently. They are to some extent equivalent, I guess, but in my mind, the subtractive approach is shorter and more intuitive when thinkint of answerclasses as "mistake types", e.g.: What are the free and bound variables in \forall x. P(x,y)

We would either way have answer classes like:

• \anscls[pts=2]{Correct}
• \anscls[pts=0]{Missing}
• \anscls[pts=0]{Nonsense}

Florian would probably write a gnote like "1 point for the free variable, 1 point for the bound one". Which now could be written either as:

• \anscls[pts=+1]{x correctly identified as bound}
• \anscls[pts=+1]{y correctly identified as free}
• \anscls{x identified as free}
• \anscls{y identified as bound} ...or as:
• \anscls[pts=-1]{x mistakenly identified as free}
• \anscls[pts=-1]{y mistakenly identified as bound} ...and we don't need classes for the partially correct cases. Does that clash with someone else's intuition? Or are there examples where we need something more complicated at all?

(and in both cases, we would of course have ones for "free variables missing" and "bound variables missing" and...)

[Hundecode]

If I remember right we had this discussion a year ago. I think this is just a matter of terminology.

We decided to use the term 'Answer Classes' for what is called here as a 'Answer Trait' and use the term 'Feedback Class' what is called 'Answer Class' here. I don't see any advantage in changing the terminology other than confusion and confounding. Conceptually everything is the same

[Jazzpirate]

No, the distinction between "answer class" and "answer trait" is entirely orthogonal to that between "answer class" and "feedback class". In the latter terminology, both answer classes and answer traits are answer classes, and as before, a feedback class is some set of answer classes (and traits).