search

numbas / Numbas

A completely browser-based e-assessment/e-learning system, with an emphasis on mathematics
http://www.numbas.org.uk
Apache License 2.0
202 stars 117 forks source link

Add option to warn when student uses unusual variable names in a JME part answer. #280

Closed christianp closed 11 years ago

christianp commented 11 years ago

Motivated by questions where the answer is something like sin(x*y). The student might write sin(xy), and xy is interpreted as a single variable.

The author should be able to give a list of 'expected' variable names for an answer to a question. If anything else is used, the student is shown a warning.

christianp commented 11 years ago

When the author doesn't give any expected variable names, should the default behaviour be to extract them from the correct answer, or not to do anything?

christianp commented 11 years ago

I've pushed this to master so you can try it at http://numbas.mas.ncl.ac.uk/question/228/issue-280/. Is it OK?

BillFoster commented 11 years ago

Needs careful testing - I have done some and there are issues, for example, the error message sometimes says that a variable is unexpected - but of course the student sees the variable in the question and it is clearly not unexpected from their point of view. See part 2 of http://numbas.mas.ncl.ac.uk/question/229/bills-copy-of-issue-280/ and answer 2x_y_z. More later.

From: Christian Perfect [notifications@github.com] Sent: 18 July 2013 15:38 To: numbas/Numbas Subject: Re: [Numbas] Add option to warn when student uses unusual variable names in a JME part answer. (#280)

I've pushed this to master so you can try it at http://numbas.mas.ncl.ac.uk/question/228/issue-280/. Is it OK?

— Reply to this email directly or view it on GitHubhttps://github.com/numbas/Numbas/issues/280#issuecomment-21187905.

christianp commented 11 years ago

"Unexpected" is from the point of view of the marking algorithm: it was expecting only to see x and y, and it saw z as well. What word or phrasing would you prefer?

The other option is to do it the other way round and give a list of variable names that should cause the warning to be shown when they're present, but writing out that list means you need to think of yx as well as xy, and then the combinatorial madness when there are more variables.

BillFoster commented 11 years ago

Not quite, it was expecting in part 2 to only see x , the error message is 'your answer was interpreted to use the unexpected variable name y' z is not mentioned. Also the answer is marked as correct and has the above message alongside it. Also if we do not include any expected variables, as is now the case in the new part 3 of the question, then x_y_z is marked as correct and no message. This is inconsistent so needs thinking about. (So each math question type has its own list of expected variables?)

Worth taking a little time to mull over this.

From: Christian Perfect [notifications@github.com] Sent: 18 July 2013 21:36 To: numbas/Numbas Cc: Bill Foster Subject: Re: [Numbas] Add option to warn when student uses unusual variable names in a JME part answer. (#280)

"Unexpected" is from the point of view of the marking algorithm: it was expecting only to see x and y, and it saw z as well. What word or phrasing would you prefer?

The other option is to do it the other way round and give a list of variable names that should cause the warning to be shown when they're present, but writing out that list means you need to think of yx as well as xy, and then the combinatorial madness when there are more variables.

— Reply to this email directly or view it on GitHubhttps://github.com/numbas/Numbas/issues/280#issuecomment-21212889.

christianp commented 11 years ago

In that part you've only put x as an expected variable name, so it's correctly saying y is unexpected.

Only the first unexpected variable is mentioned. I could make it list them all, but the warning popup box is only small and it might end up being hard to read.

I don't understand what you're trying to do with part 3. The correct answer is x*y*z, so x*y*z is marked correct. Why would there be a message? You haven't set any expected variables.

BillFoster commented 11 years ago

The problem is that in part 3 there are no expected variables, so the way I would read it is that they should be a warning message similar to that of part 2 saying that x, y and z are unexpected!

From: Christian Perfect [mailto:notifications@github.com] Sent: 18 July 2013 10:50 PM To: numbas/Numbas Cc: Bill Foster Subject: Re: [Numbas] Add option to warn when student uses unusual variable names in a JME part answer. (#280)

In that part you've only put x as an expected variable name, so it's correctly saying y is unexpected.

Only the first unexpected variable is mentioned. I could make it list them all, but the warning popup box is only small and it might end up being hard to read.

I don't understand what you're trying to do with part 3. The correct answer is x_y_z, so x_y_z is marked correct. Why would there be a message? You haven't set any expected variables.

— Reply to this email directly or view it on GitHubhttps://github.com/numbas/Numbas/issues/280#issuecomment-21217583.

christianp commented 11 years ago

You're right, it's a bad design I thought I could get away with. I'll add a toggle option for the check, and an empty list will mean any variable prompts the warning. On 19 Jul 2013 08:42, "Bill Foster" notifications@github.com wrote:

The problem is that in part 3 there are no expected variables, so the way I would read it is that they should be a warning message similar to that of part 2 saying that x, y and z are unexpected!

From: Christian Perfect [mailto:notifications@github.com] Sent: 18 July 2013 10:50 PM To: numbas/Numbas Cc: Bill Foster Subject: Re: [Numbas] Add option to warn when student uses unusual variable names in a JME part answer. (#280)

In that part you've only put x as an expected variable name, so it's correctly saying y is unexpected.

Only the first unexpected variable is mentioned. I could make it list them all, but the warning popup box is only small and it might end up being hard to read.

I don't understand what you're trying to do with part 3. The correct answer is x_y_z, so x_y_z is marked correct. Why would there be a message? You haven't set any expected variables.

— Reply to this email directly or view it on GitHub< https://github.com/numbas/Numbas/issues/280#issuecomment-21217583>.

— Reply to this email directly or view it on GitHubhttps://github.com/numbas/Numbas/issues/280#issuecomment-21235964 .

christianp commented 11 years ago

I've documented the expected variable names at http://numbas-editor.readthedocs.org/en/latest/question-parts.html#mathematical-expression. Should I close this now?

BillFoster commented 11 years ago

Yes.

From: Christian Perfect [mailto:notifications@github.com] Sent: 21 August 2013 11:03 AM To: numbas/Numbas Cc: Bill Foster Subject: Re: [Numbas] Add option to warn when student uses unusual variable names in a JME part answer. (#280)

I've documented the expected variable names at http://numbas-editor.readthedocs.org/en/latest/question-parts.html#mathematical-expression. Should I close this now?

— Reply to this email directly or view it on GitHubhttps://github.com/numbas/Numbas/issues/280#issuecomment-23006948.