FormulasQuestion / moodle-qtype_formulas

Formulas question type for Moodle
17 stars 30 forks source link

"One possible correct answer is:" #95

Closed dbauer-ets closed 11 months ago

dbauer-ets commented 1 year ago

Philipp,

To systematically say that there are several possible answers when there is only one, that is to say in the majority of cases, is a little misleading. I do not agree with the principle.

When displaying the correct answer automatically, only one answer is displayed. Saying that there are several possible answers and displaying only one is not logical.

Even when there are several possible answers, with certain problems, often it is not important to specify it. In these cases, the student is simply satisfied to get a correct answer.

In cases where it is important to specify that there are several possible answers, I would leave it to the teacher to do so in the wording of the question, and to confirm it in a tailor-made feedback.

In short, I suggest that we use exactly the same wording as in the other qtypes, that is to say "The correct answer is: {$a}", which seems to me sufficient and adequate in all cases.

It will also be necessary to modify the language packs of the other languages (French, Spanish, etc.). Can you do it? I think I know how to do it, if you want.

PhilippImhof commented 1 year ago

To systematically say that there are several possible answers when there is only one, that is to say in the majority of cases, is a little misleading. I do not agree with the principle.

I agree (with you).

When displaying the correct answer automatically, only one answer is displayed. Saying that there are several possible answers and displaying only one is not logical.

In many cases, it is not possible to display all answers, e.g. when we ask a student to determine a vector that is orthogonal to a given vector, we might specify a model answer, but the grading will be done based on the dot product being zero, so the question does not "know" the answers, it just checks whether the student's answer satisfies a certain condition.

Even when there are several possible answers, with certain problems, often it is not important to specify it. In these cases, the student is simply satisfied to get a correct answer.

In cases where it is important to specify that there are several possible answers, I would leave it to the teacher to do so in the wording of the question, and to confirm it in a tailor-made feedback.

In short, I suggest that we use exactly the same wording as in the other qtypes, that is to say "The correct answer is: {$a}", which seems to me sufficient and adequate in all cases.

My preferred way would be to write: Correct answer: {$a}. Note: Some questions can have multiple correct answers.

But as a compromise, I suggest the following: Model answer: {$a}. By leaving out the the, we do not imply the answer is unique. And calling it a model answer makes clear that this is the answer specified by the teacher.

It will also be necessary to modify the language packs of the other languages (French, Spanish, etc.). Can you do it? I think I know how to do it, if you want.

All those messages are automatically submitted to AMOS (Automatic Manipulation of Strings) upon release. They will then be translated by volunteers and/or the language pack maintainer. Plugins are not supposed to ship other languages than English, so we do not have to do anything. (I am an AMOS contributor, so I will probably do the German translation.)

PhilippImhof commented 1 year ago

To add one more possibility: we could add an option in the edit form

If the teacher activates that option, the text is

Model answer: ... Other answers are possible.

And otherwise, it is just

Correct answer: ...

The default would be to leave the option unchecked. That would also be applied to all existing questions, because -- as you said -- most questions probably have a single correct answer.

dbauer-ets commented 1 year ago

Say the question is "Give a number between 1 and 4" (or a more complex question of the same form). Then, 2 and 3 are correct answers. There is no "model" answer. In this case the wording "One possible answer is:" would have been a better wording, but yet this would not be quite satisfactory.

In the majority of cases the wording "The correct answer is:" is adequate. Don't forget that, contrary for example to the multiple choice question, the fact that there is may be several possible answers is never announced in the Formulas question and only one correct answer is given even if there are many. I don't suggest we go that way. The point is that if the teacher feels it is important to announce that there may be multiple correct answer, it is up to him to say it in the question text. Also in that case, the teacher will probably want to give explanations in the feedback and he will not use the default correct answer wording.

So, I would just put "The correct answer is" which fits most cases. When this is not appropriate, the teacher can use a custom feedback and explain that there are infinite correct versions (e.g. vectors), or that the possible answers are 2, 3, etc.

In summary:

  1. "The correct answer is:" will fit the majority of cases. This is the only wording I would keep for the default "Correct answer" wording.
  2. "One possible answer", "Model answer", etc. are cases where the teacher would want to provide a more detailed feedback and not use a default wording. For example, if the problem is "Give a number between 1 and 4", none of the wording "Correct answer", "Model answer", "One possible answer" is adequate. In this case, the teacher should give a detailed feedback saying that the possible answers are 2 and 3. He could not use any default wording.
dbauer-ets commented 1 year ago

The Formulas question accepts 1/2, 2/4, 3/6 as correct answers. However the correct answer is 1/2. Take any math or physics textbook, and you will see that the correct answer is 1/2, without exception.

dbauer-ets commented 1 year ago

In many cases, it is not possible to display all answers, e.g. when we ask a student to determine a vector that is orthogonal to a given vector, we might specify a model answer, but the grading will be done based on the dot product being zero, so the question does not "know" the answers, it just checks whether the student's answer satisfies a certain condition.

I would say not too many cases. Anyway, in the case that you bring up, the verification is done using a grading criterion. The Part's answer is a dummy value, i.e. there is no correct answer, so the teacher can not use "Right answer" in the Review options and any default wording can not be used.

My preferred way would be to write: Correct answer: {$a}. Note: Some questions can have multiple correct answers.

This still sounds a bit confusing. The Number answer type is probably used 99 % of the time and has only one correct answer.

But as a compromise, I suggest the following: Model answer: {$a}. By leaving out the the, we do not imply the answer is unique. And calling it a model answer makes clear that this is the answer specified by the teacher.

This still sound a bit confusing 99% of the time when there is just one correct answer.

To add one more possibility: we could add an option in the edit form This question has multiple answers If the teacher activates that option, the text is Model answer: ... Other answers are possible.

Still confusing and not required in many cases because special cases are treated with a Grading criterion. In those cases, the teacher needs to use a "Specific feedback" anyway.

And otherwise, it is just Correct answer: ...

We should use "The correct answer is:". Even if there are many possible answer, the model answer can be called the correct answer, it is not a problem. For example, √2 - 1, 1/(1 + √2) and 0.41421356 would be equally accepted. However, there is nothing wrong to say that √2 - 1, the simplified form, is the "Correct" answer. Nobody will complain even if 1/(1 + √2) and 0.41421356 are also correct answers.

PhilippImhof commented 1 year ago

The Formulas question accepts 1/2, 2/4, 3/6 as correct answers. However the correct answer is 1/2. Take any math or physics textbook, and you will see that the correct answer is 1/2, without exception.

I'll take this one first, because it is not related to the issue at hand.

There is basically nothing we can do about that, because 1/2, 2/4 and 3/6 all evaluate to 0.5 (plus/minus some small amount due to the way floating point numbers work). Also, just to do some nitpicking, 2/4 might be the correct answer in some cases, e.g. when the question is to bring the fraction 1/2 to the same denominator as 3/4, which one might ask younger students who are about to start learning how to add fractions.

One solution could be to add a "fraction" answer type. In that case, the teacher would need an option to specify the penalty for fractions that are not in lowest terms, because 2/4 is not totally wrong, it's just not perfectly simplified. This, however, would add to complexity in the sense that we would not just add an answer type, but also introduce options that depend on the answer type.

When I need this and want fractions to be in lowest terms, I use two input fields and check for the gcd being 1 in the grading criterion:

simulated fraction with two input fields

We cannot offer a general solution to this. Also, where would we stop? Should 1/sqrt(2) be wrong, because fractions should not normally have an irrational denominator? What about sqrt(12) which, in its normal form, would be written as 2*sqrt(3)?

PhilippImhof commented 1 year ago

Say the question is "Give a number between 1 and 4" (or a more complex question of the same form). Then, 2 and 3 are correct answers. There is no "model" answer. In this case the wording "One possible answer is:" would have been a better wording, but yet this would not be quite satisfactory.

I see nothing wrong with the wording "One possible answer is:", even if it would probably be nicer to indicate all possible answers, but that is often not possible and sometimes it is possible, but not desirable.

Also, you are right about the fact that there is no "model" answer in this case. However, in many cases there is a model answer in the sense that this is the easiest or most elegant solution, e.g. when asking for a point on the curve of f(x) = 2x^2 + 4x - 1, the easiest answer would be (0,-1) because of the y intercept.

In the majority of cases the wording "The correct answer is:" is adequate. Don't forget that, contrary for example to the multiple choice question, the fact that there is may be several possible answers is never announced in the Formulas question and only one correct answer is given even if there are many.

I am not sure it is the majority of cases. We should not forget the "algebraic formula" answer type where there is almost never a unique answer. Also, the fact that there may be several possible answers can be very well announced explicitly or implicitly in the question, depending on how the teacher formulates it, e.g. "Find a solution to ..." implies that there might be many. It might also be clear from the context, e.g. "Find an eigenvector for …" where every student should know that all multiples of one eigenvector are also eigenvectors.

I don't suggest we go that way. The point is that if the teacher feels it is important to announce that there may be multiple correct answer, it is up to him to say it in the question text. Also in that case, the teacher will probably want to give explanations in the feedback and he will not use the default correct answer wording.

I rarely use the feedback for that purpose, though.

So, I would just put "The correct answer is" which fits most cases. When this is not appropriate, the teacher can use a custom feedback and explain that there are infinite correct versions (e.g. vectors), or that the possible answers are 2, 3, etc.

We currently do have a solution that is not satisfying, because it is confusing in some cases, namely in all cases where there is a unique answer. If we change the wording to "The correct answer is …", it becomes confusing in other cases, namely in all cases where there are multiple correct answers. I don't think that is a good thing. If we change it, we should find an improvement that is not a worsening for many other cases.

PhilippImhof commented 1 year ago

In many cases, it is not possible to display all answers, e.g. when we ask a student to determine a vector that is orthogonal to a given vector, we might specify a model answer, but the grading will be done based on the dot product being zero, so the question does not "know" the answers, it just checks whether the student's answer satisfies a certain condition.

I would say not too many cases. Anyway, in the case that you bring up, the verification is done using a grading criterion. The Part's answer is a dummy value, i.e. there is no correct answer, so the teacher can not use "Right answer" in the Review options and any default wording can not be used.

Well I do use it and I put a valid answer in the box, knowing that the student will see: "One correct answer is: …" If possible, I put the most elegant or easiest answer in the box.

My preferred way would be to write: Correct answer: {$a}. Note: Some questions can have multiple correct answers.

This still sounds a bit confusing. The Number answer type is probably used 99 % of the time and has only one correct answer.

I think it is difficult to make such assumptions. Also, even with numeric answer types, there can be multiple solutions, e.g. in linear algebra or analytic geometry.

But as a compromise, I suggest the following: Model answer: {$a}. By leaving out the the, we do not imply the answer is unique. And calling it a model answer makes clear that this is the answer specified by the teacher.

This still sound a bit confusing 99% of the time when there is just one correct answer.

As I said, it's a compromise. Saying "One correct answer" is confusing when there is only one. Saying "The correct answer" is confusion when there is more than one. My compromise mentions that there might be other solutions, but it does it does neither imply that there are nor that there aren't.

[My suggestion about offering a config option.] Still confusing and not required in many cases because special cases are treated with a Grading criterion. In those cases, the teacher needs to use a "Specific feedback" anyway.

I almost never use the specific feedback and use the grading criterion a lot. You once said that the folks who use our plugin are used to complex stuff. If we want the feedback to be accurate in all cases, it must be on the teacher to decide what wording to use, no?

We should use "The correct answer is:".

I don't mind whether we use "correct" or "model". What I don't like is the word "the", because it implies the answer is unique. If we decide to change the wording, because we don't like the fact that it implies multiple solutions even when there is only one, we should not change it to something that implies the opposite IMHO.

Even if there are many possible answer, the model answer can be called the correct answer, it is not a problem. For example, √2 - 1, 1/(1 + √2) and 0.41421356 would be equally accepted. However, there is nothing wrong to say that √2 - 1, the simplified form, is the "Correct" answer. Nobody will complain even if 1/(1 + √2) and 0.41421356 are also correct answers.

That's for cases where one correct answer is clearly better than others. But that is not always the case. Oftentimes, the various correct answers are all equally good.

dbauer-ets commented 1 year ago

The Formulas question accepts 1/2, 2/4, 3/6 as correct answers. However the correct answer is 1/2. Take any math or physics textbook, and you will see that the correct answer is 1/2, without exception.

I'll take this one first, because it is not related to the issue at hand. ....

This is a simple example, maybe not a good one because it has nothing to do with fractions but just different forms of an expression. My point is that although Formulas can accept 2/4, 3/6, etc. as correct answesr, there is nothing wrong is saying that the correct answer is 1/2 or in fact, as you point out, that the correct answer is 0.5. You don't need to indicate each time that there are other possible answer such as 2/4, 3/6, or that 0.5 is a model answer. Everybody understand when we say that the simplified version of an expression is the correct answer.

dbauer-ets commented 1 year ago

e.g. when asking for a point on the curve of f(x) = 2x^2 + 4x - 1, the easiest answer would be (0,-1) because of the y intercept.

It is better to use specific examples in this conversation.

So in this specific example, how would you set up the question in order to accept any point on the curve? Answer: with a grading criterion. In this case, the part's answer is a dummy value and the Right answer "One possible feedback" has absolutely no meaning. The teacher has to use a "Specific feedback" to explain the correct answer.

dbauer-ets commented 1 year ago

I am not sure it is the majority of cases. We should not forget the "algebraic formula" answer type where there is almost never a unique answer. Also, the fact that there may be several possible answers can be very well announced explicitly or implicitly in the question, depending on how the teacher formulates it, e.g. "Find a solution to ..." implies that there might be many. It might also be clear from the context, e.g. "Find an eigenvector for …" where every student should know that all multiples of one eigenvector are also eigenvectors.

Again, I think it is more conventional and much clearer to say that "The correct answer is:" and display the 'simplified' form of an expression, the 'normalised' form of a vector or matrix, etc.

In my opinion, to say "One possible solution.." is a bad shortcut. These sorts of questions require more detailed feedback explanations.

So the teachers should use "The correct answer is:" if they think it is appropriate or, if not, they should use a Specific feedback.

"One possible answer", "Model answer", etc. adds to the complexity of the question form, has limited use, can be confusing to students, and is a bad shortcut to giving a proper specific feedback.

dbauer-ets commented 1 year ago

I rarely use the feedback for that purpose, though.

That's what it is for. You should use it.

PhilippImhof commented 1 year ago

This is a simple example, maybe not a good one because it has nothing to do with fractions but just different forms of an expression. My point is that although Formulas can accept 2/4, 3/6, etc. as correct answesr, there is nothing wrong is saying that the correct answer is 1/2 or in fact, as you point out, that the correct answer is 0.5. You don't need to indicate each time that there are other possible answer such as 2/4, 3/6, or that 0.5 is a model answer. Everybody understand when we say that the simplified version of an expression is the correct answer.

I understand better what you wanted to say. I thought you were criticising the fact that 2/4 is accepted even if in most settings, one would probably make a deduction, because the student should write 1/2.

e.g. when asking for a point on the curve of f(x) = 2x^2 + 4x - 1, the easiest answer would be (0,-1) because of the y intercept.

So in this specific example, how would you set up the question in order to accept any point on the curve? Answer: with a grading criterion. In this case, the part's answer is a dummy value and the Right answer "One possible feedback" has absolutely no meaning. The teacher has to use a "Specific feedback" to explain the correct answer.

While it is indeed possible to set up a pure dummy answer like [0,0], there is no need to do so and I strongly discourage doing so whenever I explain people how to use the Formulas Question plugin. I always put a meaningful model answer, exactly because I know that the answer will be shown to the student. And in such cases, the fact that the message does not pretend that my answer is the only valid solution, saves me from setting up a specific feedback. I appreciate that and I would not want to miss it, because I would have to search through a somewhat large collection of questions and update them.

I am not sure it is the majority of cases. We should not forget the "algebraic formula" answer type where there is almost never a unique answer. Also, the fact that there may be several possible answers can be very well announced explicitly or implicitly in the question, depending on how the teacher formulates it, e.g. "Find a solution to ..." implies that there might be many. It might also be clear from the context, e.g. "Find an eigenvector for …" where every student should know that all multiples of one eigenvector are also eigenvectors.

Again, I think it is more conventional and much clearer to say that "The correct answer is:" and display the 'simplified' form of an expression, the 'normalised' form of a vector or matrix, etc.

In analytic geometry, there is often not 'the normalised' answer. When students have to write the equation (vector form) of a line through two points A and B, they are free to choose the "position vector". Most people would agree that r_A or r_B are the most reasonable choices, but they are both equally good. (The position vector to any other point on the line would fit, but I agree that would be like writing sqrt(4)instead of 2.) They then have to indicate the "direction vector" which could be r_A - r_B or r_B - r_A or any multiple thereof. In cases where all components of the direction vector have a common divisor > 1, it would be reasonable to simplify the vector, so that would make four absolutely reasonable solutions plus infinitely many less reasonable ones.

In my opinion, to say "One possible solution.." is a bad shortcut. These sorts of questions require more detailed feedback explanations.

The thing is that the current wording makes it possible to not use detailed feedback. Some people might rely on that.

So the teachers should use "The correct answer is:" if they think it is appropriate or, if not, they should use a Specific feedback.

I do not think it is our role to tell the people how they should use our plugin. We should not force our ways of doing stuff upon them. Even less so, if there is already a disagreement among us.

"One possible answer", "Model answer", etc. adds to the complexity of the question form,

Changing the message from "One possible answer" to "Correct answer" or "Model answer" does not add any complexity. Nor does adding a note saying that other answers may be possible. We would add complexity to the form by adding an option where the teacher can choose among both variants, but that would be minimal complexity.

has limited use, can be confusing to students,

"The correct answer is" also has limited use and can also be confusing to students, as shown above.

and is a bad shortcut to giving a proper specific feedback.

Again, I do not like the idea of judging how people make usage of our plugin.

We both agree that the current wording is not ideal and we both want to improve it. What we don't like about the current wording is that it implies something that is not true for many questions. IMHO we should not replace it by a wording that implies the opposite and, therefore, is not appropriate for many other questions. So let us find a better, more neutral wording.

dbauer-ets commented 1 year ago

We currently do have a solution that is not satisfying, because it is confusing in some cases, namely in all cases where there is a unique answer. If we change the wording to "The correct answer is …", it becomes confusing in other cases, namely in all cases where there are multiple correct answers. I don't think that is a good thing. If we change it, we should find an improvement that is not a worsening for many other cases.

Obviously, we don't agree. I think the opposite of you.

dbauer-ets commented 1 year ago

Well I do use it and I put a valid answer in the box, knowing that the student will see: "One correct answer is: …" If possible, I put the most elegant or easiest answer in the box.

It would be much better not to use it and put the appropriate answer in a specific feedback.

dbauer-ets commented 1 year ago

I think it is difficult to make such assumptions. Also, even with numeric answer types, there can be multiple solutions, e.g. in linear algebra or analytic geometry.

Not too difficult. 99% of the people use only T/F and MCQ. This is a fact. (Don't repeat this, but it is most probably because the other types of questions are too difficult for them to set up).

I would bet that the number answer type is used 99% of the time. That does not mean that you are not in the 1%.

Again, use a 'normalised' vectors as the correct answer. If this is not satisfactory to you, use a Specific feedback.

We should not impose our specific preferences on the development of the Formulas question, but think of what is efficient, logical, convenient for the majority and allow for specific requirements. In this case, the specific feedback is totally adequate. Adding the wording "One possible answer is" is a bad shortcut and should not appear in the form.

dbauer-ets commented 1 year ago

I think it is difficult to make such assumptions. Also, even with numeric answer types, there can be multiple solutions, e.g. in linear algebra or analytic geometry.

Not too difficult. 99% of the people use only T/F and MCQ. This is a fact. (Don't repeat this, but it is most probably because the other types of questions are too difficult for them to set up).

I would bet that the number answer type is used 99% of the time. That does not mean that you are not in the 1%.

Again, use a 'normalised' vectors as the correct answer. If this is not satisfactory to you, use a Specific feedback.

We should not impose our specific preferences on the development of the Formulas question, but think of what is efficient, logical, convenient for the majority and allow for specific requirements. In this case, the specific feedback is totally adequate. Adding the wording "One possible answer is" is a bad shortcut and should not appear in the form.

dbauer-ets commented 1 year ago

I think it is difficult to make such assumptions. Also, even with numeric answer types, there can be multiple solutions, e.g. in linear algebra or analytic geometry.

Not too difficult. 99% of the people use only T/F and MCQ. This is a fact. (Don't repeat this, but it is most probably because the other types of questions are too difficult for them to set up).

I would bet that the number answer type is used 99% of the time. That does not mean that you are not in the 1%.

dbauer-ets commented 1 year ago

oooooooooooooooooooooooooooooooooooooo

So we disagree. Some people teach math, others teach physics, engineering, etc. We all have different requirements. I teach in engineering. I can assure you that in all my quizzes, the answer "One possible answer", or any other variation, makes no sense.

Here is what I suggest. Let the teacher decide.

In the same way that a default formulation is proposed for the combined feedbacks "Your answer is correct", "Your answer is partially correct" and "Your answer is incorrect", which the teacher can easily modify, why not add a "Right answer" field in the question form that the teacher can edit and put whatever they want in it. The contents of this field would override the Right answer message of the quiz's Review options.

Also, the default wording could be specified in the plugin settings.

This would also have the advantage that a different right answer message could be given for each part of the question, instead of being limited to the same message for the whole quiz.

What do you think ?

PhilippImhof commented 1 year ago

Here is what I suggest. Let the teacher decide.

Yes, that's a good idea and was also among my suggestions.

In the same way that a default formulation is proposed for the combined feedbacks "Your answer is correct", "Your answer is partially correct" and "Your answer is incorrect", which the teacher can easily modify, why not add a "Right answer" field in the question form that the teacher can edit and put whatever they want in it. The contents of this field would override the Right answer message of the quiz's Review options.

Also, the default wording could be specified in the plugin settings.

This would also have the advantage that a different right answer message could be given for each part of the question, instead of being limited to the same message for the whole quiz.

IMHO that's overkill. There are not a thousand ways to introduce the correct answer(s), so I'd say it is enough if the teacher can choose between the existing "One correct answer is: …" and "The correct answer is: …". If they want more than that, they are free to use the existing feedback fields.

The advantage of this approach is that it just adds one checkbox below the answer field, whereas a fully customised message would need a textarea and thus take up more space in an edit form that is already very full.

I'd further suggest that we set the default for new questions to "The correct answer is: …", but we leave it at "One correct answer is: …" for existing questions where the option is not set.

PhilippImhof commented 1 year ago

Voilà my suggestion:

example for new option

The text can probably be improved.

sangwinc commented 1 year ago

I don't normally comment here, because as the lead developer of the STACK plugin (qtype_stack) I have my hands full looking after that! But I hope these observations of the thread are relevant and helpful.

(1) In STACK we don't use "One possible answer is:" we use "The teacher's answer is:" to avoid the confusion in both directions.

(2) The Formulas question accepts 1/2, 2/4, 3/6 as correct answers. In the STACK system the teacher's responsibility (not the system designer note!) is to specify the properties that characterises correctness. If "algebraic equivalence" is the only property they all will be considered correct, along with 0.5, sum(1/2^k,k,1,infinity), .... So, we need a slightly stricter equivalence class, or perhaps we really only accept 1/2. It is also up to the teacher to decide what is, and is not, worth partial credit (and how much). Saying "take any math or physics textbook, and you will see that the correct answer is 1/2, without exception." would be too easy to refute to bother doing here. In any case, the individual teacher is responsible for what happens in their own class.

In my experience it's very rare to only accept one exact expression. E.g. you would probably want to accept both -1+x and x-1. I have found that "Equality up to Associativity and Commutativity" is very often needed (rather than algebraic equivalence), and this is explained here: https://docs.stack-assessment.org/en/Authoring/Answer_tests_rules_based/

(3) "The Number answer type is probably used 99 % of the time and has only one correct answer.!" Even when you have numbers, and want an exact expression, there is very often more than one acceptable answer. For example, you might want to only accept the first of these, but what about the second and third? [ 3-2\sqrt{2},\quad -2\sqrt{2}+3,\quad 3-\sqrt{8},\quad (\sqrt{2}-1)^2,\quad \frac{\sqrt{2}-1}{\sqrt{2}+1},\quad \frac{1}{(\sqrt{2}+1)^2},\quad \frac{1}{3+2\sqrt{2}}. ] Similar things happen to complex numbers. Students type things like 2+1i since they have been told it should be a+bi. I will always accept 2+i and 2+1*i. I might give feedback, but in my class, both are fine. My point stands: there is normally more than one acceptable answer.

(4) You rightly note there are differences between disciplines. The history of development of rules for significant figures is a really interesting case study of how consensus develops. @ARTICLE{Carter2013, author = {Carter, A.~R.}, title = {Evolution of the Significant Figure Rules}, journal = {The Physics Teacher}, year = {2013}, volume = {51}, pages = {340--343}, month = {11}, doi = {10.1119/1.4818368}, } We had lots of discussion, and uncovered some inconsistency and disagreement, in developing significant figure testing rules for STACK.

@christianp what do you do in NUMBAS?

I hope this helps!

Chris

PhilippImhof commented 1 year ago

Thanks very much for your insights, Chris.

(1) In STACK we don't use "One possible answer is:" we use "The teacher's answer is:" to avoid the confusion in both directions.

I had thought about that one, but I first wanted to make sure the plugin could determine the role name for "teacher" in case it had been overridden in the course settings.

(2) The Formulas question accepts 1/2, 2/4, 3/6 as correct answers. In the STACK system the teacher's responsibility (not the system designer note!) is to specify the properties that characterises correctness.

Funny that you are pointing that out, because when I wrote my answer, I first wanted to say: if teachers need more sophisticated rules, e.g. to allow 1/2 but not 2/4, they might need to use STACK instead. :)

My point stands: there is normally more than one acceptable answer.

I fully agree. I can, however, understand Dominique's point that -- let's take 2+sqrt(3) as an example -- it is fine to say this is the answer and leaving it implied that sqrt(3)+2 is also correct. For me, the distinction between the correct answer and one correct answer is mainly important when there are multiple answers which have a "true" difference, e.g. an eigenvector.

dbauer-ets commented 1 year ago

Hello Chris,

Thank you for taking the time to join the conversation. :-)

Dominique

dbauer-ets commented 1 year ago

I'd further suggest that we set the default for new questions to "The correct answer is: …", but we leave it at "One correct answer is: …" for existing questions where the option is not set.

If this is possible, then we have found an agreement.

PhilippImhof commented 1 year ago

Eureka! :-)

dbauer-ets commented 1 year ago

The text can probably be improved.

Sounds good to me.

Shouldn't there be a capital t: "There are..."

christianp commented 1 year ago

@sangwinc are you asking specifically about how Numbas marks questions where the answer is a number? The standard behaviour is that the question author gives a range of acceptable values, and any value in that range is marked correct. These are called "minimum accepted value" and "maximum accepted value". There are options to add further constraints on the answer: must be an integer, a fraction, must be reduced, given to however many decimal places or significant figures, and so on. The student gets feedback like "your answer is correct", and if any partial marks are awarded, or penalties applied, then they're explained in the feedback section. When a correct answer is shown, it's labelled "Expected answer". I think this is nicely ambiguous about whether there's a single correct answer, or if there are several.

dbauer-ets commented 1 year ago

Christian,

Thank you for contributing to this conversation.