significantfigures and NumSigFigs

We are excited about the possibilities that are offered by Stack with Moodle. And we often use the commands "significantfigures" and "NumSigFigs" for the implementation of our physics exercises at Moodle with stack. Both commands have similar functionality, but are used differently.

If we take the possibility to enter the code in "feedback variables", we use the command "significantfigures".

Example: If (x = significantfigures (y, n)), then …

To meet the condition, x must have n significant digits and must also match y on n significant digits.

If we put the corresponding command "NumSigFigs" as an option in a Potential-Response-Tree, you can choose "[n, m]" as test options. So it is possible to require that x has n significant digits, but only m significant digits must match y.

We would also like to use this test option "[n, m]" from "NumSigFigs" for "significant figures" as well.

This test option is very important to us because the parameters used in physical questions are often of finite accuracy. Therefore the answers can be slightly different. The accuracy that is queried depends on the number of significant digits, but should be one digit (or sometimes two digits) less.

In principle we can solve all our problems with the command "NumSigFigs" using Potential-Response-Trees. But we have found that when using a large number of nodes in a row, the processing becomes incredibly slow (especially when programming, but also when the students use the test).

Therefore we have started to solve our problems using the alternative method using code in "feedback variables". To show the complexity of our tasks, I copied an example of our code in "feedback variables" below (but unfortunately without "[n, m]"-test option). If we implemented the same problem with Potential-Response-Trees, 9 nodes should be used – and the processing is slowed down considerably.

Is there a possibility to use the "[n, m]"-test option with the command "significantfigures"? We searched for a long time but found nothing. Or is there any other suggestion to solve this problem?

Regards Sophie

Example of our code in "feedback variables" (Sorry, feedback-text is in German):

if (unit=0) then (feedback: "Falsche Einheit; kann nicht geprüft werden" , x : 0) else if ((ans1 =erg1) or (ans1 = significantfigures(erg1,2)) or (ans1 = significantfigures(erg1,3))) then (feedback: "richtig" , x : 1) else if ((ans1 = significantfigures(erg1,4))) then (feedback: "richtig berechnet, aber mit zu viele Stellen angegeben" , x : 1) else if ((ans1 = significantfigures(erg1,5)) or (ans1 = significantfigures(erg1,6)) or (ans1 = significantfigures(erg1,7))) then (feedback: "richtig berechnet, aber mit viel zu viele Stellen angegeben" , x : 1) else if (numabsolutep(ans1,erg1,erg1*0.05)) then (feedback: "stimmt ungefähr, aber wahrscheinlich falsch gerundet, oder zu viel gerundet, oder viel zu wenig gerundet.") else (feedback: "falsch: falsch berechnet, falsch gerundet, viel zu viele Stellen angegeben oder vielleicht auch falsche Einheit angegeben." , x : 0);

Sorry if I am unable to answer the question itself but I'll just dump some technical details here that may help in this issue. Also I will give some information about what is coming in the not so near future:

The significantfigures()-function itself cannot do all the things that the PRT-level tests can do but your example code, if I understand it correctly, does actually have better logic than the internal logic for [n,m] where the accuracy is directly tested with absolute tolerances.
Your code will however have trouble with trailing zeros as it cannot detect the lack of them as it does not actually have access to the raw input from which to check for them, the PRT-level test however has a side-channel through which it can gain that information if the thing it tests is a raw input reference.
I assume you are aware of the [n,0] and [n,-1] forms of the tests? With them one can in two tests divide the answer to those that have less significant figures, those that have too many significant figures, and those that have something between. However, that testing will only test for presence of significant figures and will then need to be folowed with a third test to check the value for required accuracy. There also exits "Units" variants of these tests so if you include an unit in the answer and it is defined in the same (unit-type) input field then you can handle that first bit of your logic at the same time, but still three steps.
It would be difficult for NumSigFigs to start to test both accuracy and presence of digits at the same time just because then we would need to receive even more options into the test, while that would be technically possible I fear that it might be somewhat too difficult from the authors side. So should that kind of behaviour be implemented I would imagine that we would have a new test for it with different default assumptions. For authors it would always be simpler if we had a ready test that would just do everything so I would expect that if your use-case is general and Chris accepts that we could get a new test tailored to do what you do. But I am not quite sure about that yet.
As to performance, I am sorry but things will get worse before they get better here. For the last two years we have been rebuilding the internals of STACK to allow us to replace the evaluation of PRTs with something much faster and less sensitive to the number of nodes in the tree, but before we can replace the PRT evaluation logic there will be some interval during which the new internals will be in use and those new internals will be slower than the old ones. They will be slower when executing the old PRT evaluation logic but will be much faster with the new form, both forms will look the same to the author though.
It seems likely that once that PRT-evaluation logic gets its performance boost it will as a side-effect give the author access to tools that will let them ask for the number of significant-figures or decimal-places of any input. From anywhere in the PRT including the feedback-variables. That may then allow even more complex logic for when the standard test are not enough.

Chris will probably want a sample question to look at and I would like to have some more context as well:

Where do unit and ans1 variables come from in the code? What types of inputs and what is expected to be inputted into them.
If unit is a separate declaration of the dimension of a numeric ans1 then you might be able to have much simpler logic with the units related inputs and tests. Unless there is some reason for dividing them into separate inputs. In some cases you could simply split the unit out of the answer to map old logic to combined inputs but I cannot guess what the logic expected is.

Thank you for the fast and detailed answer!

I have concluded from your text, that it is not yet entirely clear where my problem/wishes lie. First I will go into the individual points of your answer and send the whole task that I showed as example in my question, as exported file in XHTML format. Then I would like to explain our specific application a little further.

2) The problem with trailing zeros is another, additional problem. It is also very interesting to find solutions to this.

3) I was not aware of the [n,0] and [n,-1] forms of the tests. We will check how far this form is useful for our application. The "Units" variant has only been known to us recently and we are still testing and implementing it.

4) A new test with different default assumptions sounds good.

5) I am happy to hear that work is being done on the performance. But in any case: Even if the calculations are sometimes slow, we are very happy about the possibilities that STACK offers with Moodle!

6) That also sounds very good.

Explanation of our specific application: In contrast to most tasks in mathematics, the numerical values used for tasks in physics are often provided with a certain degree of uncertainty. A simple example are "free-fall"-tasks (in Newtonian physics), that use the Earth's acceleration value. The numerical value used for g may be 9.8 m/s^2 or 9.81 m/s^2 or even 9.8127 m/s^2 (value for Berlin) or 9.80665 m/s^2 (conventional standard value, found on the English wikipedia-page). If we do not specify this value in the task text (which I do not want to do), then a certain bandwidth of values must be allowed for the verification with STACK. Nevertheless, the accuracy in the verification should be adapted to the number of significant digits (that means: if a student specifies 3 significant digits, the value should match 2 significant digits; if a student specifies 4 significant digits, the value should match 3 significant digits; ...).

We have many similar examples with "spongy" parameter values. If uncertainties come from different "spongy" parameter values in the same task, sometimes the demanded match is even (n-2).

I hope it has become clear what the problem is?!

The example I sent does not address this particular problem. It was chosen only to show how we replaced long Potential-Response-Tree by code in "feedback variables". If there are problems with the file format or any other questions on the example, please let me know.

example-physics-sk-HTW.txt

So I now think I know what you want. You wish to have something that has the following options:

Minimum number of significant-figures present
Maximum number of significant-figures present
How many of the least-significant digits can differ in the digits present i.e. absolute tolerance tied to the number of significant-figures detected in your textual example this would be 1 for allowing variations on the last one

Which of these would that one mean:

sans: 1.239 tans: 1.2345 test sans in tans +- 0.005 i.e. [1.229, 1.239]
sans: 1.239 tans: 1.2345 test sans in [1.230, 1.239] i.e. only that digit an never more changes the first one is the logic currently in play in various tests but there has been some question about the latter one related to rounding issues.

That is a new use-case, our physics people tend to require that the number of significant-figures matches a well defined number based on the significant-figures in the presented values in the question. For them the tolerances are always the same (well naturally tied to the teachers answers scale), but for you this would not be the case as you would tie the tolerance both to the teachers answer and the number of significant-figures present in the answer.

Given that requirement I fear that [n,0] and [n,-1] may not help that much, if you only wanted to test if the number of significant-figures is within a range they would help but as the tolerance need to be tied to that number the help they provide is rather minimal. Well [n,0] might help a little.

The trailing zeros (1.10000 vs. 1.1) problem will matter here as if they are not noted your use case will then increase the numerical tolerance quite a lot, which does not really matter but what matters is the checks against maximum and minimum number of significant-figures and those need to note the zeros. To do that one would need to have access to the raw text inputted and the existing SigFigs and DecimalPlaces tests do have that access and a new test could as well.

Unfortunately, that XHTML output format does not work with STACK, it leaves quite a lot of details out. If one wants to move STACK questions around one needs to use the Moodle XML format.

Sorry for using the wrong format! Now I send the example again in Moodle XML format (example-physics-sk-HTW.xml): example-physics-sk-HTW.xml.txt

You got that very well. Your explanation for what I want is just right! I'm excited to see if Chris develops anything.

Just to show the differenz to "your physics people ", I explain our application again in more detail: Often, we let the students look for values from the formulary. It is important to us that the students learn to adjust the number of digits to the accuracy. Sometimes it is their decision: They are allowed to use rounded parameters and give roughly rounded values of the results. But if they want to give more significant digits for the result, they should also use the corresponding accuracy for the parameters. And the devil is in the details, especially when rounding.

And also just to show, I am sending another example to demonstrate the complexity of another task we are dealing with (file: example-physics-laboratory-sk-HTW.xml ): We implemented the verification of the results of the student's lab in Moodle (including error calculation). For this purpose, the students must enter the measurement results in Moodle. The evaluation is then based on these measured values (not on predetermined values). Here, the requested number of significant numbers depends on the previously calculated error bar. example-physics-laboratory-sk-HTW.xml.txt

The first one could really benefit from the units input and units answer-tests, although if you wish to limit the answers unit to km/h and m/s as opposed to allowing everything including cm/h and km/ms, then you will need to check it separately, but that is not hard. The test would handle the unit conversions itself and check the value and significant-figures, but not with the style you would like.

That second one is impressive, I can imagine how authoring that array of inputs must have been hard work and the editor did not help there I would imagine. In any case somewhere in the future is a matrix input where one can add labels and specific unit or representation accuracy limits for each column and even allow the student to add rows if they so choose. That "data"-mode matrix input is in development especially for that use-case you demonstrated. Before we get this new input you will at least get the new compact type of validation messages that Chris recently added, those should make that behave better when students fill in he values, basically it would not draw those big boxes with the messages and instead would use smaller ones if you use the compact mode but it is not yet available in your STACK. data entry Naturally, extracting the number of significant-figures or decimal-places from each cell in that data-matrix will still be a hassle but we should be able to provide the tools for that and thus allow you to loop over it and check whatever interests you.

Our physics people tend to guide the students so that if they need to get constants from elsewhere they should use enough digits to not need to drop any digits present in the given values, but the approach where the student could actually use a constant that would have less accuracy than the provided values is something new and needs new tooling from STACK.

I'm sorry this issue has taken a long time to resolve.

Since dealing with trailing zeros requires the raw answer input, you can't deal with this programmatically in the feedback variables. This restriction is hard-wired for now and we don't have plans to fix it in the near future, sorry.

Actually, Chris, you can. Since the PRTs became compiled and all the significant figures tests started to count those digits in CAS, the PRT logic has needed to have the raw input strings there. Whether it is sane for users to access it is another matter, but it will be there if any part of the PRT logic needs it (i.e., certain tests are active or it is used in the code).

Simply check what the _INPUT_STRING variable contains. It is a map, and if you, for example, explore it by calling rawans1: stackmap_get_ifexists(_INPUT_STRING, "ans1"), you should be getting a string, which you can then throw to the function sig_figs_from_str(rawans1) to get another map. Naturally, this only works inside PRTs, and only valid inputs can be found from the map. Note that evaluating or execution of the string input is impossible and we provide very few parsing tools for even extracting numeric values from it.

Indeed @aharjula, so the question for us as developers is whether we want to document and support this as a feature in the future. Currently, we could change that design and not break too many questions!

Given that we will likely keep on supporting CAS side SigFigs testing forever, we will need to keep on transferring those string form inputs there anyway. The current transfer format is pretty handy so it is unlikely to change and while we might want to keep that _INPUT_STRING variable read-only do we really have any reason to turn it into a forbidden one?

We might as well fix that identifier and commit to any imaginable future PRT implementation having that same thing in it. Unlike those PRT-score and feedback variables that would allow zero-node PRTs, this is fairly universal and easy to understand. Those other variables, I do believe, should be kept somewhat hidden as mapping those over to any future logic might be harder.

We are currently only using it for decimal places and significant figures, but maybe people can come up with some more ideas if they have access to the raw digits through the input-string. Given the way even SigFigs have so many definitions, we will never satisfy every interpretation, so having the option to access the raw string and interpret that according to whatever local rules apply would probably be handy for some users.

maths / moodle-qtype_stack

significantfigures and NumSigFigs #555