search

numbas / Numbas

A completely browser-based e-assessment/e-learning system, with an emphasis on mathematics
http://www.numbas.org.uk
Apache License 2.0
198 stars 118 forks source link

Floating-point error can be introduced when restoring variable values from suspend data #988

Closed christianp closed 1 year ago

christianp commented 1 year ago

We had a report of a question where a variable is displayed as an integer when the attempt is first run, but has .000000001 on the end when the attempt is resumed in a new session. There must be some error in the rendering of the number in the SCORM suspend data.

christianp commented 1 year ago

A minimal example: define a variable a = 0.2 + 10*0.01. Its value is saved in the suspend data as "0.30000000000000004", but it should be "0.3".

christianp commented 1 year ago

I wondered if this was a regression. It seems not: this happens in commits going back to 2015, and then I stopped checking!

ugoertz commented 1 year ago

This reminds me of point (2) in https://github.com/numbas/Numbas/issues/774 (but could also be entirely unrelated). (I did not really follow up on that issue at the time since I switched to the Numbas LTI provider.)

christianp commented 1 year ago

This is a regression! I just wasn't testing the right thing.

The problem is that number literals now get precision and precisionType attributes, which the display code uses to decide how many digits to display. Numbers obtained through calculation, like random(range), don't get these attributes, so the display code just uses its default behaviour, which is to round to 10 decimal places.

Variables are saved to suspend data as JME expressions which should produce exactly the same value. I think that for number values, we need to make sure to store the precision information (or lack of it). Should treeToJME when called with nicenumber: false wrap numbers without precision attributes in a function called something like noprecision(...)?