numbas / Numbas

A completely browser-based e-assessment/e-learning system, with an emphasis on mathematics
http://www.numbas.org.uk
Apache License 2.0
205 stars 120 forks source link

Number part types unable to cope with small numbers #942

Closed aarchiba closed 2 years ago

aarchiba commented 2 years ago

I want to ask questions about numerical precision, and the Number part type is failing to recognize entered numbers or produce valid revealed answers.

For one question, the smallest acceptable answer is 1e-300, and the largest is 2.22e-16 (roughly epsilon for 64-bit floating-point). The question fails to accept 1e-100, it incorrectly accepts zero, and it reports "0e+0" as the "correct" answer when "Reveal" is pressed. I have set "Scientific" as the only acceptable number format.

The system also fails for the range 2.22e-16 to 4.45e-16, but seems to work for the range 2.22e-14 to 4.45e-14.

All of these are acceptable floating-point numbers in standard double precision.

aarchiba commented 2 years ago

I don't understand how gapfill types identify correct answers, so it's not clear whether they can be used for this; code parts could be made to work but are extremely awkward. Are there other ways to solve this? A numeric-range part type that simply checks whether the number is in the approved range? Accepting only "scientific" format ever?

Even if another part type exists, it would be valuable to have a warning that "number" part types don't actually check whether the value is in the specified range.

christianp commented 2 years ago

I think you're using the standard number data type, which uses JavaScript's Number object as its underlying representation. This is theoretically an IEEE754 double-precision float, so can't specify very small or very large numbers precisely.

Because the vast majority of uses of the "number entry" part type are for numbers in the middle-size range, calculated by several steps of arithmetic, the marking algorithm adds in wiggle room of ± the 12th significant figure automatically.

When working to high precision, you should use the decimal data type. This has a fixed precision for the mantissa, and can store an arbitrarily large exponent (in terms of bits used in the representation).

I've made an example question showing this, using the numbers you mentioned: https://numbas.mathcentre.ac.uk/question/129478/very-small-answer-for-a-number-entry-part/ For quick reference here: the minimum accepted value is dec("1e-300") and the maximum is dec("2.22e-16").

aarchiba commented 2 years ago

These numbers are limits of double-precision floating-point, and are by construction easily representable in that format. I think if the default number type does not function correctly when such numbers are supplied, it should warn the user.

I am also unclear on why wiggle room is added to a numeric range specification? Specifying a range easily allows wiggle room to be added (which I did).

Is the type used internally defined by the types of the variables at either edge of the range? Do decimal values get wiggle room added?

christianp commented 2 years ago

decimal values don't get wiggle room added.

The wiggle room is added to number values because it's very easy to construct the "same" number two ways that don't produce exactly the same float. For example, (1+1/7)*2 - 2 = 0.2857142857142856, while 2/7 = 0.2857142857142857.

When you're doing basic arithmetic, it's not reasonable to have to worry about and understand these kinds of errors. I believe that adding this wiggle room automatically is much better than asking authors to systematically add - 10^-12 and + 10^-12 each time they use a number entry part, or to think really hard about whether floating point error might accumulate in their calculations. (and whether the min and max values are the wrong way round, which would mean the plus and minus signs on the wiggle terms would have to be swapped)

For each of the minimum and maximum value considered separately, if the value used is a number type, it will have the wiggle room added before being converted to a decimal. If it's already a decimal, or something which converts automatically to a decimal such as a rational, then it's used as-is.