Number entry part: 1e-12 wiggle room on non-decimals sometimes has no effect

numbas / Numbas

A completely browser-based e-assessment/e-learning system, with an emphasis on mathematics

http://www.numbas.org.uk

Apache License 2.0

199 stars 118 forks source link

Number entry part: 1e-12 wiggle room on non-decimals sometimes has no effect #919

Closed christianp closed 2 years ago

christianp commented 2 years ago

When the answer is precround(4^7.9,1) = 57052.4, the 1e-12 wiggle room added by NumberEntryPart.getCorrectAnswer has no effect: it's smaller than the underlying float can represent.

In JS: (57052.4 - 1e-12) == 57052.4 returns true.

I think that we instead need to change the 12th significant digit, instead of the 12th decimal place.

aarchiba commented 1 year ago

This may be related to #942 ?

christianp commented 1 year ago

@aarchiba no, I don't think this is related: the problem was that the wiggle room didn't scale with the given number. Your issue is asking for more precision than JS's number type can give, so you want to avoid the wiggle room entirely by using the decimal data type. I'll explain in detail in that issue.