numbas / Numbas

A completely browser-based e-assessment/e-learning system, with an emphasis on mathematics
http://www.numbas.org.uk
Apache License 2.0
205 stars 120 forks source link

Floating-point error introduced in range with non-integer step size #989

Closed christianp closed 1 year ago

christianp commented 1 year ago

The expression dpformat(list(0.2..4#0.2)[6],20) should produce exactly 1.4, but instead it currently produces 1.40000000000000010.

Some avoidable accumulation of rounding error must be happening when computing the entries in the range.