roo-rb / roo

Roo provides an interface to spreadsheets of several sorts.
MIT License
2.8k stars 502 forks source link

Incorrect formatted value for `0%`-formatted percentages due to floating point inaccuracy #547

Closed leboshi closed 1 year ago

leboshi commented 4 years ago

PR coming momentarily!

Steps to reproduce

See Gist at https://gist.github.com/leboshi/e0e6302b50eaef325744087048f16924; download and run self-contained (plus XLSX) test script.

Issue

In some very specific cases, floating-point storage inaccuracy causes :formatted_value to return an unexpected value; specifically, when 57% is stored as a 0%-formatted percentage, :formatted_value returns 56%. Ultimately, this is because the formatter implementation truncates instead of rounding:

Kernel.format('%d%%', number.to_f * 100)

In this specific case, 57% is stored as 0.56999999999999995. Multiplied by 100, that's 56.999999999999995. The %d token truncates that to 56%, whereas %.0f would correctly round to 57%.

(Possibly related to issue #384?)

System configuration

Roo version: 2.8.3

Ruby version: 2.7.2