search

ggb / numeral-elm

Numeral.js port to Elm
MIT License
22 stars 9 forks source link

Inconsistent rounding #7

Closed pdamoc closed 8 years ago

pdamoc commented 8 years ago

I'm seeing an inconsistent rounding problem:

format "0,0.00" 2.385 == "2.38" but shouldn't it be "2.39" ? for example format "0,0.00" 28.885 == "28.89"

ggb commented 8 years ago

Hi Petre,

numeral.js seems to have the same inconsistency and I think it is caused by floating point arithmetic: To round the significant figures the input number is multiplied by 10^precision, rounded and divided by 10^precision; multiplying goes kind of wrong... 2.385 * 100 => 238.49999999999997, but 28.885 * 100 => 2888.5

I've found a simple fix for this problem and will republish the package soon.

Thanks, Gregor

pdamoc commented 8 years ago

wonderful! Thank you!