InseeFr / Trevas

Transformation engine and validator for statistics.
MIT License
9 stars 5 forks source link

Calculation accuracies #332

Open NicoLaval opened 1 month ago

NicoLaval commented 1 month ago

@noahboerger reported:

I think every language will give this kind of result. To be sure of the result, if a particular precision is needed, user can use round operator. @hadrienk, can you confirm?

noahboerger commented 1 month ago

I fully agree on this. This is not an issue on trevas, but more on the testcases we have been using. I think these should be adjusted to use the rounding operation where necessary.

hadrienk commented 1 month ago

@NicoLaval yes, rounding is the solution. This is a case of the good old 0.1 + 0.2 != 0.3. https://0.30000000000000004.com/