search

MikeMcl / decimal.js

An arbitrary-precision Decimal type for JavaScript
http://mikemcl.github.io/decimal.js
MIT License
6.45k stars 475 forks source link

Rounding issue #178

Closed rain1 closed 3 years ago

rain1 commented 3 years ago

Hi!

I am looking for JS library that can do stuff precisely.

My problem is: d1=new Decimal(-399.711896245283);d2= new Decimal(1000);d1.add(d2).toNumber();

Will give me result 600.288103754717 which is wrong because when I add integer I expect decimal part to remain untouched. But with this example it changes from .7 to .2.

BigFav commented 3 years ago

Have you tried: d1=new Decimal('-399.711896245283');d2= new Decimal('1000');d1.add(d2).toNumber();

In my experience using strings tends to work better. JS numbers are inherently imprecise.

rain1 commented 3 years ago

Still same result.

MikeMcl commented 3 years ago

-9.8 + 10 = 0.2

rain1 commented 3 years ago

Yea, my bad.

munrocket commented 3 years ago

Hello @MikeMcl I am trying to extend JS language with fma/invSqr etc. https://es.discourse.group/t/math-float-optimisation/829/4 Can you comment something on this? It’s wort it? They recently created decimal-proposal which is probably too big dependency for browsers and it can be a userspace libraries.