Lacking precision when printing numbers

PaddiM8 / kalker

Scientific calculator with math syntax that supports user-defined variables and functions, complex numbers, and estimation of derivatives and integrals

https://kalker.xyz

MIT License

1.64k stars 74 forks source link

Lacking precision when printing numbers #14

Closed mulimoen closed 3 years ago

mulimoen commented 4 years ago

In the latest version on crates.io (0.2.0), and on the current master, there seems to be a problem when printing some numbers, such as

>> 1000.0 - 0.1
1000

It seems the intermediary is computed correctly, as this produces the correct result:

>> (1000.0 - 0.1)*10
9999

PaddiM8 commented 3 years ago

There has been rounding errors since I switched to rug, but it should be better now (only slight ones, I'm trying to find a solution for this as well).

PaddiM8 commented 3 years ago

Wow, I think I might have found a solution. Gonna have to test it a bit more, though.

>> Cu = 63.55
>> Cu
63.55
>> Cu / 2
31.775
>> 0.00053895398/0.0000000000001
5.3895398*10^9 
>> 0.00053895398/0.001
0.53895398
>> 35.348593498/10
3.5348593498
>> Γ(4.5)
11.6317283966
>> 2.3844389/1.348922
1.7676625483
>> 1000.0 - 0.1
999.9

PaddiM8 commented 3 years ago

Seems to be working, I'll close the issue for now. Should be published shortly.