Trig functions result in floating point error

probablykasper / cpc

Text calculator with support for units and conversion

https://crates.io/crates/cpc

MIT License

121 stars 14 forks source link

Open EsotericDryad opened 11 months ago

EsotericDryad commented 11 months ago

Description: using "pi" with trig functions results in close but wrong floating point results

Steps to reproduce: cpc 'sin(pi)' or cpc 'sin(pi/2)'

Expected result: 0 and 1 respectively

Actual result: 5.012535993076719585277299108368256E-35 and 1.000000000000000000000000000000001

probablykasper commented 11 months ago

Not really sure how to fix that, any ideas?

EsotericDryad commented 10 months ago

To prevent this from going stale I'll update, but I don't really have a simple and satisfying solution yet...

A few options are

pass the buck: submit a feature request to the decimal library to include features like constant::PI and sin(). Both are included for floats in the standard library, as well as other decimal libraries, so it's not too outlandish to request them in a numerical data-type library
use another decimal library. For example rust_decimal offers constants and trig functions... but this option probably requires a total rewrite
use f128 (or standard library floats) for trig and convert to d128 if possible? The drawback being the loss of precision as magnitude increases
use lookups for well-defined angles (i.e. multiples of pi/3 and pi/4)

probablykasper commented 10 months ago

I probably won't work on this, but would welcome a PR. fend is a great alternative to try out, trigonometry seems to work there

One thing about other decimal libraries is some have a much smaller range of possible numbers