rekka / meval-rs

Math expression parser and evaluation library for Rust
The Unlicense
153 stars 30 forks source link

f64 precision problem #31

Closed whsqjss closed 11 months ago

whsqjss commented 11 months ago
(2.27 - 2) / 3 should be 0.09, but got 0.09000000000000000001

Is there a solution to this problem? Looking forward to your reply. Thanks err

rekka commented 11 months ago

This is a limitation of how computers handle floating point numbers so there is not much that can be done about this if we want to use the fast hardware floating point operations. See for example Python documentation: Floating Point Arithmetic: Issues and Limitations

You can try the same expression in Rust directly:

fn main() {
    let x = (2.27 - 2.) / 3.;
    println!("{x}");
    let x = 0.3 * 3.;
    println!("{x}");
}

This prints

0.09000000000000001
0.8999999999999999

Hope this helps.