dimforge / alga

Abstract algebra for Rust.
194 stars 39 forks source link

Considering NaN, do floating point values form a lattice? #96

Open olson-sean-k opened 4 years ago

olson-sean-k commented 4 years ago

alga implements the Lattice trait for primitive f32 and f64 types while providing infallible pair-wise meet and join operations. This seems to interact poorly with NaNs, which do not compare with other floating point values regardless of what they encode (the result is always false with the sole exception of !=).

Given this comparison and some floating point value x, (NaN ∨ x) produces x while (x ∨ NaN) produces NaN, because NaN >= x and x >= NaN are never true. Considering that NaN has no ordering w.r.t. any elements in the set of floating point values, can there be a greatest lower bound or least upper bound?

If the intention is to simply propagate NaNs as a sort of special case (rather than using a fallible API via Option or Result), then perhaps the implementation should produce NaN if either operand is NaN. "Garbage in, garbage out" probably applies here as well. :-) Do you have any thoughts on this?