purescript / purescript-prelude

The PureScript Prelude
BSD 3-Clause "New" or "Revised" License
161 stars 88 forks source link

`ordNumber` is not compatible with IEEE 754 floats #306

Open anttih opened 1 year ago

anttih commented 1 year ago

purs compiles the following:

(1.0 < nan) == false

to

1.0 < nan === false

but if we instead go through the Ord API we can observe different behavior that is not compatible with IEEE 754:

compare 1.0 nan == GT

In addition to just being wrong, the more practical issue is that any alternative backend that does not inline the use of comparison operators to operators that implement IEEE 754 correctly won't pass the prelude test suite. For example backend-es does not pass the prelude test suite.

One simple fix that was floated around in Discord is to just remove the tests using NaN from the test suite and leave the behavior undefined.

garyb commented 1 year ago

This is one of the situations where Poset would have been nice to have, since comparing NaN with something should be NC.

There are issues with Eq too though, since NaN comparisons aren't reflexive.

natefaubion commented 1 year ago

I think the only solution for status quo is to consider NaN behavior undefined for the purposes of Eq and Ord, and we just shouldn't test it.