Convex-Dev / convex

Convex Main Repository - Decentralised platform for the Internet of Value
https://convex.world
Other
94 stars 30 forks source link

Inconsistent behavior in `inc` and `dec` with NaN and Infinity #59

Closed helins closed 3 years ago

helins commented 3 years ago

Unlike in other languages, those two functions cast NaN and +-Infinity to 0 producing this kind of inconsistent behavior:

(= 1 (inc NaN))  ;;  A

(= NaN (+ 1 NaN))  ;;  B

This is due to the fact that in Java, casting those to long indeed produces 0. Indeed:

(= 0 (long NaN))  ;; Valid by itself, expected on the JVM

However, B is the right way to handle it. Other math functions seem to behave well because they don't cast doubles to longs (they do the opposite if needed), meaning that those special values are left intact and follow proper float arithmetics.

mikera commented 3 years ago

Interesting and slightly scary that the JVM casts NaN to 0!

inc currently performs an implicit cast to long (behaves like (+ 1 (long arg)). Perhaps we should change this to retain the type of its argument, so that is behaves like (+ 1 arg) which is probably more intuitive.

helins commented 3 years ago

(Actually, my bad, the JVM cast Infinity and -Infinity to Long.MAX_VALUE and Long.MIN_VALUE respectively, but still the same logic)

Yes, and I believe it is a floating math rule that any computation involving NaN results in NaN.

helins commented 3 years ago

Fixed in a recent-ish commit.