Closed hinxx closed 1 month ago
The precision of 2^-63 is much smaller than the machine epsilon of JavaScript (which is double precision floating point, in which case the machine epsilon is something like 2^-53). So this is a very reasonable rounding error. Not much we can do about that in JavaScript.
If I understand correctly with Q1.63 signed conversion values are in range 0.99999999 .. -1 and trying to convert 1 should raise an
Out of Range
error (like it happens when doing Q1.31 signed conversion).