cortex-js / cortexjs.io

Source for the cortexjs.io website
http://cortexjs.io/
49 stars 18 forks source link

Error when talking about float range in MathJSON docs #22

Open somebody1234 opened 1 year ago

somebody1234 commented 1 year ago

https://github.com/cortex-js/cortex-js.github.io/blob/2cdfc0e0c6bcf84aa090f4cddd9d7367e19e5e81/math-json/index.html#L1913-L1914

<li>the number is in the range \([-(2^{53})+1, (2^{53})-1]\) so it fits in a
64-bit float (<strong>IEEE 754-2008</strong>, 52-bit, about 15 digits of precision).</li>

The example unescaped number given (-234.534e-46) is beyond -2^53+1 - which is around 1e16

arnog commented 1 year ago

The description is indeed incorrect. The number has to be representable as a 64-bit float, within a range of -1.797693134862315E+308 to -2.225073858507201E-308, or from 2.225073858507201E-308 to 1.797693134862315E+308 and with a precision of 15 digits.