search

cvickery / IEEE-754

Web page to analyze IEEE-754 floating point values.
MIT License
19 stars 14 forks source link

Rounding mode should (optionally?) affect binary downconversion #1

Open no-identd opened 6 years ago

no-identd commented 6 years ago

Try putting this decimal in:

7.634541525402449498693660266570045949932980514669426596282135488204245721277629854739643633365631103515625E1

And change the rounding modes. Observe how this affects the conversion to binary64 and binary32.

Now try putting 40053161B4894E4C8835E319B8AED224 in, the hexadecimal representation of the above decimal. Change the rounding mode. Observe how it doesn't affect the conversion to binary 64 and binary32.

cvickery commented 6 years ago

The difference here is that the decimal requires more bits to represent than are available in any mode. When you enter a hex value, the bits you enter are the bits you get. But there are no extra bits of information to trigger rounding.