search

jk-jeon / dragonbox

Reference implementation of Dragonbox in C++
Apache License 2.0
588 stars 37 forks source link

Wrong constant? #55

Closed gifti258 closed 7 months ago

gifti258 commented 7 months ago

In line 832 of dragonbox.h there is a right shift of 7 bits, whereas in the paper it is a right shift of 71 bits (section 5.1.7). It seems that 71 is indeed correct.

Alcaro commented 7 months ago

That is a shift by 71. The remaining 64 bits are in the umul128_upper64; that function multiplies, then shifts the return value by 64.

But yes, it certainly looks wrong to the uninitiated.

gifti258 commented 7 months ago

Oh, my bad! Thanks!