databendlabs / databend

𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
https://docs.databend.com
Other
7.91k stars 752 forks source link

test: cross-platform float-point values inconsistency #12515

Open ariesdevil opened 1 year ago

ariesdevil commented 1 year ago

@ct20000901 The core issue in this situation is caused by the bit width provided by different float processor registers. This was addressed by IBM when they provided an 'Extended Precision' extension in their standard. This extension allows the float bit width to be expanded to 80-bit. For instance, the Motorola 88k architecture employs this extension, as does Apple (iOS).

The specific differences in parameters are explained in the link below, which discusses the relationship and specifics of processor register precision: https://en.wikipedia.org/wiki/Processor_register

In other words, this is a computer problem.

I understand this, but it may cause unit tests to fail on the macOS platform. Therefore, it might be necessary to modify the test cases or optimize this part to address the issue?

Originally posted by @ct20000901 in https://github.com/datafuselabs/databend/issues/12510#issuecomment-1685228938

ct20000901 commented 1 year ago

Perhaps I can take charge of improving this part?

ariesdevil commented 1 year ago

Perhaps I can take charge of improving this part?↳

PR is always welcome

ariesdevil commented 1 year ago

FYI: You could use x87 floating point instructions to enforce consistent behavior across all platforms. I haven't delved deeply into this, but you might want to investigate if you're interested. But I'm not sure Apple's chips respect this instruction set.

And though some compilers like GCC supports x87, its manual says:

-mfpmath=unit
           Generate floating-point arithmetic for selected unit unit.  The choices for unit are:

           387 Use the standard 387 floating-point coprocessor present on the majority of chips and emulated
               otherwise.  Code compiled with this option runs almost everywhere.  The temporary results are computed
               in 80-bit precision instead of the precision specified by the type, resulting in slightly different
               results compared to most of other chips.  See -ffloat-store for more detailed description.

               This is the default choice for non-Darwin x86-32 targets.
ct20000901 commented 1 year ago

FYI: You could use x87 floating point instructions to enforce consistent behavior across all platforms. I haven't delved deeply into this, but you might want to investigate if you're interested. But I'm not sure Apple's chips respect this instruction set.

And though some compilers like GCC supports x87, its manual says:

-mfpmath=unit
           Generate floating-point arithmetic for selected unit unit.  The choices for unit are:

           387 Use the standard 387 floating-point coprocessor present on the majority of chips and emulated
               otherwise.  Code compiled with this option runs almost everywhere.  The temporary results are computed
               in 80-bit precision instead of the precision specified by the type, resulting in slightly different
               results compared to most of other chips.  See -ffloat-store for more detailed description.

               This is the default choice for non-Darwin x86-32 targets.

Get. Thank you for your suggestion.

ct20000901 commented 1 year ago

FYI: You could use x87 floating point instructions to enforce consistent behavior across all platforms. I haven't delved deeply into this, but you might want to investigate if you're interested. But I'm not sure Apple's chips respect this instruction set.

And though some compilers like GCC supports x87, its manual says:

-mfpmath=unit
           Generate floating-point arithmetic for selected unit unit.  The choices for unit are:

           387 Use the standard 387 floating-point coprocessor present on the majority of chips and emulated
               otherwise.  Code compiled with this option runs almost everywhere.  The temporary results are computed
               in 80-bit precision instead of the precision specified by the type, resulting in slightly different
               results compared to most of other chips.  See -ffloat-store for more detailed description.

               This is the default choice for non-Darwin x86-32 targets.

I have done some research on this matter and found that there are multiple possible causes for this issue. You can refer to this link: https://github.com/rust-lang/rust/issues/114479#issue-1837197600 for more details. It seems that achieving complete consistency in floating-point precision across all platforms is not entirely feasible. Perhaps it would be more practical to focus on modifying the test cases for now?

ariesdevil commented 1 year ago

Agreed, the main target is to make test cases pass.