Make q10-11 compression floating-point-free

shao-hua-li commented 2 years ago

Hi there,

I found that brotli would emit different compressed files if I compiled with different compilers.

Version : git head at the time of report (commit 62662f87cdd96deda90ac817de94e3c4af75226a)
Compiler : clang13 and gcc11
Reproduce: 1) compile with clang13 and gcc11 separately. 2) run brotli -fq 11 diff_0 -o output with two brotli compiled by different compilers. 3) You'll see they emit different compressed output files.
Test case: diff_0.tar.gz (decompress it first).

Thanks, Shaohua

eustas commented 2 years ago

Yes, this might happen. We did not completely got rid of floating-point ops, so result might be compiler- and CPU-dependent. Going to keep this issue open, as this is one of our future goals.

shao-hua-li commented 2 years ago

Thanks for the reply! I just worried that developers may use checksum to validate the integrity, which might be falsely broken due to this issue. Looking forward to your solution! :).

eustas commented 2 years ago

IIRC, compression levels 0-9 should does not use FP, so should produce same output across compilers/CPUs.

google / brotli

Make q10-11 compression floating-point-free #935