Incorrect compressed output from Android build

[mhsmith]

I'm trying to make a Python build of Brotli for Chaquopy on Android (https://github.com/chaquo/chaquopy/issues/137). The build process itself is no problem (I'm using the 1.0.7 sdist ZIP from PyPI), but the resulting package doesn't work correctly.

With the official Linux build from PyPI:

>>> plain = b"it was the best of times, it was the worst of times"
>>> compressed = brotli.compress(plain)
>>> compressed.hex()
'1b3200e015a1d23bed4dee0e4ce480bd2df5f0808f5a17442f98540f37f0a583fe06'
>>> brotli.decompress(compressed)
b'it was the best of times, it was the worst of times'

With my own Android build:

>>> compressed.hex()
'1b3200e015a1d29df626770726728f5a17442f98540f37f0a583fe06'

>>> brotli.decompress(compressed)
Traceback (most recent call last):
   File "<console>", line 1, in <module>
brotli.error: BrotliDecompress failed

Notice that the invalid output is the same at the beginning and end, but the middle section is both different and shorter. I get the same output on both x86 and arm64-v8a.

Can anyone suggest what might be going on here?

[mhsmith]

I should mention, I'm using the Crystax NDK with a relatively old compiler (GCC 4.9.3).

[mhsmith]

I eventually tracked this down to BROTLI_RESTRICT. With that macro disabled, compression worked correctly on all ABIs.

However, decompression still gave the same error on arm64-v8a until I added BROTLI_BUILD_NO_RBIT. My test device was a Nexus 5X (Qualcomm Snapdragon 808, 2x Cortex-A57, 4x Cortex-A53), but I suspect the compiler is at fault here.

This compiler version is no longer supported on Android, and I'll be moving off it myself as soon as possible. So there's no need to take any further action on this, unless you want to.

[eustas]

Thanks for the investigation. Still makes sense to:

• warn developers about possible problems
• see what is wrong with restrict - this might mean that something is marked wrong, i.e. two areas could intersect
• add unit test that checks that rbit works well if compiler thinks it should

[jinfeihan57]

GCC 4.8 is not working . The compiler version is too low, try a newer version. GCC 5.4 is OK. or try a cross compiler。

[mhsmith]

I tried building brotli 1.0.7 using the Google NDK r18 (Clang version 7.0.2), and it produces the correct output on all ABIs, without needing any of the above changes. So I think that confirms it was the Crystax NDK that was at fault.

[eustas]

@mhsmith thanks for investigating.