Missing package: blosc2

[bennuttall]

Package name

blosc2

Package version

2.1.0+

PyPI URL

https://pypi.org/project/blosc2/

piwheels URL

https://www.piwheels.org/project/blosc2/

Python version

• [X] Python 3.7
• [X] Python 3.9
• [X] Python 3.11

I am aware this is the issue tracker for a Python package index specifically for Raspberry Pi

• [X] Yes

I have checked for duplicate issues

• [X] Yes

I am the maintainer

• [ ] Yes

More information

Build log: https://www.piwheels.org/logs/0000/1095/0453.txt

This is causing #400

[xmatthias]

That error is quite odd to me (if we ignore the apparent encoding errors in quotes)

It first complains about invalid value for -mfpu - but then points out exactly the -mfpu=neon as being one of the supported arguments ...

2023-10-05T20:35:20,880   FAILED: blosc2/c-blosc2/blosc/CMakeFiles/blosc2_shared.dir/shuffle-neon.c.o
2023-10-05T20:35:20,881   /usr/bin/cc -DBLOSC_SHARED_LIBRARY -DUSING_CMAKE -DZLIB_COMPAT -DZSTD_DISABLE_ASM -Dblosc2_shared_EXPORTS -I/tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/_skbuild/linux-armv7l-3.11/cmake-build/blosc2/c-blosc2/blosc -I/tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/blosc2/c-blosc2/include -I/tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/blosc2/c-blosc2/plugins/codecs/zfp/include -I/tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/blosc2/c-blosc2/blosc -I/tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/blosc2/c-blosc2/internal-complibs/lz4-1.9.4 -I/tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/blosc2/c-blosc2/internal-complibs/zlib-ng-2.0.7 -I/tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/blosc2/c-blosc2/internal-complibs/zstd-1.5.5 -I/tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/blosc2/c-blosc2/internal-complibs/zstd-1.5.5/common -std=gnu99 -O3 -DNDEBUG -fPIC -fvisibility=hidden -Wall -Wextra "-mfpu=neon -flax-vector-conversions" -MD -MT blosc2/c-blosc2/blosc/CMakeFiles/blosc2_shared.dir/shuffle-neon.c.o -MF blosc2/c-blosc2/blosc/CMakeFiles/blosc2_shared.dir/shuffle-neon.c.o.d -o blosc2/c-blosc2/blosc/CMakeFiles/blosc2_shared.dir/shuffle-neon.c.o -c /tmp/pip-wheel-y4c1fcdj/blosc2_c7ff9ef5dd7c4b5eb0d9bf6623616b32/blosc2/c-blosc2/blosc/shuffle-neon.c
2023-10-05T20:35:20,882   cc: error: unrecognized argument in option ‘-mfpu=neon -flax-vector-conversions’
2023-10-05T20:35:20,884   cc: note: valid arguments to ‘-mfpu=’ are: auto crypto-neon-fp-armv8 fp-armv8 fpv4-sp-d16 fpv5-d16 fpv5-sp-d16 neon neon-fp-armv8 neon-fp16 neon-vfpv3 neon-vfpv4 vfp vfp3 vfpv2 vfpv3 vfpv3-d16 vfpv3-d16-fp16 vfpv3-fp16 vfpv3xd vfpv3xd-fp16 vfpv4 vfpv4-d16

[xmatthias]

seems like others found / reported this too: https://github.com/Blosc/python-blosc2/issues/143

[FrancescAlted]

We don't have access to a RPi for testing. Would you mind send a PR with the necessary fixes?

[xmatthias]

To add to this issue: Manual compilation of both c-blosc2 and python-blosc2 both fail at the same step (at bitshuffle-neon).

It seems that the pi does support NEON though - as the package https://packages.debian.org/bullseye/neon-support installs just fine - and it should fail if neon is not supported.

Interestingly, mine fails at a slightly different step (bitshuffle-neon.c.o instead of shuffle in the above logs).

while i do have a pi at hand (multiple, actually) - i do lack the appropriate skills in C / C++ / compiler land to be actually helpful.

[FrancescAlted]

If you can you put a rpi accessible, I will have a look.

[bennuttall]

You can't use NEON as the Pi 1/Zero doesn't have it - so wheels built with it will be incompatible.

@FrancescAlted I've set up a cloud Pi and imported your SSH keys from github:

SSH commands:
  ssh -p 5370 root@ssh.piwheels-blosc2.hostedpi.com  # IPv4
  ssh root@[2a00:1098:8:172::1]  # IPv6

It's running Bullseye (oldstable) but that should be sufficient for getting it working.

Email me if you have any issues or questions: ben at piwheels.org

[FrancescAlted]

Ok. I have found a couple of different issues that should have been fixed in latest release: https://github.com/Blosc/c-blosc2/releases/tag/v2.11.2

With that, the tests are passing well:

$ cmake .. -DDEACTIVATE_ZLIB=ON -DDEACTIVATE_ZSTD=ON -DBUILD_PLUGINS=OFF
<snip>
$ make -j 4
<snip>
$ ctest
<snip>
1319/1320 Test #1319: b2nd_example_print_meta ...................................   Passed    0.01 sec
          Start 1320: b2nd_example_serialize
1320/1320 Test #1320: b2nd_example_serialize ....................................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 1320

Label Time Summary:
b2nd    =   2.41 sec*proc (6 tests)

Total Test time (real) = 415.45 sec

Thanks @bennuttall for providing the Pi.

[bennuttall]

Great. Will look out for the next release.

[xmatthias]

3.9 and 3.11 are now building successfully - which essentially means we could also give the tables (#400) package a re-run ...

[bennuttall]

Great!