search

Blosc / c-blosc2

A fast, compressed, persistent binary data store library for C.
https://www.blosc.org
Other
447 stars 84 forks source link

Fix musl linux build #603

Closed milesgranger closed 6 months ago

milesgranger commented 6 months ago

Will close #602

Seems this is the idiom to check for musl libc and appears happy on blosc2-rs' end using this branch.

Not sure if you want the added trouble of a musl build in your CI as well, but I'll be happy to complain if mine breaks down the line. ;-)

FrancescAlted commented 6 months ago

It looks like there are several failures in CI. I am not too worried about the builds for Ubuntu (temporary failure in upstream repos?), but the one with fuzzer seems significant: https://github.com/Blosc/c-blosc2/actions/runs/8814267732/job/24194115220?pr=603

Would you be able to see what it is happening there?

milesgranger commented 6 months ago

Interesting, I should have some time tomorrow to try and figure it out. :+1:

milesgranger commented 6 months ago

Alrighty, it appears adding logic to identify musl is bad practice which after some digging found that to resolve the undefined __cpu_model refs is to link libgcc. I think technically...maybe, this ought to be in c-blosc2 configuration somewhere if the target env is musl but I'm happy to configure this in the rust bindings.

I added one commit here to call __builtin_cpu_init() as I thought initially this was needed from reading the docs, but I think it's not needed given the caveat there:

...only when used in a function that is executed before any constructors are called. The CPU detection code is automatically executed in a very high priority constructor.

So I think there is nothing to do here now, closing it. Thanks for the help. :)

FrancescAlted commented 6 months ago

Sounds good. FWIW, we are building binary wheels for blosc2 and musl, including binaries for libs and include files:

faltet@Mac-mini-de-Francesc ~/b/c-blosc2 (main)> python -c "import blosc2; print(blosc2.__path__)"       (python-blosc2)
['/Users/faltet/miniconda3/envs/python-blosc2/lib/python3.12/site-packages/blosc2']
faltet@Mac-mini-de-Francesc ~/b/c-blosc2 (main)> ls /Users/faltet/miniconda3/envs/python-blosc2/lib/libblosc2*
/Users/faltet/miniconda3/envs/python-blosc2/lib/libblosc2.2.14.3.dylib*
/Users/faltet/miniconda3/envs/python-blosc2/lib/libblosc2.2.dylib*
/Users/faltet/miniconda3/envs/python-blosc2/lib/libblosc2.a
/Users/faltet/miniconda3/envs/python-blosc2/lib/libblosc2.dylib*
faltet@Mac-mini-de-Francesc ~/b/c-blosc2 (main)> ls /Users/faltet/miniconda3/envs/python-blosc2/include/blosc2*
/Users/faltet/miniconda3/envs/python-blosc2/include/blosc2.h

/Users/faltet/miniconda3/envs/python-blosc2/include/blosc2:
blosc2-common.h     blosc2-export.h     blosc2-stdio.h      codecs-registry.h   filters-registry.h  tuners-registry.h

so having access to a development environment for Blosc2 is just a pip install blosc2 away.