Missing symbol using musl version

rgaudin commented 1 year ago

@mgautierfr I'm trying to use the 8.2.1-1 musl libzim (only aarch64!) but:

ldd on alpine complains
pylibzim wheel repair fails due to missing libm.so.6. I could fix that by adding libc6-compat package to the builder (it simply simlinks it to alpine's C lib)
testing resulting pylibzim fails on missing symbol

ldd libzim_linux-aarch64-musl-8.2.1-1/lib/aarch64-linux-musl/libzim.so.8
    /lib/ld-musl-aarch64.so.1 (0x5500000000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x55029e5000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x5502b9a000)
    libc.so => /lib/ld-musl-aarch64.so.1 (0x5500000000)
Error relocating libzim_linux-aarch64-musl-8.2.1-1/lib/aarch64-linux-musl/libzim.so.8: _ZSt28__throw_bad_array_new_lengthv: symbol not found

import libzim.writer

> ImportError: Error relocating /tmp/tmp.peHenH/venv/lib/python3.8/site-packages/libzim//libzim.so.8: __strcpy_chk: symbol not found

kelson42 commented 11 months ago

@mgautierfr Any feedback on this?

rgaudin commented 11 months ago

I still get the same issue on x86_64

ldd lib/x86_64-linux-musl/libzim.so
    /lib/ld-musl-x86_64.so.1 (0x7f9ea06d9000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7f9e9f022000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f9e9f00e000)
    libc.so => /lib/ld-musl-x86_64.so.1 (0x7f9ea06d9000)
Error relocating lib/x86_64-linux-musl/libzim.so: _ZSt28__throw_bad_array_new_lengthv: symbol not found

mgautierfr commented 10 months ago

It seems it is (still) a problem of libstdc++ versioning.

Docker image is quay.io/pypa/musllinux_1_1_x86_64
[HOST] podman cp libzim.so.8.2.1 <container>:/

[CONTAINER]

/ # ldd libzim.so.8.2.1 
/lib/ld-musl-x86_64.so.1 (0x7efe269be000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7efe25307000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7efe252f3000)
libc.so => /lib/ld-musl-x86_64.so.1 (0x7efe269be000)
Error relocating libzim.so.8.2.1: _ZSt28__throw_bad_array_new_lengthv: symbol not found

[HOST] podman cp <x86-64_musl toolchain>/x86_64-linux-musl/lib/libstdc++.so* <container>:/

[CONTAINER]

/ # LD_LIBRARY_PATH=. ldd libzim.so.8.2.1 
/lib/ld-musl-x86_64.so.1 (0x7efee32c5000)
libstdc++.so.6 => ./libstdc++.so.6 (0x7efee19a5000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7efee1991000)
libc.so => /lib/ld-musl-x86_64.so.1 (0x7efee32c5000)

The libstdc++ version in the toolchain is 6.0.29. The version in the image is 6.0.28.

rgaudin commented 10 months ago

Can you try with quay.io/pypa/musllinux_1_2_x86_64?

kelson42 commented 10 months ago

@mgautierfr Any feedback on this?

mgautierfr commented 10 months ago

Should be fixed with https://github.com/kiwix/kiwix-build/pull/636 which now use a musl TC with gcc 10.

I haven't try with quay.io/pypa/musllinux_1_2_x86_64 but it is now working (x86_64 at least) with quay.io/pypa/musllinux_1_1_x86_64.

kelson42 commented 10 months ago

@rgaudin Have you been able to verify that all looks good your side so we close this ticket?

rgaudin commented 10 months ago

Yes, it's OK! Thank you very much @mgautierfr but I can't close it as there's an issue in the archive file names:

libzim-javascript_asm_2023-11-06.zip               06-Nov-2023 02:32      3M
libzim-javascript_wasm_2023-11-06.zip              06-Nov-2023 02:32      4M
libzim_android-arm-2023-11-06.tar.gz               06-Nov-2023 01:30      8M
libzim_android-arm64-2023-11-06.tar.gz             06-Nov-2023 01:31      8M
libzim_android-x86-2023-11-06.tar.gz               06-Nov-2023 01:31      8M
libzim_android-x86_64-2023-11-06.tar.gz            06-Nov-2023 01:31      8M
libzim_linux-aarch64-2023-11-06.tar.gz             06-Nov-2023 01:30      8M
libzim_linux-aarch64-bionic-2023-11-06.tar.gz      06-Nov-2023 01:29      8M
libzim_linux-aarch64-musl-2023-11-06.tar.gz        06-Nov-2023 01:30      9M
libzim_linux-armv6-2023-11-06.tar.gz               06-Nov-2023 01:30      8M
libzim_linux-armv8-2023-11-06.tar.gz               06-Nov-2023 01:30      8M
libzim_linux-x86-64-musl-2023-11-06.tar.gz         06-Nov-2023 01:30      9M
libzim_linux-x86_64-bionic-2023-11-06.tar.gz       06-Nov-2023 01:32      9M
libzim_macos-arm64-2023-11-06.tar.gz               06-Nov-2023 01:31     26M
libzim_macos-x86_64-2023-11-06.tar.gz              06-Nov-2023 01:37      6M
libzim_wasm-emscripten-2023-11-06.tar.gz           06-Nov-2023 01:30     10M
zim-tools_linux-aarch64-2023-11-06.tar.gz          06-Nov-2023 01:30     93M
zim-tools_linux-aarch64-musl-2023-11-06.tar.gz     06-Nov-2023 01:32     93M
zim-tools_linux-armv6-2023-11-06.tar.gz            06-Nov-2023 01:31    105M
zim-tools_linux-armv8-2023-11-06.tar.gz            06-Nov-2023 01:31    104M
zim-tools_linux-i586-2023-11-06.tar.gz             06-Nov-2023 01:30     65M
zim-tools_linux-x86-64-musl-2023-11-06.tar.gz      06-Nov-2023 01:31     94M
zim-tools_linux-x86_64-2023-11-06.tar.gz           06-Nov-2023 01:31     89M
zim-tools_macos-x86_64-2023-11-06.tar.gz           06-Nov-2023 01:41     51M
zim-tools_win-i686-2023-11-06.zip                  06-Nov-2023 01:30     62M

As you can see, only musl ones are marked x86-64 while the other are named x86_64. This breaks the expected URL in pylibzim (I had to recreate one with proper naming to test)

openzim / libzim

Missing symbol using musl version #820