[Bug]: MUSL CMAKE unsupported relocation type 37

[mgood7123]

Describe the bug

localhost:~/llvm_18$ release_BUILD/LIBXZ/OUT/bin/lzma
Error relocating release_BUILD/LIBXZ/BUILD/lzma: unsupported relocation type 37
Error relocating release_BUILD/LIBXZ/BUILD/lzma: unsupported relocation type 37

if (NOT EXISTS ${CMAKE_BINARY_DIR}/LIBXZ)
  execute_process(
    COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_CURRENT_SOURCE_DIR}/xz ${CMAKE_BINARY_DIR}/LIBXZ
  )
endif()

if (NOT EXISTS ${CMAKE_BINARY_DIR}/LIBXZ/OUT/lib/liblzma.a)
  execute_process(
    COMMAND
      ${CMAKE_COMMAND}
      -D CMAKE_BUILD_TYPE=Release
      -D "CMAKE_C_FLAGS=${CMAKE_C_FLAGS}"
      -D "CMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS}"
      -D CMAKE_C_COMPILER=${CMAKE_C_COMPILER}
      -D CMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
      -D CMAKE_INSTALL_PREFIX:PATH=${CMAKE_BINARY_DIR}/LIBXZ/OUT
      -D BUILD_SHARED_LIBS=OFF
      -B ${CMAKE_BINARY_DIR}/LIBXZ/BUILD
    WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/LIBXZ
    COMMAND_ECHO STDOUT
  )
  execute_process(
    COMMAND
      ${CMAKE_COMMAND}
      --build ${CMAKE_BINARY_DIR}/LIBXZ/BUILD
    WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/LIBXZ
    COMMAND_ECHO STDOUT
  )
  execute_process(
    COMMAND
      ${CMAKE_COMMAND}
      --install ${CMAKE_BINARY_DIR}/LIBXZ/BUILD
    WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/LIBXZ
    COMMAND_ECHO STDOUT
  )
endif()

Version

5.5.0 git master https://github.com/tukaani-project/xz/commit/837ea40b1c9d4998cac4500b55171bf33e0c31a6

Operating System

alpine linux

Relevant log output

No response

[mgood7123]

so this is caused by musl not supporting _ifunc_ attribute and current alpine linux clang compilers (clang 14, 15, 16, and 17 to 17.0.3) not emitting a warning if ifunc is used for a musl target, (unlike gcc which does emit a warning)

fixable via -D ALLOW_ATTR_IFUNC=OFF

[thesamesam]

I think we should keep this open so the build system can work around it.

[Larhzu]

In CMake-based build, the test for attributes both compiles and links. Perhaps the problem is that linker can omit the function as it's not used in the program.

CMakeLists.txt line 810:

int main(void) { return 0; }

I wonder if this would help:

int main(void) { func_ifunc(); return 0; }

(Edited: Accidental Ctrl-Enter submits in GH. That's a dangerously easy keyboard shortcut for such a critical action. Oh well.)

[Larhzu]

My above comment is wrong. The failure is from the dynamic linker, not static linker, sorry.

Perhaps it's that GCC knows that ifunc isn't supported with musl. config.gcc doesn't set default_gnu_indirect_function with musl. Maybe Clang/LLVM needs to learn this too if it currently doesn't warn.

[mgood7123]

In CMake-based build, the test for attributes both compiles and links. Perhaps the problem is that linker can omit the function as it's not used in the program.

CMakeLists.txt line 810:

int main(void) { return 0; }

I wonder if this would help:

int main(void) { func_ifunc(); return 0; }

(Edited: Accidental Ctrl-Enter submits in GH. That's a dangerously easy keyboard shortcut for such a critical action. Oh well.)

an ifunc WILL compile and link successfully in a dynamically built program, but musl does not implement ifunc (GNU glibc only extension)

[mgood7123]

a crude hack to detect musl is to detect /lib/ld-musl-*

[Larhzu]

an ifunc WILL compile and link successfully in a dynamically built program, but musl does not implement ifunc (GNU glibc only extension)

Yes, I relealized this as I wrote in my next message. Since GCC upstream knows that musl doesn't support ifunc, I wonder if Clang/LLVM should know it too and then warn or error if the ifunc attribute is used. That is, I wonder if this could be a Clang/LLVM bug.

a crude hack to detect musl is to detect /lib/ld-musl-*

In Autoconf, checking if $host_os equals linux-musl probably is the correct method. I don't know right now how to detect it in CMake.

According to musl's FAQ, there intentionally is no easy #ifdef to detect musl in C code.

Hacks like checking file paths wouldn't work when cross-compiling.

On the second thought, uClibc might not support ifunc either. It could be better to detect glibc, so linux-gnu in case of Autoconf (maybe FreeBSD too). But once again I don't know right now how to detect the libc in CMake.

[mgood7123]

we could extract the triple on gcc/clang via -dumpmachine

$ clang -dumpmachine
x86_64-alpine-linux-musl

[JiaT75]

we could extract the triple on gcc/clang via -dumpmachine

I don't think this will work for us since a tool like musl-gcc (a GCC wrapper for using musl libc) still outputs x86_64-linux-gnu with -dumpmachine. So this would fix things for Alpine, but our builds would still be broken for anyone compiling for musl libc using a wrapper like this.

[mgood7123]

we could extract the triple on gcc/clang via -dumpmachine

I don't think this will work for us since a tool like musl-gcc (a GCC wrapper for using musl libc) still outputs x86_64-linux-gnu with -dumpmachine. So this would fix things for Alpine, but our builds would still be broken for anyone compiling for musl libc using a wrapper like this.

Hmmm

[mgood7123]

Maybe we could just add a define for musl

make -D MUSL=1

and trust the builder to pass this define when building on a musl distro

[JiaT75]

Maybe we could just add a define for musl

make -D MUSL=1

The default build options need to create a working build on all of our supported platforms. Right now people can just disable ifunc for a working musl build as you discovered with -DALLOW_ATTR_IFUNC=OFF. That is only a temporary workaround since we want to make our build systems as easy to use for people as possible.

I have been working a branch to address this. The idea is to change the ifunc option for our CMake and Autotools build from ON/OFF or enable/disable to auto/ON/OFF. 'auto' will try to enforce using ifunc with glibc or BSD platforms only. ON will always try to use ifunc and OFF will obviously disable ifunc completely. Both auto and ON will still test the compiler if it supports __attribute__((__ifunc__())).

If you can test the new branch on Alpine that would be very helpful. I tested with musl-gcc wrapper and things seemed to work. We haven't decided 100% if this is the approach we want to take but it seems promising so far.

Thanks for all of your help so far!

[mgood7123]

Alright

On Mon, 27 Nov 2023, 11:04 pm Jia Tan, @.***> wrote:

Maybe we could just add a define for musl

make -D MUSL=1

The default build options need to create a working build on all of our supported platforms. Right now people can just disable ifunc for a working musl build as you discovered with -DALLOW_ATTR_IFUNC=OFF. That is only a temporary workaround since we want to make our build systems as easy to use for people as possible.

I have been working a branch https://github.com/tukaani-project/xz/tree/ifunc_detect_fix to address this. The idea is to change the ifunc option for our CMake and Autotools build from ON/OFF or enable/disable to auto/ON/OFF. 'auto' will try to enforce using ifunc with glibc or BSD platforms only. ON will always try to use ifunc and OFF will obviously disable ifunc completely. Both auto and ON will still test the compiler if it supports attribute((ifunc())).

If you can test the new branch on Alpine that would be very helpful. I tested with musl-gcc wrapper and things seemed to work. We haven't decided 100% if this is the approach we want to take but it seems promising so far.

Thanks for all of your help so far!

— Reply to this email directly, view it on GitHub https://github.com/tukaani-project/xz/issues/70#issuecomment-1827795707, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGLITH3CC7DIVMAJSDP2IMTYGSFUPAVCNFSM6AAAAAA7LYSUISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRXG44TKNZQG4 . You are receiving this because you modified the open/close state.Message ID: @.***>

[mgood7123]

Maybe we could just add a define for musl

make -D MUSL=1

The default build options need to create a working build on all of our supported platforms. Right now people can just disable ifunc for a working musl build as you discovered with -DALLOW_ATTR_IFUNC=OFF. That is only a temporary workaround since we want to make our build systems as easy to use for people as possible.

I have been working a branch to address this. The idea is to change the ifunc option for our CMake and Autotools build from ON/OFF or enable/disable to auto/ON/OFF. 'auto' will try to enforce using ifunc with glibc or BSD platforms only. ON will always try to use ifunc and OFF will obviously disable ifunc completely. Both auto and ON will still test the compiler if it supports __attribute__((__ifunc__())).

If you can test the new branch on Alpine that would be very helpful. I tested with musl-gcc wrapper and things seemed to work. We haven't decided 100% if this is the approach we want to take but it seems promising so far.

Thanks for all of your help so far!

Tested on alpine, it works ^^

[JiaT75]

@mgood7123 Thanks again for reporting and helping us test this. I would like to add you to our THANKS file, but I did not see your name on your GitHub profile. Is there a name you would like us to use for you in our THANKS file? Otherwise you may remain anonymous :)

[mgood7123]

@mgood7123 Thanks again for reporting and helping us test this. I would like to add you to our THANKS file, but I did not see your name on your GitHub profile. Is there a name you would like us to use for you in our THANKS file? Otherwise you may remain anonymous :)

Matthew Good