rustc 1.59/1.60 builds musl binaries that segfault, when compiling with musl-gcc wrappers, due to static-pie default

[joshtriplett]

With the x86_64-unknown-linux-musl target installed:

$ cat hello.rs 
fn main() {
    println!("Hello, world!");
}
$ rustc hello.rs --target x86_64-unknown-linux-musl && file hello && ./hello 
hello: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), static-pie linked, BuildID[sha1]=29ba716c85a1f3f9990246f160b61003b38b63a9, with debug_info, not stripped
Hello, world!
$ rustc hello.rs --target x86_64-unknown-linux-musl -C linker=musl-gcc && file hello && ./hello 
hello: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
Segmentation fault

A gdb backtrace shows the segfault in _start_c () at ../src_musl/crt/../ldso/dlstart.c:141.

This segfault happens because the musl-gcc wrappers (including cross-compile wrappers like x86_64-linux-musl-gcc and aarch64-linux-musl-gcc) don't support static-pie in the way rustc is invoking them.

This is a regression in Rust 1.59 and 1.60; 1.58 and before produced working binaries. 1.59 includes https://github.com/rust-lang/rust/pull/70740 , which not only marked the musl targets as supporting static-pie (which they may or may not depending on the linker), but also made x86_64-unknown-linux-musl default to static-pie.

For context, the musl-gcc wrappers are commonly used for a variety of reasons, including cross-compilation (particularly when linking native C code). Attempting to link using x86_64-linux-gnu-gcc or aarch64-linux-gnu-gcc cross-compilers (rather than the corresponding -musl-gcc cross-compilers) produces compatibility issues, especially with projects that build and link native C dependencies.

I noticed this problem when working with someone trying to cross-compile a project from aarch64 to x86-64, using the x86_64-linux-musl-gcc cross-compiler wrapper, installed via Debian. (I hadn't previously noticed the regression myself, because I hadn't arranged to use the cross-compiler wrappers on x86_64 since my native host is x86_64.)

This problem doesn't currently show up when cross-compiling from x86_64 to aarch64, since aarch64 doesn't use static-pie by default, but enabling PIE with -C relocation-model=pie triggers the same bug and makes the resulting binaries segfault.

I'm currently working around this, but I think this is a stable regression.

(For reference, two possible workarounds: use -C relocation-model=static, or use linker=x86_64-linux-gnu-gcc and hope it works for you.)

[12101111]

Can you provide the linker command? It's fixed in 2018: https://git.musl-libc.org/cgit/musl/commit/?id=7dad9c212587267818de919dd9c5886f18f99779 (Unless you are using Ubuntu 18.04 which is unfixed)

[joshtriplett]

~$ cat /usr/bin/x86_64-linux-musl-gcc
#!/bin/sh
exec "${REALGCC:-x86_64-linux-gnu-gcc}" "$@" -specs "/usr/lib/x86_64-linux-musl/musl-gcc.specs"
~$ cat /usr/lib/x86_64-linux-musl/musl-gcc.specs
%rename cpp_options old_cpp_options

*cpp_options:
-nostdinc -isystem /usr/include/x86_64-linux-musl -isystem include%s %(old_cpp_options)

*cc1:
%(cc1_cpu) -nostdinc -isystem /usr/include/x86_64-linux-musl -isystem include%s

*link_libgcc:
-L/usr/lib/x86_64-linux-musl -L .%s

*libgcc:
libgcc.a%s %:if-exists(libgcc_eh.a%s)

*startfile:
%{!shared: /usr/lib/x86_64-linux-musl/Scrt1.o} /usr/lib/x86_64-linux-musl/crti.o crtbeginS.o%s

*endfile:
crtendS.o%s /usr/lib/x86_64-linux-musl/crtn.o

*link:
-dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib %{shared:-shared} %{static:-static} %{rdynamic:-export-dynamic}

*esp_link:

*esp_options:

*esp_cpp_options:

[12101111]

https://github.com/rust-lang/rust/blob/327caac4d01aef74d6577b87c295270608be09fa/compiler/rustc_codegen_ssa/src/back/linker.rs#L382-L402

The musl-gcc wrapper don't recongnize -static-pie/-pie/-no-pie, but do recongnize -static. The combination of -dynamic-linker and -static also break lld.

> cat hello.c
#include<stdio.h>
int main() {
  printf("Hello, world!\n");
  return 0;
}
> musl-gcc hello.c -o hello -static-pie
ld -plugin /usr/lib/gcc/x86_64-linux-gnu/10/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper -plugin-opt=-fresolution=/tmp/cciwsIZK.res -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -plugin-opt=-pass-through=-lc -dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib -pie -o hello /usr/lib/x86_64-linux-musl/Scrt1.o /usr/lib/x86_64-linux-musl/crti.o /usr/lib/gcc/x86_64-linux-gnu/10/crtbeginS.o -L/usr/lib/x86_64-linux-musl -L /usr/lib/gcc/x86_64-linux-gnu/10/. /tmp/ccNzdDMJ.o --start-group /usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/10/crtendS.o /usr/lib/x86_64-linux-musl/crtn.o
> file hello
hello: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
> musl-gcc hello.c -o hello -static
ld -plugin /usr/lib/gcc/x86_64-linux-gnu/10/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper -plugin-opt=-fresolution=/tmp/cciPPY3o.res -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -plugin-opt=-pass-through=-lc -dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib -static -o hello /usr/lib/x86_64-linux-musl/Scrt1.o /usr/lib/x86_64-linux-musl/crti.o /usr/lib/gcc/x86_64-linux-gnu/10/crtbeginS.o -L/usr/lib/x86_64-linux-musl -L /usr/lib/gcc/x86_64-linux-gnu/10/. /tmp/cckhAQSp.o --start-group /usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/10/crtendS.o /usr/lib/x86_64-linux-musl/crtn.o
> file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped
 > musl-gcc -fuse-ld=lld -static hello.c -o hello
ld.lld -plugin /usr/lib/gcc/x86_64-linux-gnu/10/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper -plugin-opt=-fresolution=/tmp/cc788cAg.res -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a -plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -plugin-opt=-pass-through=-lc -dynamic-linker /lib/ld-musl-x86_64.so.1 -nostdlib -static -o hello /usr/lib/x86_64-linux-musl/Scrt1.o /usr/lib/x86_64-linux-musl/crti.o /usr/lib/gcc/x86_64-linux-gnu/10/crtbeginS.o -L/usr/lib/x86_64-linux-musl -L /usr/lib/gcc/x86_64-linux-gnu/10/. /tmp/ccsEJC6i.o --start-group /usr/lib/gcc/x86_64-linux-gnu/10/libgcc.a /usr/lib/gcc/x86_64-linux-gnu/10/libgcc_eh.a -lc --end-group /usr/lib/gcc/x86_64-linux-gnu/10/crtendS.o /usr/lib/x86_64-linux-musl/crtn.o
> file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
> ./hello
zsh: segmentation fault  ./hello

Fedora fix this https://src.fedoraproject.org/rpms/musl/blob/rawhide/f/musl-1.2.0-Support-static-pie-with-musl-gcc-specs.patch and Debian don't have this patch.

[apiraino]

thanks @12101111 for the investigation. This looks like a distro specific issue that should be probably handled by them, correct? Just trying to understand if there's an actionable on our side.

[12101111]

Yes, we can't fix the broken wrapper in our side ( we can detect it but can't fix it expect creating a new wrapper).

[joshtriplett]

@apiraino The action on the Rust side would be to roll back the use of this by default until it's more widely supported. Or, alternatively, attempt to detect the use of the wrapper and roll it back in that circumstance, but that seems fraught.

[joshtriplett]

(Also, long-term, we could fix this by making the wrappers not necessary anymore for cross compiling with musl.)

[apiraino]

Assigning priority as discussed in the Zulip thread of the Prioritization Working Group.

@rustbot label -I-prioritize +P-high

[ghost]

This seems possibly related to two issues I've been having: https://github.com/deltaphc/raylib-rs/issues/119 https://github.com/gtk-rs/gtk4-rs/issues/1096

But when I try to follow this example, the second command gives me:

error: linker `musl-gcc` not found
  |
  = note: No such file or directory (os error 2)

error: aborting due to previous error

[pnkfelix]

Discussed at P-high review

At this point I doubt we would roll-back the default setting, at least not silently. (Maybe via an MCP or something, if there was a push.)

But I suspect the best short term act to take would be a diagnostic warning when we attempt to compile with the known-to-be-problematic combination of flags? @joshtriplett would that satisify you here?

[joshtriplett]

@pnkfelix Some kind of detection and warning would be helpful at a minimum, yes.

[pnkfelix]

Discussed at 2023-Q1 P-high review

I'm willing (*) to mentor someone who would like to add the aforementioned detection+issuing-of-diagnostic of the problematic combination of flags.

I'm mainly wondering whether doing so will be E-easy, or E-medium.

(): However, I will be away from computers for a week, coming back on 2023-04-24. So not available to mentor until late April at earliest. You can probably find someone* who can assist. I'll try to write up some guidance in the meantime.

[Timmmm]

or use linker=x86_64-linux-gnu-gcc

I don't understand this whatever this is about, but in case this helps anyone here, I used to set

CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-linux-musl-gcc

And it totally worked, but now it causes a segfault. Both -C relocation-model=static and -C target-feature=-crt-static led to linker errors, but in the end I tried unsetting that environment variable and it just works now. I can't really remember why it was needed years ago but apparently it isn't now.