search

rust-cross / cargo-zigbuild

Compile Cargo project with zig as linker
MIT License
1.43k stars 51 forks source link

musl targets are broken with latest zig (0.11.0) #158

Closed haohaolee closed 1 year ago

haohaolee commented 1 year ago

As is showed in this PR.

The cause behind this is because even latest stable rust is using musl 1.2.3, while latest zig is using musl 1.2.4. And musl 1.2.4 remove all the weak alias for some 64bit symbols (e.g., lstat64, stat64, open64....)

AFAIK, zigbuild's strategy is to use zig's musl instead of the one from rust, which is good, but the embarrassing thing here is, the musl with a higher version removed some symbols needed by rust's libc, which causes the linking errors.

After rust starts to use musl 1.2.4+, everything would be OK, but we still need to figure out how to address this in a short term. Thus, I open this issue for the discussion.

One approach I can think of is to link to rust's musl, the other is to add the missing weak symbols.

messense commented 1 year ago

Feel free to send a PR.

NobodyXu commented 1 year ago

BTW, is it possible to use rust-lld for linking? That will enable robust cross-lang-lto, which otherwise is a bit fragile since rustc often has its llvm upgraded before zig.

messense commented 1 year ago

BTW, is it possible to use rust-lld for linking?

I'm not sure, can you send a PR to test it out?

NobodyXu commented 1 year ago

BTW, is it possible to use rust-lld for linking?

I'm not sure, can you send a PR to test it out?

Sorry, I'm not familiar with rust-lld internal or zig, so I don't know how to do this, but is just wondering if this is possible.

messense commented 1 year ago

@NobodyXu Actually you can test it without changing cargo-zigbuild, just need to set CARGO_TARGET_<triple>_LINKER=rust-lld env var.

NobodyXu commented 1 year ago

@NobodyXu Actually you can test it without changing cargo-zigbuild, just need to set CARGO_TARGET_<triple>_LINKER=rust-lld env var.

Thanks, I've opened https://github.com/cargo-bins/cargo-binstall/pull/1253 for this.

haohaolee commented 1 year ago

Feel free to send a PR.

I can give it a try. Currently I have tested the 1st approach, which is successful, but I prefer the 2nd for which I am still figuring out how.

BTW, zig also has an internal lld interface zig ld.lld for linux

NobodyXu commented 1 year ago

BTW, zig also has an internal lld interface zig ld.lld for linux

Thanks, the problem is that zig could use an older LLVM version than rustc, which cannot parse LLVM bitcode from newer version.

NobodyXu commented 1 year ago

@NobodyXu Actually you can test it without changing cargo-zigbuild, just need to set CARGO_TARGET_<triple>_LINKER=rust-lld env var.

@messense It works when we use --target x86_64-unknown-linux-gnu.2.17, I thought that zig also need to do some magic at link stage to link with glibc 2.17, but it seems that I'm wrong and it only needs to provide a different glibc header at compile-time.

messense commented 1 year ago

Better also check that the artifact is actually only depends on glibc 2.17.

haohaolee commented 1 year ago

BTW, zig also has an internal lld interface zig ld.lld for linux

Thanks, the problem is that zig could use an older LLVM version than rustc, which cannot parse LLVM bitcode from newer version.

I just checked rust-lld, its location on my machine is ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld

I guess it is not for the targets

haohaolee commented 1 year ago

@NobodyXu Actually you can test it without changing cargo-zigbuild, just need to set CARGO_TARGET_<triple>_LINKER=rust-lld env var.

@messense It works when we use --target x86_64-unknown-linux-gnu.2.17, I thought that zig also need to do some magic at link stage to link with glibc 2.17, but it seems that I'm wrong and it only needs to provide a different glibc header at compile-time.

zig's approach is to build a libc (glibc or musl) on the fly, and since glibc has a very good backward compatibility (a relatively newer glibc.so is enough), so you are right, zig will choose different glibc symbols based on the version to do the linking at last. You can take a look at this tool https://github.com/ziglang/glibc-abi-tool

haohaolee commented 1 year ago

And for the 1st approach I mentioned, I can put a quick workaround here for those who wanna try latest zig and rust

$ cat ld.sh
#!/bin/sh
exec zig ld.lld "$@"
$ CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=$(pwd)/ld.sh cargo zigbuild --target x86_64-unknown-linux-musl
......
   Compiling hello-rustls v0.1.0 (/tmp/hello-rustls)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 12s
NobodyXu commented 1 year ago

Better also check that the artifact is actually only depends on glibc 2.17.

Tested in centos:7 that it can run.

NobodyXu commented 1 year ago

zig's approach is to build a libc (glibc or musl) on the fly, and since glibc has a very good backward compatibility (a relatively newer glibc.so is enough), so you are right, zig will choose different glibc symbols based on the version to do the linking at last. You can take a look at this tool https://github.com/ziglang/glibc-abi-tool

Thanks, I'm now a bit worried despite it actually runs on centos:7 with glibc 2.17

NobodyXu commented 1 year ago

BTW, does the CARGO_TARGET_<triple>_LINKER has to be all in upper case? Could the <triple> be specified in lower-case?

haohaolee commented 1 year ago

And for the 1st approach I mentioned, I can put a quick workaround here for those who wanna try latest zig and rust

$ cat ld.sh
#!/bin/sh
exec zig ld.lld "$@"
$ CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=$(pwd)/ld.sh cargo zigbuild --target x86_64-unknown-linux-musl
......
   Compiling hello-rustls v0.1.0 (/tmp/hello-rustls)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 12s

CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=rust-lld also works

messense commented 1 year ago

BTW, does the CARGO_TARGET_<triple>_LINKER has to be all in upper case? Could the <triple> be specified in lower-case?

cc-rs supports both I think, but currently in cargo-zigbuild we only check upper-case.

https://github.com/rust-cross/cargo-zigbuild/blob/41e7f7866ea3aea3a6e0b5ccbd5033ae9e4c934d/src/zig.rs#L390

NobodyXu commented 1 year ago

cc-rs supports both I think, but currently in cargo-zigbuild we only check upper-case.

Thanks!

So I think it's safe to assume that cargo also supports both.

NobodyXu commented 1 year ago

I tried to enable cross-lang-lto and it seems that rust-lld cannot even find -lgcc_s, I would just revert the PR using rust-lld

haohaolee commented 1 year ago

I tried to enable cross-lang-lto and it seems that rust-lld cannot even find -lgcc_s, I would just revert the PR using rust-lld

rust-lld should just be a workaround for the musl scenario, I don't think it will work for gnu targets. I assume gnu targets would work without any change

NobodyXu commented 1 year ago

I tried to enable cross-lang-lto and it seems that rust-lld cannot even find -lgcc_s, I would just revert the PR using rust-lld

rust-lld should just be a workaround for the musl scenario, I don't think it will work for gnu targets. I assume gnu targets would work without any change

Thanks