Open madsmtm opened 6 months ago
CC people whom I know have worked with macOS linking before: @BlackHoleFox, @bjorn3, @simlay
This would be great to have! --print=native-static-libs
is a workaround for static libraries not being able to list the dependencies they have on most platforms. We should probably create a separate object file with this information which is only added to staticlibs. This way rustc still has control over all linker args when it itself invokes the linker.
For ELF I recently found the __.LIBDEP
archive member which was introduced not too long ago: https://sourceware.org/binutils/docs/ld/libdep-Plugin.html This only works with quite recent ld.bfd versions, not lld or mold afaik. As such using it is not an option, just like using LLVM's .deplibs
section is not an option due to only working with lld.
On Windows we could embed static dependencies in static libraries (using the .drectve
section) instead of requiring them to be passed on the command line, but Rust doesn't currently.
Seems neat to me. Is auto-linking available in ld
for every XCode version we support (XCode 9 min iirc)?
Is auto-linking available in
ld
for every XCode version we support (XCode 9 min iirc)?
Just checked, LC_LINKER_OPTION
is parsed since revision 224.1 of ld
/ld64
, which is included in Xcode 5.0.
We should probably create a separate object file with this information which is only added to staticlibs. This way rustc still has control over all linker args when it itself invokes the linker.
Agreed.
For ELF I recently found the
__.LIBDEP
archive member which was introduced not too long ago: https://sourceware.org/binutils/docs/ld/libdep-Plugin.html This only works with quite recent ld.bfd versions, not lld or mold afaik. As such using it is not an option, just like using LLVM's.deplibs
section is not an option due to only working with lld.
I think I'd argue that:
In this specific case, it's a bit more difficult, as we'd have to choose one of these options (__.LIBDEP
or .deplibs
), since some linkers might start supporting both, and then the argument order would be jumbled up; so I'll probably file some issues with the linkers themselves, to see which one they'll agree on using.
On Windows we could embed static dependencies in static libraries (using the
.drectve
section) instead of requiring them to be passed on the command line, but Rust doesn't currently.
Hmm, I don't know much about Windows, are there different linkers there, and what are their names? Seems like LLVM's lld
COFF supports the section at least.
But for the linkers that do support it, we should emit the information, since it will still help the users of those linkers, and won't interfere with the others.
Rustc doesn't know which linker will be used and the build system that invokes the linker likely allows picking a linker which doesn't support this feature. As such it has to respect --print native-static-libs
anyway. And if the linker does support it and we were emitting the section/archive member to pass the linker args, then having the build system respect --print native-static-libs
too would duplicate the linker args. If we knew for sure the linker supported it, like we do on macOS, then --print native-static-libs
can simply return an empty list, but for ELF that is not the case.
Problem statement
Using a statically compiled Rust library from another language / with other toolchains can be somewhat troublesome, since the Rust library may have linking requirements that the user then has to manually pass to the linker.
Specific example: I have a static Rust library compiled for macOS that uses Winit and Wgpu and exposes a
rust_main
function, which I then pass to Xcode to link together with a small C main file that calls into Rust. Under the hood, Winit/Wgpu requires linking toAppKit
,Metal
and other such system frameworks. I'm required to also specify in Xcode the frameworks that my binary requires.Apart from being difficult to set up (as I, the user, has to know enough about linking to understand the inscrutable error messages), this workflow is also a possible SemVer hazard, as a library like Winit cannot add a dependency on a new the system library in a minor version without possibly breaking the user's build.
This can be fixed by using the
--print=native-static-libs
rustc
flag, but that probably requires more work for the user to integrate into the workflow than the quick fix of "just link to the broken library", and is also not very discoverable.Proposed solution
Swift and Clang have the concept of "auto linking", where the compiler will instruct the linker to link to the correct external libraries if the user imports code from one of these libraries. This is enabled in Clang with
-fmodules
, and can be further controlled with the-fno-autolink
flag.On the surface, this provides much the same functionality as Rust with its
#[link(...)]
attribute, but there's an important distinction: the dependency information is embedded in the object file, and understood by the linker itself, which allows it to work without Swift/Clang driving the linker invocation!This would fix https://github.com/rust-lang/rust/issues/110143.
Implementation notes
Mach-O binaries uses the
LC_LINKER_OPTION
load command for this, which is understood by LLVM's lld, and Apple's ld64. A few resources on that:ELF binaries linking with LLVM's lld can use the
.linker-options
section or the.deplibs
section.As you can see, this is unfortunately quite platform-specific, and depends on the capabilities of the linker, so it probably isn't solvable in the general case; but I'd argue that this is still something that we could slowly improve support for, since this has a clear graceful fallback.
I'd be willing to (attempt to) implement this if you think this is desired?
Drawbacks / Unknowns
More complex linking integration, would this work for "Rust lib depending on Rust static lib" use-case, if that's even possible today?
Linker arguments are inherently ordered, and may be unexpectedly jumbled by this transformation? Would have to be properly researched and tested.
Sometimes, the user may want to opt-out of this behaviour. This can be done with the linker flag
-ignore_auto_link
, though we should probably document that somewhere once this has been implemented.