Support $ld$ symbols

[smeenai]

Bugzilla Link	49800
Version	unspecified
OS	All
CC	@alexshap,@gkmhub,@int3,@keith,@nico,@smeenai

Extended Description

There's a bunch of symbols prefixed with $ld$ that have special meaning to the linker. The ones I could find:

$ld$hide$ $ld$add$ $ld$weak$ $ld$install_name$ $ld$compatibility_version$ $ld$previous$

All of these except $ld$previous$ have been around for quite some time, and have the effects you would expect from their names. Apple's open source release of tapi has code to handle them, as does ld64's dylib parser. The tapi in LLVM does not appear to have code to handle them so far, as far as I can see.

$ld$previous$ is new in the ld64 609 source code release (corresponding to Xcode 12). It seems to be able to override the install name of the entire library or the install name that particular symbols are associated with. The former functionality is used in Xcode 12.5's SDK for the newly introduced AVFAudio framework; its install name is overridden to AVFoundation if you're targeting iOS 14.5 or newer. The following demonstrates this behavior:

$ cat use.s .long _AVAudioBitRateStrategy_Constant@GOTPCREL

$ llvm-mc -filetype obj -triple x86_64-apple-ios14.0.0-simulator -o use.o use.s $ ld -platform_version ios-simulator 14.0.0 14.5 -arch x86_64 -dylib \ -o libuse.dylib use.o -framework AVFoundation \ -syslibroot /Applications/Xcode_12.5_beta_3.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk $ objdump --macho --dylibs-used libuse.dylib libuse.dylib: libuse.dylib (compatibility version 0.0.0, current version 0.0.0) /System/Library/Frameworks/AVFoundation.framework/AVFoundation (compatibility version 1.0.0, current version 2.0.0)

$ llvm-mc -filetype obj -triple x86_64-apple-ios14.5.0-simulator -o use.o use.s $ ld -platform_version ios-simulator 14.5.0 14.5 -arch x86_64 -dylib \ -o libuse.dylib use.o -framework AVFoundation \ -syslibroot /Applications/Xcode_12.5_beta_3.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk $ objdump --macho --dylibs-used libuse.dylib libuse.dylib: libuse.dylib (compatibility version 0.0.0, current version 0.0.0) /System/Library/Frameworks/AVFoundation.framework/AVFoundation (compatibility version 1.0.0, current version 2.0.0) /System/Library/Frameworks/AVFAudio.framework/AVFAudio (compatibility version 1.0.0, current version 1.0.0)

In the first case, since our minimum version is < 14.5, the install name override kicks in. In the second case, our minimum version is >= 14.5, so we see AVFAudio in the dylibs list. Since LLD doesn't implement this, both cases will add AVFAudio to the dylibs list right now, and we'll produce a binary that can't be run on older OS versions.

Interestingly enough, $ld$previous$ is implemented in a centralized location in ld64 (in its generic dylib parser, which is the base parser class for Mach-O dylibs and TBDs), whereas the other symbols are separately implemented in ld64 and tapi. There's also a FIXME in ld64 609's source to issue warnings for the other $ld$ symbols once $ld$previous$ is submitted and documented, so perhaps the eventual plan is for $ld$previous$ to subsume the functionality of the others and be the only one?

[nico]

mentioned in issue llvm/llvm-bugzilla-archive#50411

[nico]

mentioned in issue llvm/llvm-bugzilla-archive#50304

[keith]

Looks like https://reviews.llvm.org/D103505 implemented some of these

[nico]

Created attachment 24915 [details] Missing $ld$ symbol support

I went through all these symbols. None of them cause problems in practice for us t the moment.

We do depend on _getentropy, but sys/random.h contains

__OSX_AVAILABLE(10.12) __IOS_AVAILABLE(10.0) __TVOS_AVAILABLE(10.0) __WATCHOS_AVAILABLE(3.0)

int getentropy(void* buffer, size_t size);

so we end up with a weak reference to it thanks to clang and __OSX_AVAILABLE and don't need the linker's help. But it'd be nice if the linker also supported $weak$, it feels safer :)

getentropy is interesting because '$ld$weak$os10.11$_getentropy' is in '/usr/lib/system/libsystem_c.dylib' (in the System umbrella), while the actual symbol _getentropy is in '/usr/lib/system/libsystem_kernel.dylib' (in the same umbrella). This suggests that $weak$ etc have to be processed not at dylib-load time, but way later (like ld64 does). (That makes it a bit less clear that https://reviews.llvm.org/D103810 is a good idea.)

[nico]

I'll look at problem (1) since it's my bug from aeae3e0ba9061

https://reviews.llvm.org/D103821

[nico]

(Also, this might be a good idea?

@@ -1033,8 +1033,10 @@ void DylibFile::handleLDPreviousSymbol(StringRef name, StringRef originalName) { std::tie(endVersion, name) = name.split('$'); std::tie(symbolName, rest) = name.split('$'); // TODO: ld64 contains some logic for non-empty symbolName as well.

• if (!symbolName.empty())
• if (!symbolName.empty()) {
• warn("non-empty symbol name, symbol '" + originalName + "' ignored"); return;
• } unsigned platform; if (platformStr.getAsInteger(10, platform) || platform != static_cast(config->platform()))

)

[nico]

I'll look at problem (1) since it's my bug from aeae3e0ba9061

[nico]

Missing $ld$ symbol support

[nico]

hack to print unimplemented magic symbols Thanks for landing partial support! I gave it a try, here's what I found:

1. For install_name, lld doesn't correctly set version fields. QuartzCore reexports CoreImage, but CoreImage has an install_name of QuartzCore on 10.11. So when targeting 10.12+, lld (correctly) writes:

   /System/Library/Frameworks/CoreImage.framework/Versions/A/CoreImage (compatibility version 1.0.1, current version 5.0.0) /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)

But when targeting 10.11, lld writes:

/System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.0.1, current version 5.0.0)

It's good that lld only writes QuartzCore, but it writes it with the version numbers of CoreImage. That's a bug.

2. I hacked up lld to print all magic $ld$ symbols it ignores (hack diff for that attached). It prints a whole bunch of "hide" symbols, and a smattering of "weak" symbols. Full output attached in next comment.

[fae48227-d7ca-407e-ae93-4fde4b3722fd]

Nico, have you tried the latest version ? I'm wondering if there is anything missing now, e.g. if we have a use case for other special symbols or not.

[nico]

Just want to re-iterate that I'm really looking forward to having this fixed and that I'm more than happy to help with this if you're busy with other things.

[nico]

Bug llvm/llvm-bugzilla-archive#50411 has been marked as a duplicate of this bug.

[fae48227-d7ca-407e-ae93-4fde4b3722fd]

This is almost done (we have had a few discussions with Jez, the initial bits will be sent for review very soon), sorry about the delay and thanks for the draft of your diff.

[nico]

(Let me know if you get busy and could use help here after all.)

[nico]

Bug llvm/llvm-bugzilla-archive#50304 has been marked as a duplicate of this bug.

[nico]

sketch Ok, cool :)

This is the last thing I'm currently aware of blocking a chrome canary linked with lld, so I'm looking forward to having this fixed :)

Here's what I had hacked up so far, maybe it's useful. It seems to handle install_name correctly; the rest isn't done. (And no tests.)

[fae48227-d7ca-407e-ae93-4fde4b3722fd]

I've almost implemented it already, have not sent for review yet. Will do ~this week.

[nico]

I'll grab this if you don't mind.

[smeenai]

llvm/llvm-bugzilla-archive#50304 is an example bug (likely) caused by LLD not respecting an $ld$install_name symbol.

[smeenai]

assigned to @alexshap