search

swiftlang / swift

The Swift Programming Language
https://swift.org
Apache License 2.0
67.31k stars 10.34k forks source link

[SR-11362] Include bitcode in Xcode toolchain libraries #53763

Open dmcyk opened 5 years ago

dmcyk commented 5 years ago
Previous ID SR-11362
Radar rdar://problem/54636425
Original Reporter @dmcyk
Type Improvement
Additional Detail from JIRA | | | |------------------|-----------------| |Votes | 0 | |Component/s | | |Labels | Improvement, Runtime | |Assignee | None | |Priority | Medium | md5: 728f539cd14772a5a86b33c3d613fa52

Issue Description:

Env:
macOS 10.14.6
Xcode 11 beta 6

Xcode toolchain recently added some new Swift 5.0 compatibility libraries (noticed in Xcode 11 beta 6 as well as open source toolchain):

Those libraries are included for all platforms, not only macOS, what is their goal? Is it to provide compatibility for e.g. dynamic replacements on older OSes?

This is problematic for testing bitcode on older macOS releases than 10.15 because Swift compiler apparently tries to always link those libraries, even when Swift 5.1 features are not used.
Running `swiftc -embed-bitcode empty.swift` with empty Swift source files fails to include bitcode because the listed libraries do not have it.

ld: warning: all bitcode will be dropped because '/Applications/Xcode_11.0_b6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftCompatibility50.a(ProtocolConformance.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Applications/Xcode_11.0_b6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftCompatibility50.a(Overrides.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Applications/Xcode_11.0_b6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftCompatibilityDynamicReplacements.a(DynamicReplaceable.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.

belkadan commented 5 years ago

cc @jckarter, aschwaighofer@apple.com (JIRA User), @mikeash. Oops.

mikeash commented 5 years ago

To answer the questions: libswiftCompatibilityDynamicReplacements.a does indeed exist to support dynamic replacements on older OSes. It should be inert if you're not using dynamic replacements (other than this serious problem you're pointing out...) and I think dead code stripping will mean the code doesn't even get included in the final binary in that case. libswiftCompatibility50.a contains a couple of important fixes for runtime bugs and uses the compatibility override mechanism in the runtime (https://github.com/apple/swift/blob/master/stdlib/public/runtime/CompatibilityOverride.cpp) to get those fixes into the 5.0 runtime.

Also: yes, oops.

jckarter commented 5 years ago

The libraries provide backward compatibility for new runtime features and also backport fixes for known runtime bugs to older OSes. They ought to be built with bitcode for platforms that support bitcode. Since bitcode isn't used for anything on macOS, it looks like we don't build bitcode slices for anything in the toolchain for macOS, not only these archives (though by their nature, archives might be the first thing that show there's a problem). `libarclite` for instance also does not include bitcode for macOS, so you'd have the same problem building bitcode when targeting macOS \<10.10.

jckarter commented 5 years ago

You can disable linking these libraries by passing -runtime-compatibility-version none to the compiler.

aschwaighofer commented 5 years ago

And -Xfrontend -disable-autolinking-runtime-compatibility-dynamic-replacements to disable linking libswiftCompatibilityDynamicReplacements.a.

belkadan commented 5 years ago

-runtime-compatibility-version should cover both libraries.

dmcyk commented 5 years ago

Thanks for the explanation, I appreciate quick feedback!
Indeed iphoneos and others do include bitcode. Guess it makes sense that those libraries do not have bitcode, though for testing and experimentation purposes it'd be convenient if they did. Adding bitcode there is something that could be considered?

With -runtime-compatibility-version only I'm still getting missing bitcode warning for the libswiftCompatibilityDynamicReplacements.a, adding also the flag aschwaighofer@apple.com (JIRA User) mentioned solved it.

swift-ci commented 3 years ago

Comment by Alex LIU (JIRA)

Hi guys, I think we are currently facing the similar situation.

We have a framework project which was written in objc but we are trying to write new features with Swift. We also configured correctly so that we can call objc function from Swift and vice versa but the same problem happens when we build it with `xcodebuild build` command. (due to CI/CD requirement we have to use command line to build a framework)

ld: warning: all bitcode will be dropped because '/tmp/DerivedData/xxxxxxxx.build/Objects-normal/x86_64/NewSwiftTestFeature.o' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibility50.a(Overrides.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibility51.a(Overrides.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibilityDynamicReplacements.a(DynamicReplaceable.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibility50.a(ProtocolConformance.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.
ld: warning: all bitcode will be dropped because '/Users/xxxxx/Downloads/Xcode12.4_RC.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftCompatibility51.a(ProtocolConformance.cpp.o)' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.

The issue is for this x86_64 arch we do need bit code enabled in this build however when we check it `otool` the framework doesn't have bitcode enabled. I guess due to the warning above at least. How should I make bitcode enabled or how can I use "-runtime-compatibility-version" argument along with `xcodebuild` tool? Please help, thanks.

mikeash commented 3 years ago

You can add the flags to Other Swift Flags in your Xcode project, and then xcodebuild will pick them up. Or you can specify them in the command like, xcodebuild OTHER_SWIFT_FLAGS='-runtime-compatibility-version none'.

swift-ci commented 3 years ago

Comment by Alex LIU (JIRA)

Thank you so much for your quick reply, @mikeash

It worked since I don't see those warnings about swift Lib compatibility anymore except one...

I have a file called `NewSwiftTestFeature` which is the only Swift file I added into the entire objc project, however i'm still getting this erorr which will make my output binary with disabled bitcode. Any configuration I should change in order to make bitcode enabled?

ld: warning: all bitcode will be dropped because '/tmp/DerivedData/xxxx-DerivedData/Build/Intermediates.noindex/xxxx.build/Release-iphonesimulator/xxxx.build/Objects-normal/arm64/NewSwiftTestFeature.o' was built without bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target.

Here's my xcodebuild command:

xcodebuild build BITCODE_GENERATION_MODE=bitcode \    
-configuration "$CONFIGURATION" \    
-derivedDataPath "${DERIVEDDATA_PATH}" \    
-scheme "$SCHEME" \    
-sdk iphonesimulator \    
-destination "generic/platform=iOS Simulator" \    
BUILD_LIBRARY_FOR_DISTRIBUTION=YES \    
SKIP_INSTALL=NO \    
PRODUCT_NAME="xxx" \    
EXECUTABLE_PREFIX=""  \    
OTHER_SWIFT_FLAGS='-runtime-compatibility-version none'
cachemeifyoucan commented 3 years ago

Here are few things about bitcode:

In your situation, you can safely ignore those warnings. The compatibility libraries are built with bitcode for iphoneos/watchOS/tvOS so you won't hit that problem during archive build for submission. On the other hand, feel free to file a bug report to feedbackassistant.apple.com or using the feedback app for suggestion to improve with spurious warnings.

swift-ci commented 2 years ago

Comment by Sarah Koop (JIRA)

We are also experiencing this issue when building with xcframeworks on an M1 Mac. Bitcode is enabled for the modules within the SDK that we are testing with. We are not able to replicate this issue on an Intel Mac.

By default, Xcode sets ENABLE_BITCODE to YES. We are able to consistently reproduce this issue, if we toggle this setting to NO and back to YES.

Steps to reproduce:

  1. Clone the braintree_ios repo and build the Demo app
  2. Toggle Demo Xcode project ENABLE_BITCODE Build Setting to NO
  3. Build the project
  4. Toggle back to YES
  5. Build the project
  6. See warnings similar to the ones posted above

When we ran the sample app on a device, these warnings/errors went away. We were also able to archive successfully.
There appears to be an Apple Developer Forum Issue] about this also.