Closed lorentey closed 1 year ago
Building using xcodebuild with library evolution enabled currently breaks due to #54, under Swift 5.7.1 (and 5.8). We will have to live with this for now.
[22 xcodebuild.library-evolution] xcodebuild -scheme swift-atomics -destination generic/platform=macOS -destination generic/platform=iOS BUILD_LIBRARY_FOR_DISTRIBUTION=YES
/Users/lorentey/Library/Developer/Xcode/DerivedData/swift-atomics-ceuvqitsfiwgtwejiuxxrfrjfari/Build/Intermediates.noindex/swift-atomics.build/Debug-iphoneos/AtomicsTests.build/Objects-normal/arm64/AtomicsTests.private.swiftinterface:4:8: error: missing required module '_AtomicsShims'
import Atomics
^
/Users/lorentey/Library/Developer/Xcode/DerivedData/swift-atomics-ceuvqitsfiwgtwejiuxxrfrjfari/Build/Intermediates.noindex/swift-atomics.build/Debug-iphoneos/AtomicsTests.build/Objects-normal/arm64/AtomicsTests.private.swiftinterface:1:1: error: failed to verify module interface of 'AtomicsTests' due to the errors above; the textual interface may be broken by project issues or a compiler bug
// swift-interface-format-version: 1.0
^
TSan tests are failing on Ubuntu with signal 9. I'm ignoring this.
xcodebuild.test.macCatalyst
is failing with 5.6.1 under macOS due to an Xcode issue with test bundles and code signing on Mac Catalyst. This is a known problem that only affects arm64 builds. Ignoring.
xctest (21578) encountered an error (Failed to load the test bundle. If you believe this error represents a bug, please attach the result bundle at /tmp/run-full-tests.sh.1BHWc/xcodebuild/Logs/Test/Test-swift-atomics-2023.03.31_17-16-10--0700.xcresult. (Underlying Error: The bundle “AtomicsTests” couldn’t be loaded. The bundle couldn’t be loaded. Try reinstalling the bundle. dlopen(/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests, 0x0109): tried: '/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests' (no such file), '/System/iOSSupport/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/System/iOSSupport/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (no such file), '/System/iOSSupport/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (no such file), '/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (code signature in <79432E30-443D-3FCD-92F6-CCEE25F4D743> '/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' not valid for use in process: Trying to load an unsigned library), '/System/Volumes/Preboot/Cryptexes/OS/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (no such file), '/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (code signature in <79432E30-443D-3FCD-92F6-CCEE25F4D743> '/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' not valid for use in process: Trying to load an unsigned library), '/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests' (no such file), '/System/iOSSupport/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/System/iOSSupport/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (no such file), '/System/iOSSupport/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (no such file), '/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (code signature in <79432E30-443D-3FCD-92F6-CCEE25F4D743> '/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' not valid for use in process: Trying to load an unsigned library), '/System/Volumes/Preboot/Cryptexes/OS/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (no such file), '/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' (code signature in <79432E30-443D-3FCD-92F6-CCEE25F4D743> '/private/tmp/run-full-tests.sh.1BHWc/xcodebuild/Build/Products/Release-maccatalyst/AtomicsTests.xctest/Contents/MacOS/AtomicsTests' not valid for use in process: Trying to load an unsigned library)))
Evidently Swift Atomics has been quietly failing on Windows since forever.
(https://github.com/apple/swift-atomics/issues/56, https://github.com/apple/swift/issues/60534, https://github.com/apple/swift-installer-scripts/pull/144)
My own Windows testing is blocked on elementary problems with Swift packages on my Windows box that I don't really have time or energy to investigate.
<unknown>:0: error: missing required module 'SwiftShims'
Ah, reinstalling VS2019 & Swift 5.9 restored SwiftPM to working order. Windows platform support continues to be blocked on https://github.com/apple/swift-atomics/issues/56. Ignoring.
Ah, reinstalling VS2019 & Swift 5.9 restored SwiftPM to working order. Windows platform support continues to be blocked on #56. Ignoring.
Windows support for Swift Atomics 1.0.x did work with VS 2022, but I’m not sure if 1.1 breaks it again.
We’ll try to solve the problem in main
, and then back port it to 5.8/5.9. The workaround continues to make sense.
I got further! My steps, for posterity:
Carefully read the Getting Started instructions at https://www.swift.org/getting-started/#on-windows. Then sigh and forget them.
Install VS 2022 with the following incantation:
curl -sOL https://aka.ms/vs/17/release/vs_community.exe
start /w vs_community.exe --passive --wait --norestart --nocache --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\Community" --add Microsoft.VisualStudio.Component.Windows10SDK.19041 --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64
del /q vs_community.exe
Download and install swift 5.8 from swift.org.
~Add a random symlink:~ (This isn't necessary with VS2022)
cd \Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\lib\swift
mklink /D clang ..\clang\15.0.0
swift build
in a local clone of swift-atomics will now progress to:
C:\Users\Karoy Lorentey\Documents\swift-atomics>swift --version
compnerd.org Swift version 5.8 (swift-5.8-RELEASE)
Target: x86_64-unknown-windows-msvc
C:\Users\Karoy Lorentey\Documents\swift-atomics>swift build
Building for debugging...
error: emit-module command failed due to exception 3 (use -v to see invocation)
error: failed parsing the Swift compiler output: unexpected JSON message: : dataCorrupted(Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "kind", intValue: nil)], debugDescription: "invalid kind", underlyingError: nil))
This happens both on main
and the 1.0.3
tag, so I won't delay the 1.1.0 release because of it.
C:\Users\Karoy Lorentey\Documents\swift-atomics>swift build Building for debugging... error: emit-module command failed due to exception 3 (use -v to see invocation) error: failed parsing the Swift compiler output: unexpected JSON message: : dataCorrupted(Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "kind", intValue: nil)], debugDescription: "invalid kind", underlyingError: nil))
Interesting… What about swift build -v
? I’d like to inspect the failing command line.
Here goes!
C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin\swift-frontend.exe -frontend -emit-module -experimental-skip-non-inlinable-function-bodies-without-types C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\AtomicInteger.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\AtomicMemoryOrderings.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\AtomicOptional.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\AtomicRawRepresentable.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\AtomicStrongReference.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\AtomicValue.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\DoubleWord.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\autogenerated\AtomicBool.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\autogenerated\AtomicLazyReference.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\autogenerated\HighLevelTypes.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\autogenerated\IntegerConformances.swift C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\Atomics\autogenerated\PointerConformances.swift -target x86_64-unknown-windows-msvc -disable-objc-interop -sdk C:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk -I C:\Users\Karoy Lorentey\Documents\swift-atomics\.build\x86_64-unknown-windows-msvc\debug -I C:\Library\Developer\Platforms\Windows.platform\Developer\Library\XCTest-development\usr\lib\swift\windows -I C:\Library\Developer\Platforms\Windows.platform\Developer\Library\XCTest-development\usr\lib\swift\windows\x86_64 -color-diagnostics -enable-testing -g -module-cache-path C:\Users\Karoy Lorentey\Documents\swift-atomics\.build\x86_64-unknown-windows-msvc\debug\ModuleCache -swift-version 5 -Onone -D SWIFT_PACKAGE -D DEBUG -empty-abi-descriptor -resource-dir C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\lib\swift -enable-anonymous-context-mangled-names -Xcc -fmodule-map-file=C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\_AtomicsShims\include\module.modulemap -Xcc -I -Xcc C:\Users\Karoy Lorentey\Documents\swift-atomics\Sources\_AtomicsShims\include -module-name Atomics -emit-module-doc-path C:\Users\Karoy Lorentey\Documents\swift-atomics\.build\x86_64-unknown-windows-msvc\debug\Atomics.swiftdoc -emit-module-source-info-path C:\Users\Karoy Lorentey\Documents\swift-atomics\.build\x86_64-unknown-windows-msvc\debug\Atomics.swiftsourceinfo -emit-dependencies-path C:\Users\Karoy Lorentey\Documents\swift-atomics\.build\x86_64-unknown-windows-msvc\debug\Atomics.build\Atomics.emit-module.d -parse-as-library -o C:\Users\Karoy Lorentey\Documents\swift-atomics\.build\x86_64-unknown-windows-msvc\debug\Atomics.swiftmodule
error: compile command failed due to exception 3 (use -v to see invocation)
error: failed parsing the Swift compiler output: unexpected JSON message: : dataCorrupted(Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "kind", intValue: nil)], debugDescription: "invalid kind", underlyingError: nil))
I cannot reproduce it with Swift 5.7.3🤔️ With VS 17.5.3 and Swift 5.7.3, you don’t even need the mklink
workaround to build Atomics.
I suspect this could be some Swift 5.8 regression. -experimental-skip-non-inlinable-function-bodies-without-types
is the most suspicious, and I didn’t notice it with Swift 5.7.
Huh. I also see this error with 5.7.3, so presumably something is messed up on my end.
Can I delegate the testing to you? 🙈 Do these pass for you on 5.7.3?
swift test -c debug
swift test -c release
The need for custom installation steps will preclude Windows from being a supported platform, but it would be good to know if the underlying code will eventually work!
Hm, xcodebuild.build.watchOS-simulator
is failing on Xcode 14.3 (Swift 5.8). Apparently the generic/platform=watchOS Simulator
destination still tries to build i386 code, but predictably Xcode doesn't ship an XCTest overlay for it.
xcrun xcodebuild -scheme swift-atomics -configuration Release -destination generic/platform=watchOS Simulator -derivedDataPath /tmp/run-full-tests.sh.s6EQf/xcodebuild build
...
Ld /tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/swift-atomics.build/Release-watchsimulator/AtomicsTests.build/Objects-normal/i386/Binary/AtomicsTests normal i386 (in target 'AtomicsTests' from project 'swift-atomics')
cd /Users/lorentey/Git/swift-atomics
/Applications/Xcode-MAS.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -Xlinker -reproducible -target i386-apple-watchos4.0-simulator -bundle -isysroot /Applications/Xcode-MAS.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator9.4.sdk -L/tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/EagerLinkingTBDs/Release-watchsimulator -L/tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Products/Release-watchsimulator -L/Applications/Xcode-MAS.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/usr/lib -F/tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/EagerLinkingTBDs/Release-watchsimulator -F/tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Products/Release-watchsimulator/PackageFrameworks -F/tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Products/Release-watchsimulator/PackageFrameworks -F/tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Products/Release-watchsimulator -iframework /Applications/Xcode-MAS.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/Library/Frameworks -iframework /Applications/Xcode-MAS.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/SDKs/WatchSimulator9.4.sdk/Developer/Library/Frameworks -filelist /tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/swift-atomics.build/Release-watchsimulator/AtomicsTests.build/Objects-normal/i386/AtomicsTests.LinkFileList -Xlinker -rpath -Xlinker /usr/lib/swift -Xlinker -rpath -Xlinker @loader_path/Frameworks -Xlinker -rpath -Xlinker @loader_path/../Frameworks -Xlinker -rpath -Xlinker @loader_path/Frameworks -dead_strip -Xlinker -object_path_lto -Xlinker /tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/swift-atomics.build/Release-watchsimulator/AtomicsTests.build/Objects-normal/i386/AtomicsTests_lto.o -Xlinker -objc_abi_version -Xlinker 2 -fobjc-link-runtime -L/Applications/Xcode-MAS.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/watchsimulator -L/usr/lib/swift -Xlinker -add_ast_path -Xlinker /tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/swift-atomics.build/Release-watchsimulator/AtomicsTests.build/Objects-normal/i386/AtomicsTests.swiftmodule -framework XCTest -Xlinker -no_adhoc_codesign -Xlinker -dependency_info -Xlinker /tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/swift-atomics.build/Release-watchsimulator/AtomicsTests.build/Objects-normal/i386/AtomicsTests_dependency_info.dat -o /tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/swift-atomics.build/Release-watchsimulator/AtomicsTests.build/Objects-normal/i386/Binary/AtomicsTests -Xlinker -add_ast_path -Xlinker /tmp/run-full-tests.sh.s6EQf/xcodebuild/Build/Intermediates.noindex/swift-atomics.build/Release-watchsimulator/Atomics.build/Objects-normal/i386/Atomics.swiftmodule
ld: warning: ignoring file /Applications/Xcode-MAS.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/Library/Frameworks/XCTest.framework/XCTest, file is universal (x86_64,arm64) but does not contain the i386 architecture: /Applications/Xcode-MAS.app/Contents/Developer/Platforms/WatchSimulator.platform/Developer/Library/Frameworks/XCTest.framework/XCTest
ld: warning: Could not find or use auto-linked library 'XCTestSwiftSupport'
Undefined symbols for architecture i386:
[...]
This is an Xcode issue (or perhaps a problem in the test script). Ignoring.
The need for custom installation steps will preclude Windows from being a supported platform, but it would be good to know if the underlying code will eventually work!
AFAIK VS 2022 (17.5+) actually eliminates the need for it. A fix through the installer is still desired anyway. We may need to clarify that at the moment we need VS 17.5+ on Windows.
swift test -c debug
(passed)swift test -c release
(in progress)That is excellent news! I'll include some text announcing experimental Windows support in the release notes. (I really hope we'll soon be able to migrate this package to native atomic intrinsics -- that should eliminate all of the C-related headaches.)
I think I'll do most Intel testing on macOS via Rosetta, with limited testing on an actual Intel CPU on Monday.
Oops:
Test Suite 'BasicAtomicDoubleWordTests' started at 2023-04-01 03:36:23.462
Test Case 'BasicAtomicDoubleWordTests.test_compareExchange_acquiring' started at 2023-04-01 03:36:23.462
Assertion failed: atomic_is_lock_free(&storage.value), file C:\Users\Administrator\Developer\swift-atomics\Sources\_AtomicsShims\include/_AtomicsShims.h, line 227
Oh, that's unfortunate. Windows definitely requires CMPXCHG16B these days, so this must be due to the C stdlib not getting updated. Native atomics should solve that, eventually.
xcodebuild.test.tvOS-simulator
failed on Xcode 13.4.1 (Swift 5.6.1) under Rosetta -- it wasn't able to find a suitable simulator device. I'm going to ignore this as unrelated noise.
xcodebuild
doesn't like building projects with swift.org toolchains, so I'll limit 5.9/main testing to SwiftPM and CMake.
2023-04-01 00:52:13.552 xcodebuild[49178:697838] Writing error result bundle to /var/folders/hb/5vvxj2m13_7_h40q22pt9s9w0000gn/T/ResultBundle_2023-01-04_00-52-0013.xcresult
xcodebuild: error: Could not resolve package dependencies:
Invalid manifest (compiled with: ["/Library/Developer/Toolchains/swift-5.9-DEVELOPMENT-SNAPSHOT-2023-03-30-a.xctoolchain/usr/bin/swiftc", "-vfsoverlay", "/var/folders/hb/5vvxj2m13_7_h40q22pt9s9w0000gn/T/TemporaryDirectory.anJQwJ/vfs.yaml", "-L", "/Applications/Xcode-MAS.app/Contents/PlugIns/IDESwiftPackageCore.framework/Versions/A/Frameworks/SwiftPM.framework/SharedSupport/ManifestAPI", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/Applications/Xcode-MAS.app/Contents/PlugIns/IDESwiftPackageCore.framework/Versions/A/Frameworks/SwiftPM.framework/SharedSupport/ManifestAPI", "-target", "x86_64-apple-macos12.0", "-sdk", "/Applications/Xcode-MAS.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk", "-swift-version", "5", "-I", "/Applications/Xcode-MAS.app/Contents/PlugIns/IDESwiftPackageCore.framework/Versions/A/Frameworks/SwiftPM.framework/SharedSupport/ManifestAPI", "-sdk", "/Applications/Xcode-MAS.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk", "-package-description-version", "5.6.0", "-Xfrontend", "-serialize-diagnostics-path", "-Xfrontend", "/Users/lorentey/Library/Caches/org.swift.swiftpm/manifests/ManifestLoading/swift-atomics.dia", "/Users/lorentey/Git/swift-atomics/Package.swift", "-disallow-use-new-driver", "-o", "/var/folders/hb/5vvxj2m13_7_h40q22pt9s9w0000gn/T/TemporaryDirectory.YEdaUn/swift-atomics-manifest"])
<unknown>:0: warning: legacy driver is now deprecated; consider avoiding specifying '-disallow-use-new-driver'
<unknown>:0: error: unable to execute command: <unknown>
Ah, my TSan tests on Linux appear to be failing because they're running out of memory, and the process gets killed by the kernel. 💡Fair enough.
OK, testing complete; 1.1 is ready to get tagged.
Apparently the
generic/platform=watchOS Simulator
destination still tries to build i386 code, but predictably Xcode doesn't ship an XCTest overlay for it.xcrun xcodebuild -scheme swift-atomics -configuration Release -destination generic/platform=watchOS Simulator -derivedDataPath /tmp/run-full-tests.sh.s6EQf/xcodebuild build
Just wanted to mention that I ran into this recently in Xcode 15 and a quick workaround that I found was to use the non-Simulator variant.
-destination "generic/platform=watchOS"
Also works in tandem with Any watchOS Device
. (Note that using Any watchOS Simulator Device
still fails.)
-destination "generic/platform=watchOS,name=Any watchOS Device"
Candidate commit: 6c89474e62719ddcc1e9614989fff2f68208fe10
Diff: https://github.com/apple/swift-atomics/compare/1.0.3...main
Testing progress: (On Mac and Linux, this is done by running
Utilities/run-full-tests.sh
. This script builds and tests this package using various build systems and build configurations.)macOS/AS
macOS/x86_64
Linux/Aarch64
Linux/x86_64
Windows 11/x86_64 (experimental)
Note: ~Swift 5.8 has not shipped yet; I'll hold testing that for now.~