search

SwiftPackageIndex / SwiftPackageIndex-Server

The Swift Package Index is the place to find Swift packages!
https://swiftpackageindex.com
Apache License 2.0
522 stars 36 forks source link

Swift 6 preview toolchain `xcodebuild` failure #2950

Open finestructure opened 3 months ago

finestructure commented 3 months ago

This is a tracking issue for the

<unknown>:0: warning: legacy driver is now deprecated; consider avoiding specifying '-disallow-use-new-driver'
<unknown>:0: error: unable to execute command: <unknown>

error with current Swift 6 toolchains.

Test command:

git clone https://github.com/SwiftPackageIndex/SemanticVersion \
  && cd SemanticVersion \
  && env DEVELOPER_DIR="/Applications/Xcode-15.3.0.app" \
  xcrun --toolchain $TOOLCHAIN_ID \
  xcodebuild build -scheme "SemanticVersion" -destination "generic/platform=macOS"

Toolchains tested:

Status Date Toolchain ID
❌ Feb 15 org.swift.59202402151a
❌ Feb 29 org.swift.59202402291a
❌ Mar 5 org.swift.59202403051a
❌ Mar 13 org.swift.59202403131a
❌ Mar 26 org.swift.59202403261a
❌ Mar 30 org.swift.59202403301a
❌ Apr 1 org.swift.59202404011a
❌ Apr 15 org.swift.600202404151a*)
❌ Apr 22 org.swift.600202404221a
❌ Apr 30 org.swift.600202404301a**)
❌ May 14 org.swift.600202405141a**)
❌ May 26 org.swift.600202405261a**)
❌ June 22 org.swift.600202406221a***)

) also tested with Xcode 15.4b1 ) tested with Xcode 15.4 ) tested with Xcode 15.4 and 16b2

finestructure commented 3 months ago

A potential workaround is

defaults write com.apple.dt.Xcode IDEPackageSupportVerboseManifestLoading -bool YES

However, this also fails, although with a different error:

Cloning into 'SemanticVersion'...
remote: Enumerating objects: 293, done.
remote: Counting objects: 100% (47/47), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 293 (delta 11), reused 15 (delta 4), pack-reused 246
Receiving objects: 100% (293/293), 66.75 KiB | 697.00 KiB/s, done.
Resolving deltas: 100% (127/127), done.
Command line invocation:
    /Applications/Xcode-15.3.0.app/Contents/Developer/usr/bin/xcodebuild build -scheme SemanticVersion -destination generic/platform=macOS

User defaults from command line:
    IDEPackageSupportToolchainOverrideForManifestLoading = org.swift.59202403301a
    IDEPackageSupportUseBuiltinSCM = YES

Build settings from command line:
    TOOLCHAINS = org.swift.59202403301a

Resolve Package Graph

Package: semanticversion

Resolved source packages:
  SemanticVersion: /Users/sas/Projects/SPI/spi-server/SemanticVersion

2024-04-01 19:36:06.823 xcodebuild[71564:26214690] Writing error result bundle to /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/ResultBundle_2024-01-04_19-36-0006.xcresult
xcodebuild: error: Could not resolve package dependencies:
  Invalid manifest (compiled with: ["/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a.xctoolchain/usr/bin/swiftc", "-vfsoverlay", "/var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.Niu6RQ/vfs.yaml", "-L", "/Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI", "-target", "arm64-apple-macos13.0", "-sdk", "/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk", "-swift-version", "5", "-I", "/Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI", "-sdk", "/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk", "-package-description-version", "5.6.0", "/Users/sas/Projects/SPI/spi-server/SemanticVersion/Package.swift", "-v", "-o", "/var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.4XXHCq/semanticversion-manifest"])
/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file /Users/sas/Projects/SPI/spi-server/SemanticVersion/Package.swift -target arm64-apple-macos13.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -sdk /Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk -I /Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI -vfsoverlay /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.Niu6RQ/vfs.yaml -swift-version 5 -package-description-version 5.6.0 -empty-abi-descriptor -Xcc -working-directory -Xcc /Users/sas/Projects/SPI/spi-server/SemanticVersion -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a.xctoolchain/usr/lib/swift -module-name main -target-sdk-version 14.4 -target-sdk-name macosx14.4 -external-plugin-path '/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/swift/host/plugins#/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server' -external-plugin-path '/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/local/lib/swift/host/plugins#/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server' -plugin-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a.xctoolchain/usr/lib/swift/host/plugins -plugin-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a.xctoolchain/usr/local/lib/swift/host/plugins -o /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.qNdkE1/Package-1.o
/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a.xctoolchain/usr/bin/clang /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.qNdkE1/Package-1.o --sysroot /Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk --target=arm64-apple-macos13.0 /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a.xctoolchain/usr/lib/swift/macosx/libswiftCompatibilityPacks.a -L /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a.xctoolchain/usr/lib/swift/macosx -L /Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk/usr/lib/swift -L /Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI -L /Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI -lPackageDescription -Xlinker -rpath -Xlinker /Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI -o /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.4XXHCq/semanticversion-manifest
Apple Swift version 6.0-dev (LLVM a8cd54c4da01fef, Swift 365da5f0ff87343)
Target: arm64-apple-macos13.0
error: link command failed with exit code 1 (use -v to see invocation)
clang: error: unable to execute command: Executable "ld" doesn't exist!
clang: error: linker command failed with exit code 1 (use -v to see invocation)
finestructure commented 3 months ago

Another suggestion is to symlink to ld, which is the underlying error.

Doing so via

cd /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/bin
sudo ln -s `xcrun -find ld` .

seems to be getting further but still errors out in the end:

Cloning into 'SemanticVersion'...
remote: Enumerating objects: 293, done.
remote: Counting objects: 100% (47/47), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 293 (delta 11), reused 15 (delta 4), pack-reused 246
Receiving objects: 100% (293/293), 66.75 KiB | 768.00 KiB/s, done.
Resolving deltas: 100% (127/127), done.
Command line invocation:
    /Applications/Xcode-15.3.0.app/Contents/Developer/usr/bin/xcodebuild build -scheme SemanticVersion -destination generic/platform=macOS

User defaults from command line:
    IDEPackageSupportToolchainOverrideForManifestLoading = org.swift.59202404011a
    IDEPackageSupportUseBuiltinSCM = YES

Build settings from command line:
    TOOLCHAINS = org.swift.59202404011a

Resolve Package Graph
/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file /Users/sas/Downloads/SemanticVersion/Package.swift -target arm64-apple-macos13.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -sdk /Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk -I /Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI -vfsoverlay /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.kAYEQP/vfs.yaml -swift-version 5 -package-description-version 5.6.0 -empty-abi-descriptor -Xcc -working-directory -Xcc /Users/sas/Downloads/SemanticVersion -resource-dir /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/lib/swift -module-name main -target-sdk-version 14.4 -target-sdk-name macosx14.4 -external-plugin-path '/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/swift/host/plugins#/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server' -external-plugin-path '/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/local/lib/swift/host/plugins#/Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server' -plugin-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/lib/swift/host/plugins -plugin-path /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/local/lib/swift/host/plugins -o /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.sAYaDq/Package-1.o
/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/bin/clang /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.sAYaDq/Package-1.o --sysroot /Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk --target=arm64-apple-macos13.0 /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/lib/swift/macosx/libswiftCompatibilityPacks.a -L /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/lib/swift/macosx -L /Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk/usr/lib/swift -L /Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI -L /Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI -lPackageDescription -Xlinker -rpath -Xlinker /Applications/Xcode-15.3.0.app/Contents/SharedFrameworks/SwiftPM.framework/SharedSupport/ManifestAPI -o /var/folders/nk/tlpts6bs799038htr6rx1xnc0000gn/T/TemporaryDirectory.5J7Ncp/semanticversion-manifest
Apple Swift version 6.0-dev (LLVM a8cd54c4da01fef, Swift bdc37178347d423)
Target: arm64-apple-macos13.0

Resolved source packages:
  SemanticVersion: /Users/sas/Downloads/SemanticVersion

--- xcodebuild: WARNING: Using the first of multiple matching destinations:
{ platform:macOS, name:Any Mac }
{ platform:macOS, variant:Mac Catalyst, name:Any Mac }
Prepare packages

ComputeTargetDependencyGraph
note: Building targets in dependency order
note: Target dependency graph (3 targets)
    Target 'SemanticVersion' in project 'SemanticVersion'
        ➜ Explicit dependency on target 'SemanticVersion' in project 'SemanticVersion'
        ➜ Explicit dependency on target 'SemanticVersion_SemanticVersion' in project 'SemanticVersion'
    Target 'SemanticVersion' in project 'SemanticVersion'
        ➜ Explicit dependency on target 'SemanticVersion_SemanticVersion' in project 'SemanticVersion'
    Target 'SemanticVersion_SemanticVersion' in project 'SemanticVersion' (no dependencies)

GatherProvisioningInputs

CreateBuildDescription

ExecuteExternalTool /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/bin/clang -v -E -dM -isysroot /Applications/Xcode-15.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk -x c -c /dev/null

ExecuteExternalTool /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/bin/swiftc -v

ExecuteExternalTool /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/bin/ld -version_details

Build description signature: 3b6795b60f3ed5c84fc3b9de0a4a9620
Build description path: /Users/sas/Library/Developer/Xcode/DerivedData/SemanticVersion-fevvtnuqvvgsudbpylpukllorhdk/Build/Intermediates.noindex/XCBuildData/3b6795b60f3ed5c84fc3b9de0a4a9620.xcbuilddata
note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'.
error: dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "Unexpected end of file" UserInfo={NSDebugDescription=Unexpected end of file})))
error: dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "Unexpected end of file" UserInfo={NSDebugDescription=Unexpected end of file})))
note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'. (in target 'SemanticVersion' from project 'SemanticVersion')
note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'. (in target 'SemanticVersion_SemanticVersion' from project 'SemanticVersion')
note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'. (in target 'SemanticVersion' from project 'SemanticVersion')
** BUILD FAILED **

The ld I linked has the following version info:

❯ /usr/bin/ld -version_details
{
    "version": "1053.12",
    "architectures": [
        "armv6",
        "armv7",
        "armv7s",
        "arm64",
        "arm64e",
        "arm64_32",
        "i386",
        "x86_64",
        "x86_64h",
        "armv6m",
        "armv7k",
        "armv7m",
        "armv7em"
    ],
    "lto": {
        "runtime_api_version": 29,
        "static_api_version": 29,
        "version_string": "LLVM version 15.0.0"
    },
    "tapi": {
        "version": "15.0.0",
        "version_string": "Apple TAPI version 15.0.0 (tapi-1500.3.2.2)"
    }
}

which doesn't seem to work with this toolchain. There doesn't seem to be another ld in the toolchain to use instead, though.

finestructure commented 3 months ago

Linking to xcrun -find ld instead:

cd /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-01-a.xctoolchain/usr/bin
sudo ln -s `xcrun -find ld` .

doesn't change the result unfortunately:

Command line invocation:
    /Applications/Xcode-15.3.0.app/Contents/Developer/usr/bin/xcodebuild build -scheme SemanticVersion -destination generic/platform=macOS

User defaults from command line:
    IDEPackageSupportToolchainOverrideForManifestLoading = org.swift.59202404011a
    IDEPackageSupportUseBuiltinSCM = YES

Build settings from command line:
    TOOLCHAINS = org.swift.59202404011a

Resolve Package Graph

Resolved source packages:
  SemanticVersion: /Users/sas/Downloads/SemanticVersion

--- xcodebuild: WARNING: Using the first of multiple matching destinations:
{ platform:macOS, name:Any Mac }
{ platform:macOS, variant:Mac Catalyst, name:Any Mac }
Prepare packages

ComputeTargetDependencyGraph
note: Building targets in dependency order
note: Target dependency graph (3 targets)
    Target 'SemanticVersion' in project 'SemanticVersion'
        ➜ Explicit dependency on target 'SemanticVersion' in project 'SemanticVersion'
        ➜ Explicit dependency on target 'SemanticVersion_SemanticVersion' in project 'SemanticVersion'
    Target 'SemanticVersion' in project 'SemanticVersion'
        ➜ Explicit dependency on target 'SemanticVersion_SemanticVersion' in project 'SemanticVersion'
    Target 'SemanticVersion_SemanticVersion' in project 'SemanticVersion' (no dependencies)

GatherProvisioningInputs

CreateBuildDescription

error: dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "Unexpected end of file" UserInfo={NSDebugDescription=Unexpected end of file})))
error: dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "Unexpected end of file" UserInfo={NSDebugDescription=Unexpected end of file})))
note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'.
note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'. (in target 'SemanticVersion_SemanticVersion' from project 'SemanticVersion')
note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'. (in target 'SemanticVersion' from project 'SemanticVersion')
note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'. (in target 'SemanticVersion' from project 'SemanticVersion')
** BUILD FAILED **
finestructure commented 3 months ago

This ld workaround does in fact work:

RegisterExecutionPolicyException /Users/sas/Library/Developer/Xcode/DerivedData/SemanticVersion-fevvtnuqvvgsudbpylpukllorhdk/Build/Products/Debug/SemanticVersion.o (in target 'SemanticVersion' from project 'SemanticVersion')
    cd /Users/sas/Downloads/SemanticVersion
    builtin-RegisterExecutionPolicyException /Users/sas/Library/Developer/Xcode/DerivedData/SemanticVersion-fevvtnuqvvgsudbpylpukllorhdk/Build/Products/Debug/SemanticVersion.o

note: Using global toolchain override 'Swift Development Snapshot 2024-04-01 (a)'. (in target 'SemanticVersion' from project 'SemanticVersion')
** BUILD SUCCEEDED ** [14.008 sec]

I don't know why this failed earlier this morning (I tried multiple times) but clearly something was off. With a new shell it builds fine 🤷‍♂️

finestructure commented 3 months ago

Also, the defaults command does not seem to be needed, just symlinking in ld is enough. This would limit the workaround to the preview toolchain.

finestructure commented 2 months ago

We have one remaining issue even with the ld fix, and this is building for visionOS. This is failing in our builder test matrix.

[x] error: Driver threw frontend job retrieving target info failed with code 1: <unknown>:0: error: unsupported target OS: 'xros1.0'
 without emitting errors. (in target 'SemanticVersion' from project 'SemanticVersion')

Test command:

git clone https://github.com/SwiftPackageIndex/SemanticVersion \
  && cd SemanticVersion \
  && env DEVELOPER_DIR="/Applications/Xcode-15.3.0.app" \
  xcrun --toolchain $TOOLCHAIN_ID \
  xcodebuild build -scheme "SemanticVersion" -destination "generic/platform=xros"

Tested with toolchains org.swift.59202404011a and org.swift.600202404151a.

finestructure commented 2 months ago

I've also tried the visionOS build with Xcode 15.4b1 and visionOS 1.2 beta 2 (21O5565d) SDK + Simulator (Installed) and it fails:

env DEVELOPER_DIR="/Applications/Xcode-15.4.0-Beta.app" \
        xcrun --toolchain $TOOLCHAIN_ID \
        xcodebuild build -scheme "SemanticVersion" -destination "generic/platform=visionos"
warning: In-process target-info query failed (The operation couldn’t be completed. (SwiftDriver.DependencyScanningError error 9.)). Using fallback mechanism.
warning: In-process target-info query failed (The operation couldn’t be completed. (SwiftDriver.DependencyScanningError error 9.)). Using fallback mechanism. (in target 'SemanticVersion' from project 'SemanticVersion')
error: Driver threw frontend job retrieving target info failed with code 1: <unknown>:0: error: unsupported target OS: 'xros1.0'
 without emitting errors. (in target 'SemanticVersion' from project 'SemanticVersion')
finestructure commented 1 month ago

Still requires the ld link, even with Xcode 15.4

git clone https://github.com/SwiftPackageIndex/SemanticVersion \
        && cd SemanticVersion \
        && env DEVELOPER_DIR="/Applications/Xcode-15.4.0.app" \
        xcrun --toolchain $TOOLCHAIN_ID \
        xcodebuild build -scheme "SemanticVersion" -destination "generic/platform=macOS"
finestructure commented 1 month ago

As of toolchain org.swift.600202405261a visionOS is building again, with the 1.1 SDK.

finestructure commented 1 week ago

The toolchains might still be affected, but as of at least Xcode 16b2 the ld link is no longer required.

finestructure commented 1 week ago

Current toolchains still require the ld link.