search

swiftlang / swift-sdk-generator

Generate Swift SDKs for cross-compilation
Apache License 2.0
182 stars 15 forks source link

Error: noCacheKeyConformance(Swift.Array<SystemPackage.FilePath.Component>) #106

Closed yingguqing closed 2 days ago

yingguqing commented 5 months ago

Hi! I am trying to cross compile for ubuntu x86_64 and arm64 architectures. I followed the steps on the readme and cloned the repo and ran swift run swift-sdk-generator make-linux-sdk.

However, I am seeing this output:

Parsing Ubuntu packages list...
Downloading 11 Ubuntu packages...
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/gcc-12/libgcc-s1_12.3.0-1ubuntu1~22.04_amd64.deb – 54 KB
http://gb.archive.ubuntu.com/ubuntu/pool/main/z/zlib/zlib1g_1.2.11.dfsg-2ubuntu9.2_amd64.deb – 58 KB
http://gb.archive.ubuntu.com/ubuntu/pool/main/z/zlib/zlib1g-dev_1.2.11.dfsg-2ubuntu9.2_amd64.deb – 164 KB
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/gcc-12/libstdc++6_12.3.0-1ubuntu1~22.04_amd64.deb – 699 KB
http://gb.archive.ubuntu.com/ubuntu/pool/main/l/linux/linux-libc-dev_5.15.0-112.122_amd64.deb – 1.3 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/glibc/libc6-dev_2.35-0ubuntu3.8_amd64.deb – 2.1 MB
http://gb.archive.ubuntu.com/ubuntu/pool/universe/g/gcc-12/libstdc++-12-dev_12.3.0-1ubuntu1~22.04_amd64.deb – 2.2 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/gcc-12/libgcc-12-dev_12.3.0-1ubuntu1~22.04_amd64.deb – 2.6 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/glibc/libc6_2.35-0ubuntu3.8_amd64.deb – 3.2 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/i/icu/libicu70_70.1-2_amd64.deb – 10.6 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/i/icu/libicu-dev_70.1-2_amd64.deb – 11.6 MB
control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

control.tar.zst
data.tar.zst
debian-binary

Unpacking and copying Swift binaries for the host triple...
8460552 blocks

Unpacking Swift distribution for the target triple...

Copying Swift core libraries for the target triple into Swift SDK bundle...

Unpacking and copying `lld` linker...
Error: noCacheKeyConformance(Swift.Array<SystemPackage.FilePath.Component>)

Edit: And running again with verbose output:

$ swift run swift-sdk-generator  make-linux-sdk -v

Building for debugging...
Build complete! (0.54s)

Looking up configuration values...

Downloading required toolchain packages...
Using downloaded artifacts in these locations:
/Users/xxxx/Desktop/SDK/swift-sdk-generator/Artifacts/llvm_17.0.5.src.tar.xz
/Users/xxxx/Desktop/SDK/swift-sdk-generator/Artifacts/target_swift_5.9.2-RELEASE_x86_64-unknown-linux-gnu.tar.gz
/Users/xxxx/Desktop/SDK/swift-sdk-generator/Artifacts/host_swift_5.9.2-RELEASE_x86_64-apple-macos13.0.0.pkg

Parsing Ubuntu packages list...
Downloading 11 Ubuntu packages...
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/gcc-12/libgcc-s1_12.3.0-1ubuntu1~22.04_amd64.deb – 54 KB
http://gb.archive.ubuntu.com/ubuntu/pool/main/z/zlib/zlib1g_1.2.11.dfsg-2ubuntu9.2_amd64.deb – 58 KB
http://gb.archive.ubuntu.com/ubuntu/pool/main/z/zlib/zlib1g-dev_1.2.11.dfsg-2ubuntu9.2_amd64.deb – 164 KB
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/gcc-12/libstdc++6_12.3.0-1ubuntu1~22.04_amd64.deb – 699 KB
http://gb.archive.ubuntu.com/ubuntu/pool/main/l/linux/linux-libc-dev_5.15.0-112.122_amd64.deb – 1.3 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/glibc/libc6-dev_2.35-0ubuntu3.8_amd64.deb – 2.1 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/gcc-12/libgcc-12-dev_12.3.0-1ubuntu1~22.04_amd64.deb – 2.6 MB
http://gb.archive.ubuntu.com/ubuntu/pool/universe/g/gcc-12/libstdc++-12-dev_12.3.0-1ubuntu1~22.04_amd64.deb – 2.2 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/g/glibc/libc6_2.35-0ubuntu3.8_amd64.deb – 3.2 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/i/icu/libicu-dev_70.1-2_amd64.deb – 11.6 MB
http://gb.archive.ubuntu.com/ubuntu/pool/main/i/icu/libicu70_70.1-2_amd64.deb – 10.6 MB
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-67D23B" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/libc6_2.35-0ubuntu3.8_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-67D23B"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-105915" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/libc6-dev_2.35-0ubuntu3.8_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-105915"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-EDFA74" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/libgcc-s1_12.3.0-1ubuntu1~22.04_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-EDFA74"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-0DA5F7" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/libgcc-12-dev_12.3.0-1ubuntu1~22.04_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-0DA5F7"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-C2DBDF" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/libicu70_70.1-2_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-C2DBDF"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-A8D1F1" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/libicu-dev_70.1-2_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-A8D1F1"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-DE9916" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/libstdc++-12-dev_12.3.0-1ubuntu1~22.04_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-DE9916"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-AD90FD" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/libstdc++6_12.3.0-1ubuntu1~22.04_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-AD90FD"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-8932D1" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/linux-libc-dev_5.15.0-112.122_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-8932D1"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-FC077C" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/zlib1g_1.2.11.dfsg-2ubuntu9.2_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-FC077C"/data.tar.*
cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-24190E" && ar -x "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-1F2A30/zlib1g-dev_1.2.11.dfsg-2ubuntu9.2_amd64.deb"
control.tar.zst
data.tar.zst
debian-binary

tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk" -xf "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-24190E"/data.tar.*

Unpacking and copying Swift binaries for the host triple...
xar -C "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-78138A" -xf "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Artifacts/host_swift_5.9.2-RELEASE_x86_64-apple-macos13.0.0.pkg"
cat "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-78138A"/*.pkg/Payload | gunzip -cd | (cd "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-593CD9" && cpio -i)
8460552 blocks
rsync -a /var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-593CD9/usr /Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/swift.xctoolchain

Unpacking Swift distribution for the target triple...
tar -C "/var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-971F80"  -xzf /Users/xxxx/Desktop/SDK/swift-sdk-generator/Artifacts/target_swift_5.9.2-RELEASE_x86_64-unknown-linux-gnu.tar.gz

Copying Swift core libraries for the target triple into Swift SDK bundle...
rsync -a /var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-971F80/swift-5.9.2-RELEASE-ubuntu22.04/usr/lib/swift/linux /Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/swift.xctoolchain/usr/lib/swift
rsync -a /var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-971F80/swift-5.9.2-RELEASE-ubuntu22.04/usr/lib/swift_static/linux /Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/swift.xctoolchain/usr/lib/swift_static
rsync -a /var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-971F80/swift-5.9.2-RELEASE-ubuntu22.04/usr/lib/swift_static/shims /Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/swift.xctoolchain/usr/lib/swift_static
rsync -a /var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-971F80/swift-5.9.2-RELEASE-ubuntu22.04/usr/lib/swift/dispatch /Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk/usr/include
rsync -a /var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-971F80/swift-5.9.2-RELEASE-ubuntu22.04/usr/lib/swift/os /Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk/usr/include
rsync -a /var/folders/5k/88pkjzgs3rj4g0jky1c7j6sw0000gn/T/swift-sdk-generator-971F80/swift-5.9.2-RELEASE-ubuntu22.04/usr/lib/swift/CoreFoundation /Users/xxxx/Desktop/SDK/swift-sdk-generator/Bundles/5.9.2-RELEASE_ubuntu_jammy_x86_64.artifactbundle/5.9.2-RELEASE_ubuntu_jammy_x86_64/x86_64-unknown-linux-gnu/ubuntu-jammy.sdk/usr/include

Unpacking and copying `lld` linker...
tar -C "/Users/xxxx/Desktop/SDK/swift-sdk-generator/Artifacts/llvm_17.0.5.src" --strip-components=1 -xzf /Users/xxxx/Desktop/SDK/swift-sdk-generator/Artifacts/llvm_17.0.5.src.tar.xz
Error: noCacheKeyConformance(Swift.Array<SystemPackage.FilePath.Component>)

Curious if this is an issue or if I am doing something wrong. I am running this on an M1 Pro Mac on macOS Sonoma with swift 5.9. Output of uname -a: Darwin <local host>.local 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:52 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T8103 x86_64 Output of swift --version:

swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1)
Target: x86_64-apple-macosx13.0

SHA of swift-sdk-generator: db85e25 Thanks!

yingguqing commented 5 months ago

Add code to end of file Sources/GeneratorEngine/Cache/CacheKeyProtocol.swift

extension Array: CacheKey where Element == FilePath.Component {
    func hash(with hashFunction: inout some HashFunction) {
        String(reflecting: Self.self).hash(with: &hashFunction)
        map(\.string).joined(separator: "\n").hash(with: &hashFunction)
    }
}

extension Range: CacheKey where Bound == Int {
    func hash(with hashFunction: inout some HashFunction) {
        String(reflecting: Self.self).hash(with: &hashFunction)
        description.hash(with: &hashFunction)
    }
}
MaxDesiatov commented 5 months ago

Great, thank you for investigating this. Now for the PR we need test coverage before it can be considered for merging.

euanh commented 2 days ago

Fixed in #149