search

swiftwasm / carton

📦 Watcher, bundler, and test runner for your SwiftWasm apps
Apache License 2.0
447 stars 45 forks source link

Invalid manifest on 'warmtransformer' building a carton project #496

Open k0pernicus opened 6 days ago

k0pernicus commented 6 days ago

I have tried to build & run a default project using carton, and it failed at parsing (?) the manifest on 'warmtransformer' package.

Reproduction

> mkdir hello_swiftwasm && cd hello_swiftwasm
> swift package init --type executable --name hello_swiftwasm
Creating executable package: hello_swiftwasm
Creating Package.swift
Creating .gitignore
Creating Sources/
Creating Sources/main.swift
> swift --version
Apple Swift version 6.0.3-dev (LLVM 81a2745a5c40b5f, Swift 8f95e04bb5d9d9a)
Target: arm64-apple-macosx14.0
> swift package add-dependency https://github.com/swiftwasm/carton --from 1.0.0

The error

> swift run carton dev
Building for debugging...
Build of product 'carton' complete! (0.13s)
- checking Swift compiler path: /Users/antonin/.carton/sdk/wasm-6.0-SNAPSHOT-2024-06-19-a/usr/bin/swift
- checking Swift compiler path: /Users/antonin/.swiftenv/versions/wasm-6.0-SNAPSHOT-2024-06-19-a/usr/bin/swift
- checking Swift compiler path: /Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swift
Inferring basic settings...
- swift executable: /Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swift
Apple Swift version 6.0-dev (LLVM de395d39a90ed7a, Swift 490cf64aee23f13)
Target: arm64-apple-macosx14.0
Running "/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swift" "package" "--triple" "wasm32-unknown-wasi" "--scratch-path" "/Users/antonin/Devel/hello_swiftwasm/.build/carton" "--disable-sandbox" "plugin" "carton-dev" "--pid" "42095"
error: 'wasmtransformer': Invalid manifest (compiled with: ["/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swiftc", "-vfsoverlay", "/var/folders/vf/qs0d0q2j4d5cqld9x11d1m_h0000gn/T/TemporaryDirectory.vRySPF/vfs.yaml", "-L", "/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-target", "arm64-apple-macosx13.0", "-sdk", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk", "-F", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks", "-I", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib", "-L", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib", "-swift-version", "5", "-I", "/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-sdk", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk", "-package-description-version", "5.4.0", "/Users/antonin/Devel/hello_swiftwasm/.build/carton/checkouts/WasmTransformer/Package.swift", "-o", "/var/folders/vf/qs0d0q2j4d5cqld9x11d1m_h0000gn/T/TemporaryDirectory.XQHTEP/wasmtransformer-manifest"])
error: compile command failed due to signal 6 (use -v to see invocation)
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/string.h:58:10: note: while building module '_string' imported from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/string.h:58:
56 |  */
57 |
58 | #include <_string.h>
   |          `- note: while building module '_string' imported from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/string.h:58:
59 |
60 | #ifdef _USE_EXTENDED_LOCALES_
...

I did not know if the best would be to submit here, or in wasmtransformer project.

kateinoigakukun commented 6 days ago

What happens if you use recent 6.0 snapshots?

k0pernicus commented 6 days ago

I just found out that carton is checking / using an old version of a snapshot (06-19) instead of my latest (10-08). Despite forcing the path & removing the old toolchain it seems that the build continues to look for the old toolchain...

...
- checking Swift compiler path: /Users/antonin/.carton/sdk/wasm-6.0-SNAPSHOT-2024-06-19-a/usr/bin/swift
- checking Swift compiler path: /Users/antonin/.swiftenv/versions/wasm-6.0-SNAPSHOT-2024-06-19-a/usr/bin/swift
- checking Swift compiler path: /Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swift
...

My toolchains:

> ls /Library/Developer/Toolchains
swift-6.0-DEVELOPMENT-SNAPSHOT-2024-10-08-a.xctoolchain swift-latest.xctoolchain

Is there an internal cache of something? I deleted Package.resolved, the .build directory and the .carton directory too, but nothing changed.

kateinoigakukun commented 6 days ago

You can change the toolchain version by writing 6.0-DEVELOPMENT-SNAPSHOT-2024-10-08-a in .swift-version file. e.g. https://www.pointfree.co/episodes/ep291-cross-platform-swift-webassembly#:~:text=We%20just%20need%20to%20create%20a%20new%20file%20at%20the%20root%20of%20the%20project%20called%20%E2%80%9C.swift%2Dversion%E2%80%9D%2C%20and%20put%20the%20identifier%20of%20the%20Swift%20snapshot%20we%20want%20to%20use.%20The%20most%20current%20one%20at%20the%20time%20of%20recording%20this%20episode%20is%3A

k0pernicus commented 6 days ago

Same thing actually:

> ls /Library/Developer/Toolchains
swift-6.0-DEVELOPMENT-SNAPSHOT-2024-10-08-a.xctoolchain swift-latest.xctoolchain

> cat .swift-version
6.0-DEVELOPMENT-SNAPSHOT-2024-10-08-a

> swift --version
Apple Swift version 6.0.3-dev (LLVM 81a2745a5c40b5f, Swift 8f95e04bb5d9d9a)
Target: arm64-apple-macosx14.0

> swift run carton dev
Fetching https://github.com/swiftwasm/carton from cache
Fetched https://github.com/swiftwasm/carton from cache (0.43s)
Computing version for https://github.com/swiftwasm/carton
Computed https://github.com/swiftwasm/carton at 1.1.2 (0.65s)
Fetching https://github.com/apple/swift-argument-parser.git from cache
Fetching https://github.com/apple/swift-log.git from cache
Fetching https://github.com/swiftwasm/WasmTransformer from cache
Fetching https://github.com/apple/swift-nio.git from cache
Fetched https://github.com/swiftwasm/WasmTransformer from cache (0.48s)
Fetched https://github.com/apple/swift-argument-parser.git from cache (0.48s)
Fetched https://github.com/apple/swift-log.git from cache (0.49s)
Fetched https://github.com/apple/swift-nio.git from cache (0.50s)
Computed https://github.com/swiftwasm/carton at 1.1.2 (0.00s)
Computing version for https://github.com/swiftwasm/WasmTransformer
Computed https://github.com/swiftwasm/WasmTransformer at 0.5.0 (0.54s)
Computing version for https://github.com/apple/swift-argument-parser.git
Computed https://github.com/apple/swift-argument-parser.git at 1.3.1 (0.41s)
Computing version for https://github.com/apple/swift-log.git
Computed https://github.com/apple/swift-log.git at 1.6.1 (0.37s)
Computing version for https://github.com/apple/swift-nio.git
Computed https://github.com/apple/swift-nio.git at 2.75.0 (0.58s)
Fetching https://github.com/apple/swift-system.git from cache
Fetching https://github.com/apple/swift-collections.git from cache
Fetching https://github.com/apple/swift-atomics.git from cache
Fetched https://github.com/apple/swift-atomics.git from cache (0.41s)
Fetched https://github.com/apple/swift-collections.git from cache (0.42s)
Fetched https://github.com/apple/swift-system.git from cache (0.44s)
Computing version for https://github.com/apple/swift-atomics.git
Computed https://github.com/apple/swift-atomics.git at 1.2.0 (0.39s)
Computing version for https://github.com/apple/swift-system.git
Computed https://github.com/apple/swift-system.git at 1.3.2 (0.37s)
Computing version for https://github.com/apple/swift-collections.git
Computed https://github.com/apple/swift-collections.git at 1.1.4 (0.59s)
Creating working copy for https://github.com/apple/swift-argument-parser.git
Working copy of https://github.com/apple/swift-argument-parser.git resolved at 1.3.1
Creating working copy for https://github.com/apple/swift-nio.git
Working copy of https://github.com/apple/swift-nio.git resolved at 2.75.0
Creating working copy for https://github.com/apple/swift-collections.git
Working copy of https://github.com/apple/swift-collections.git resolved at 1.1.4
Creating working copy for https://github.com/swiftwasm/carton
Working copy of https://github.com/swiftwasm/carton resolved at 1.1.2
Creating working copy for https://github.com/swiftwasm/WasmTransformer
Working copy of https://github.com/swiftwasm/WasmTransformer resolved at 0.5.0
Creating working copy for https://github.com/apple/swift-atomics.git
Working copy of https://github.com/apple/swift-atomics.git resolved at 1.2.0
Creating working copy for https://github.com/apple/swift-log.git
Working copy of https://github.com/apple/swift-log.git resolved at 1.6.1
Creating working copy for https://github.com/apple/swift-system.git
Working copy of https://github.com/apple/swift-system.git resolved at 1.3.2
Building for debugging...
[58/58] Applying carton
Build of product 'carton' complete! (15.06s)
- checking Swift compiler path: /Users/antonin/.carton/sdk/wasm-6.0-SNAPSHOT-2024-06-19-a/usr/bin/swift
- checking Swift compiler path: /Users/antonin/.swiftenv/versions/wasm-6.0-SNAPSHOT-2024-06-19-a/usr/bin/swift
- checking Swift compiler path: /Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swift
Inferring basic settings...
- swift executable: /Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swift
Apple Swift version 6.0-dev (LLVM de395d39a90ed7a, Swift 490cf64aee23f13)
Target: arm64-apple-macosx14.0
Running "/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swift" "package" "--triple" "wasm32-unknown-wasi" "--scratch-path" "/Users/antonin/Devel/hello_swiftwasm/.build/carton" "--disable-sandbox" "plugin" "carton-dev" "--pid" "58666"
Fetching https://github.com/apple/swift-log.git from cache
Fetching https://github.com/apple/swift-collections.git from cache
Fetching https://github.com/apple/swift-argument-parser.git from cache
Fetching https://github.com/swiftwasm/WasmTransformer from cache
Fetching https://github.com/apple/swift-system.git from cache
Fetching https://github.com/apple/swift-atomics.git from cache
Fetched https://github.com/apple/swift-atomics.git from cache (0.02s)
Fetching https://github.com/swiftwasm/carton from cache
Fetched https://github.com/swiftwasm/WasmTransformer from cache (0.02s)
Fetching https://github.com/apple/swift-nio.git from cache
Fetched https://github.com/apple/swift-argument-parser.git from cache (0.03s)
Fetched https://github.com/apple/swift-collections.git from cache (0.03s)
Fetched https://github.com/apple/swift-log.git from cache (0.03s)
Fetched https://github.com/swiftwasm/carton from cache (0.02s)
Fetched https://github.com/apple/swift-nio.git from cache (0.03s)
Fetched https://github.com/apple/swift-system.git from cache (0.06s)
Creating working copy for https://github.com/apple/swift-collections.git
Working copy of https://github.com/apple/swift-collections.git resolved at 1.1.4
Creating working copy for https://github.com/swiftwasm/WasmTransformer
Working copy of https://github.com/swiftwasm/WasmTransformer resolved at 0.5.0
Creating working copy for https://github.com/apple/swift-argument-parser.git
Working copy of https://github.com/apple/swift-argument-parser.git resolved at 1.3.1
Creating working copy for https://github.com/apple/swift-system.git
Working copy of https://github.com/apple/swift-system.git resolved at 1.3.2
Creating working copy for https://github.com/apple/swift-log.git
Working copy of https://github.com/apple/swift-log.git resolved at 1.6.1
Creating working copy for https://github.com/apple/swift-atomics.git
Working copy of https://github.com/apple/swift-atomics.git resolved at 1.2.0
Creating working copy for https://github.com/swiftwasm/carton
Working copy of https://github.com/swiftwasm/carton resolved at 1.1.2
Creating working copy for https://github.com/apple/swift-nio.git
Working copy of https://github.com/apple/swift-nio.git resolved at 2.75.0
error: 'wasmtransformer': Invalid manifest (compiled with: ["/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/bin/swiftc", "-vfsoverlay", "/var/folders/vf/qs0d0q2j4d5cqld9x11d1m_h0000gn/T/TemporaryDirectory.cvIlsH/vfs.yaml", "-L", "/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-target", "arm64-apple-macosx13.0", "-sdk", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk", "-F", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks", "-I", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib", "-L", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib", "-swift-version", "5", "-I", "/Users/antonin/Library/Developer/Toolchains/swift-wasm-6.0-SNAPSHOT-2024-06-19-a.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-sdk", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk", "-package-description-version", "5.4.0", "/Users/antonin/Devel/hello_swiftwasm/.build/carton/checkouts/WasmTransformer/Package.swift", "-o", "/var/folders/vf/qs0d0q2j4d5cqld9x11d1m_h0000gn/T/TemporaryDirectory.d2ONtz/wasmtransformer-manifest"])

However, when I take a look at your example, it seems to use this toolchain too:

> Local installation of Swift version wasm-6.0-SNAPSHOT-2024-06-19-a not found
Swift toolchain/SDK download URL: https://github.com/swiftwasm/swift/releases/download/swift-wasm-6.0-SNAPSHOT-2024-06-19-a/swift-wasm-6.0-SNAPSHOT-2024-06-19-a-macos_arm64.pkg
Archive size is 1468 MB
...

Did the carton build "packager" needs to download something? Is it "hardcoded" to use the 06-19 snapshot or something?

k0pernicus commented 6 days ago

Ooooh ok, I did not noticed that the tool was checking for Users/user/Library/Developer/Toolchains, and not /Library/Developer/Toolchains which contains all my latest toolchains...

k0pernicus commented 6 days ago

So, if I delete the 06-19 sdk from Users/user/Library/Developer/Toolchains, and put the .swift-version in my project with the 10-08 snapshot, it will try to download again the 06-19 snapshot again.

I am really confused about why it tries to download this snapshot particularly. Is it coming from my current setup?