PCH module cache path crash

[jpsim]

I'm seeing a swiftc compiler crash only when compiling my project with Bazel.

Here's some of the relevant output:

ERROR: /opt/homebrew/var/buildkite-agent/builds/SwiftLint-Buildkite-1/swiftlint/swiftlint/BUILD:26:13: Compiling Swift module //:swiftlint failed: (Exit 1): worker failed: error executing command bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/build_bazel_rules_swift/tools/worker/worker swiftc @bazel-out/darwin_arm64-fastbuild/bin/swiftlint.swiftmodule-0.params
error: compile command failed due to signal 11 (use -v to see invocation)
<unknown>:0: error: PCH was compiled with module cache path '/var/folders/l3/dr7rdmqj3jj8n48nf1957r800000gn/T/swift_module_cache.NaZm5G/1F8N9ED0WJXH9', but the path is currently '/var/folders/l3/dr7rdmqj3jj8n48nf1957r800000gn/T/swift_module_cache.NaZm5G/9LQMAOY29AJ2'
...
1.  Apple Swift version 5.6 (swiftlang-5.6.0.323.62 clang-1316.0.20.8)
2.  Compiling with the current language version
3.  While evaluating request ASTLoweringRequest(Lowering AST to SIL for file "Source/swiftlint/Commands/SwiftLint.swift")
4.  While silgen emitArtificialTopLevel SIL function "@async_Main".

Full output

``` ERROR: /opt/homebrew/var/buildkite-agent/builds/SwiftLint-Buildkite-1/swiftlint/swiftlint/BUILD:26:13: Compiling Swift module //:swiftlint failed: (Exit 1): worker failed: error executing command bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/build_bazel_rules_swift/tools/worker/worker swiftc @bazel-out/darwin_arm64-fastbuild/bin/swiftlint.swiftmodule-0.params error: compile command failed due to signal 11 (use -v to see invocation) :0: error: PCH was compiled with module cache path '/var/folders/l3/dr7rdmqj3jj8n48nf1957r800000gn/T/swift_module_cache.NaZm5G/1F8N9ED0WJXH9', but the path is currently '/var/folders/l3/dr7rdmqj3jj8n48nf1957r800000gn/T/swift_module_cache.NaZm5G/9LQMAOY29AJ2' Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace. Stack dump: 0. Program arguments: /Applications/Xcode-13.3.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file Source/swiftlint/Commands/Analyze.swift -primary-file Source/swiftlint/Commands/Docs.swift -primary-file Source/swiftlint/Commands/GenerateDocs.swift -primary-file Source/swiftlint/Commands/Lint.swift -primary-file Source/swiftlint/Commands/Rules.swift -primary-file Source/swiftlint/Commands/SwiftLint.swift -primary-file Source/swiftlint/Commands/Version.swift -primary-file Source/swiftlint/Extensions/Configuration+CommandLine.swift -primary-file Source/swiftlint/Extensions/Reporter+CommandLine.swift -primary-file Source/swiftlint/Helpers/Benchmark.swift -primary-file Source/swiftlint/Helpers/CompilerArgumentsExtractor.swift -primary-file Source/swiftlint/Helpers/LintOrAnalyzeArguments.swift -primary-file Source/swiftlint/Helpers/LintOrAnalyzeCommand.swift -primary-file Source/swiftlint/Helpers/LintableFilesVisitor.swift -primary-file Source/swiftlint/Helpers/Signposts.swift -primary-file Source/swiftlint/Helpers/SwiftLintError.swift -supplementary-output-file-map /var/folders/l3/dr7rdmqj3jj8n48nf1957r800000gn/T/TemporaryDirectory.BN4JXA/supplementaryOutputs-1 -target arm64-apple-macosx12.3 -Xllvm -aarch64-use-tbi -enable-objc-interop -stack-check -sdk /Applications/Xcode-13.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -I /Applications/Xcode-13.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -I bazel-out/darwin_arm64-fastbuild/bin/external/sourcekitten_com_github_drmohundro_SWXMLHash -I bazel-out/darwin_arm64-fastbuild/bin/external/sourcekitten_com_github_jpsim_yams -I bazel-out/darwin_arm64-fastbuild/bin/external/com_github_jpsim_sourcekitten -I bazel-out/darwin_arm64-fastbuild/bin/external/com_github_keith_swift_syntax -I bazel-out/darwin_arm64-fastbuild/bin -I bazel-out/darwin_arm64-fastbuild/bin/external/sourcekitten_com_github_apple_swift_argument_parser -I bazel-out/darwin_arm64-fastbuild/bin/external/com_github_johnsundell_collectionconcurrencykit -I bazel-out/darwin_arm64-fastbuild/bin/external/swiftlint_com_github_scottrhoyt_swifty_text_table -F /Applications/Xcode-13.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -enable-testing -gline-tables-only -module-cache-path /var/folders/l3/dr7rdmqj3jj8n48nf1957r800000gn/T/swift_module_cache.NaZm5G -Onone -D DEBUG -D SWIFT_PACKAGE -debug-prefix-map /private/var/tmp/_bazel_administrator/06abefa18b46b376d06898cac1735dbd/execroot/SwiftLint=. -new-driver-path /Applications/Xcode-13.3.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver -no-clang-module-breadcrumbs -serialize-debugging-options -color-diagnostics -resource-dir /Applications/Xcode-13.3.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -Xcc -iquoteexternal/sourcekitten_com_github_jpsim_yams -Xcc -iquotebazel-out/darwin_arm64-fastbuild/bin/external/sourcekitten_com_github_jpsim_yams -Xcc -iquoteexternal/com_github_jpsim_sourcekitten -Xcc -iquotebazel-out/darwin_arm64-fastbuild/bin/external/com_github_jpsim_sourcekitten -Xcc -iquote. -Xcc -iquotebazel-out/darwin_arm64-fastbuild/bin -Xcc -isystemexternal/sourcekitten_com_github_jpsim_yams/Sources/CYaml/include -Xcc -isystembazel-out/darwin_arm64-fastbuild/bin/external/sourcekitten_com_github_jpsim_yams/Sources/CYaml/include -Xcc -DSWIFT_PACKAGE -Xcc -fmodule-map-file=bazel-out/darwin_arm64-fastbuild/bin/external/sourcekitten_com_github_jpsim_yams/CYaml.swift.modulemap -Xcc -fmodule-map-file=external/com_github_jpsim_sourcekitten/Source/SourceKit/SourceKit.modulemap -Xcc -fmodule-map-file=external/com_github_jpsim_sourcekitten/Source/Clang_C/Clang_C.modulemap -Xcc -O0 -Xcc -DDEBUG=1 -module-name swiftlint -target-sdk-version 12.3 -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Commands/Analyze.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Commands/Docs.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Commands/GenerateDocs.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Commands/Lint.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Commands/Rules.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Commands/SwiftLint.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Commands/Version.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Extensions/Configuration+CommandLine.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Extensions/Reporter+CommandLine.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Helpers/Benchmark.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Helpers/CompilerArgumentsExtractor.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Helpers/LintOrAnalyzeArguments.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Helpers/LintOrAnalyzeCommand.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Helpers/LintableFilesVisitor.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Helpers/Signposts.swift.o -o bazel-out/darwin_arm64-fastbuild/bin/_swift_incremental/swiftlint_objs/Source/swiftlint/Helpers/SwiftLintError.swift.o 1. Apple Swift version 5.6 (swiftlang-5.6.0.323.62 clang-1316.0.20.8) 2. Compiling with the current language version 3. While evaluating request ASTLoweringRequest(Lowering AST to SIL for file "Source/swiftlint/Commands/SwiftLint.swift") 4. While silgen emitArtificialTopLevel SIL function "@async_Main". Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): 0 swift-frontend 0x0000000106e42f88 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56 1 swift-frontend 0x0000000106e41f9c llvm::sys::RunSignalHandlers() + 112 2 swift-frontend 0x0000000106e43618 SignalHandler(int) + 344 3 libsystem_platform.dylib 0x000000019e5dd4c4 _sigtramp + 56 4 swift-frontend 0x000000010289faa8 swift::SILFunctionBuilder::getOrCreateFunction(swift::SILLocation, swift::SILDeclRef, swift::ForDefinition_t, llvm::function_ref, swift::ProfileCounter) + 84 5 swift-frontend 0x000000010289faa8 swift::SILFunctionBuilder::getOrCreateFunction(swift::SILLocation, swift::SILDeclRef, swift::ForDefinition_t, llvm::function_ref, swift::ProfileCounter) + 84 6 swift-frontend 0x0000000102d1fa58 swift::Lowering::SILGenModule::getFunction(swift::SILDeclRef, swift::ForDefinition_t) + 380 7 swift-frontend 0x0000000102daf7a4 swift::Lowering::SILGenFunction::emitArtificialTopLevel(swift::Decl*) + 2252 8 swift-frontend 0x0000000102d20f30 swift::Lowering::SILGenModule::emitFunctionDefinition(swift::SILDeclRef, swift::SILFunction*) + 3412 9 swift-frontend 0x0000000102d276a4 swift::ASTLoweringRequest::evaluate(swift::Evaluator&, swift::ASTLoweringDescriptor) const + 4100 10 swift-frontend 0x0000000102df2e74 swift::SimpleRequest > (swift::ASTLoweringDescriptor), (swift::RequestFlags)9>::evaluateRequest(swift::ASTLoweringRequest const&, swift::Evaluator&) + 216 11 swift-frontend 0x0000000102d2a8c8 llvm::Expected swift::Evaluator::getResultUncached(swift::ASTLoweringRequest const&) + 608 12 swift-frontend 0x00000001026e47f8 swift::performFrontend(llvm::ArrayRef, char const*, void*, swift::FrontendObserver*) + 5272 13 swift-frontend 0x00000001026ac130 swift::mainEntry(int, char const**) + 808 14 dyld 0x0000000109565088 start + 516 ```

The fact that this is crashing when generating SIL for @async_Main seems pertinent. I haven't seen any similar compiler crashes when building with xcodebuild or SwiftPM on macOS or Linux. Which leads me to believe maybe there's something that's not being done correctly in rules_swift. In particular, maybe the -parse-as-library trick here doesn't play well with async @main? https://github.com/bazelbuild/rules_swift/blob/16cac51e7e5dd01832008177294f280febdf9f17/swift/internal/swift_library.bzl#L49-L73

Here's a link to the CI jobs where this happened, twice in a row on two different machines before passing again on one of them after retrying: https://buildkite.com/swiftlint/swiftlint/builds/2936#01829956-3d7c-4672-8a63-1c6bf31712fb

Here's the file it's compiling when the crash happens: https://github.com/realm/SwiftLint/blob/15970311e133feb166e2af1de3c2a876012e6575/Source/swiftlint/Commands/SwiftLint.swift

Environment:

• Xcode 13.3 (13E113)
• Swift 5.6
• Bazel 5.2.0
• rules_apple 1.0.1
• rules_swift 1.0.0

[keith]

This log is kind of interesting:

<unknown>:0: error: PCH was compiled with module cache path '/var/folders/l3/dr7rdmqj3jj8n48nf1957r800000gn/T/swift_module_cache.NaZm5G/1F8N9ED0WJXH9', but the path is currently '/var/folders/l3/dr7rdmqj3jj8n48nf1957r800000gn/T/swift_module_cache.NaZm5G/9LQMAOY29AJ2'

When I've seen something similar to this in the past the issue was something like artifacts pointing to 1 module cache path were being referenced from a build pointing to another, and that caused the failure. The CI machines appear to re-use artifacts across builds, but I would expect it would require a pretty big change, maybe an Xcode version bump?, without cleaning to cause something like this. 🤔

[jpsim]

The CI machines appear to re-use artifacts across builds

Correct. I cleaned both CI machines earlier today for a different change, so I'll report back if I see this again.

[luispadron]

Did ya'll ever figure out the issue here, I'm able to reproduce with a sort of simple setup. A swift_library in a swift_binary that fails when built with --complilation_module=dbg but not with --compilation_mode=opt

Error:

ERROR: Tools/ModuleWeights/BUILD.bazel:9:14: Compiling Swift module //Tools/ModuleWeights:ModuleWeights.library failed: (Exit 1): worker failed: error executing command (from target //Tools/ModuleWeights:ModuleWeights.library) 
  (cd /private/var/tmp/_bazel_lpadron/9ae9416857eb79bb978de35a53d54970/execroot/cash-ios && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=13.1 \
    SWIFT_AVOID_WARNING_USING_OLD_DRIVER=1 \
    XCODE_VERSION_OVERRIDE=14.2.0.14C18 \
  bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/build_bazel_rules_swift/tools/worker/worker swiftc @bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.swiftmodule-0.params)
# Configuration: 7f4d8ee0b96abffee3a3a00333e5bb49718726c089d95165cd6973076f1084c4
# Execution platform: @local_config_platform//:host
error: compile command failed due to signal 11 (use -v to see invocation)
<unknown>:0: error: PCH was compiled with module cache path '/private/var/tmp/_bazel_lpadron/9ae9416857eb79bb978de35a53d54970/execroot/cash-ios/bazel-out/darwin_arm64-dbg/bin/_swift_module_cache/NRNUEGJDXPX2', but the path is currently '/private/var/tmp/_bazel_lpadron/9ae9416857eb79bb978de35a53d54970/execroot/cash-ios/bazel-out/darwin_arm64-dbg/bin/_swift_module_cache/2CGXMPK9674O2'
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
Stack dump:
0.      Program arguments: /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file Tools/ModuleWeights/Sources/ModuleWeights/ModuleGraph+LinesOfCodeSummary.swift -primary-file Tools/ModuleWeights/Sources/ModuleWeights/ModuleLinesOfCodeSummary.swift -primary-file Tools/ModuleWeights/Sources/ModuleWeights/Tool.swift -target arm64-apple-macos12.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -stack-check -sdk /Applications/Xcode-14.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -I bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception -I bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_glob -I bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_yams -I bazel-out/darwin_arm64-dbg/bin/Tools/Frameworks/ModuleGraph -I bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_swift_argument_parser -enable-testing -g -module-cache-path bazel-out/darwin_arm64-dbg/bin/_swift_module_cache -static -Onone -D DEBUG -D SWIFT_PACKAGE -D SWIFT_PACKAGE -D YAML_DECLARE_STATIC -debug-prefix-map /Applications/Xcode-14.2.0.app/Contents/Developer=DEVELOPER_DIR -file-prefix-map /Applications/Xcode-14.2.0.app/Contents/Developer=DEVELOPER_DIR -debug-prefix-map /private/var/tmp/_bazel_lpadron/9ae9416857eb79bb978de35a53d54970/execroot/cash-ios=. -file-prefix-map /private/var/tmp/_bazel_lpadron/9ae9416857eb79bb978de35a53d54970/execroot/cash-ios=. -new-driver-path /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver -no-clang-module-breadcrumbs -no-serialize-debugging-options -color-diagnostics -enable-bare-slash-regex -empty-abi-descriptor -resource-dir /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -Xcc -Iexternal/swiftpkg_cwlcatchexception/Sources/CwlCatchExceptionSupport/include -Xcc -Ibazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception/Sources/CwlCatchExceptionSupport/include -Xcc -Ibazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception/CwlCatchExceptionSupport -Xcc -iquoteexternal/swiftpkg_cwlcatchexception -Xcc -iquotebazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception -Xcc -iquoteexternal/swiftpkg_yams -Xcc -iquotebazel-out/darwin_arm64-dbg/bin/external/swiftpkg_yams -Xcc -iquote. -Xcc -iquotebazel-out/darwin_arm64-dbg/bin -Xcc -isystemexternal/swiftpkg_yams/Sources/CYaml/include -Xcc -isystembazel-out/darwin_arm64-dbg/bin/external/swiftpkg_yams/Sources/CYaml/include -Xcc -DSWIFT_PACKAGE -Xcc -DSWIFT_PACKAGE=1 -Xcc -DYAML_DECLARE_STATIC -Xcc -fmodule-map-file=bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception/Sources_CwlCatchExceptionSupport.swift.modulemap -Xcc -fmodule-map-file=bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception/Sources_CwlCatchExceptionSupport_modulemap.swift.modulemap -Xcc -fmodule-map-file=bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_yams/Sources_CYaml.swift.modulemap -Xcc -O0 -Xcc -DDEBUG=1 -Xcc -fstack-protector -Xcc -fstack-protector-all -module-name ModuleWeights -disable-clang-spi -target-sdk-version 13.1 -parse-as-library -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/ModuleGraph+LinesOfCodeSummary.swift.o -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/ModuleLinesOfCodeSummary.swift.o -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/Tool.swift.o
1.      Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51)
2.      Compiling with the current language version
3.      While evaluating request ASTLoweringRequest(Lowering AST to SIL for file "Tools/ModuleWeights/Sources/ModuleWeights/Tool.swift")
4.      While silgen emitArtificialTopLevel SIL function "@async_Main".
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x0000000106cfb5b0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
1  swift-frontend           0x0000000106cfa5b4 llvm::sys::RunSignalHandlers() + 112
2  swift-frontend           0x0000000106cfbc34 SignalHandler(int) + 344
3  libsystem_platform.dylib 0x0000000195b5aa24 _sigtramp + 56
4  swift-frontend           0x00000001024048b0 swift::SILFunctionBuilder::getOrCreateFunction(swift::SILLocation, swift::SILDeclRef, swift::ForDefinition_t, llvm::function_ref<swift::SILFunction* (swift::SILLocation, swift::SILDeclRef)>, swift::ProfileCounter) + 84
5  swift-frontend           0x00000001024048b0 swift::SILFunctionBuilder::getOrCreateFunction(swift::SILLocation, swift::SILDeclRef, swift::ForDefinition_t, llvm::function_ref<swift::SILFunction* (swift::SILLocation, swift::SILDeclRef)>, swift::ProfileCounter) + 84
6  swift-frontend           0x00000001028fbfa8 swift::Lowering::SILGenModule::getFunction(swift::SILDeclRef, swift::ForDefinition_t) + 384
7  swift-frontend           0x000000010298e3cc swift::Lowering::SILGenFunction::emitArtificialTopLevel(swift::Decl*) + 1844
8  swift-frontend           0x00000001028fd854 swift::Lowering::SILGenModule::emitFunctionDefinition(swift::SILDeclRef, swift::SILFunction*) + 4216
9  swift-frontend           0x000000010290419c swift::ASTLoweringRequest::evaluate(swift::Evaluator&, swift::ASTLoweringDescriptor) const + 4052
10 swift-frontend           0x00000001029d0f10 swift::SimpleRequest<swift::ASTLoweringRequest, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> > (swift::ASTLoweringDescriptor), (swift::RequestFlags)9>::evaluateRequest(swift::ASTLoweringRequest const&, swift::Evaluator&) + 216
11 swift-frontend           0x00000001029076e4 llvm::Expected<swift::ASTLoweringRequest::OutputType> swift::Evaluator::getResultUncached<swift::ASTLoweringRequest>(swift::ASTLoweringRequest const&) + 628
12 swift-frontend           0x0000000102348758 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 5860
13 swift-frontend           0x00000001022e9294 swift::mainEntry(int, char const**) + 3940
14 dyld                     0x00000001957d3f28 start + 2236

The package is pretty simple:

swift_binary(
    name = "ModuleWeights",
    visibility = ["//visibility:public"],
    deps = [":ModuleWeights.library"],
)

swift_library(
    name = "ModuleWeights.library",
    srcs = glob(["Sources/ModuleWeights/**/*.swift"]),
    module_name = "ModuleWeights",
    tags = ["manual"],
    deps = [
        "//Tools/Frameworks/ModuleGraph",
        "//Tools/Frameworks/ModuleGraph:ModuleGraph_LinesOfCode",
        "@swiftpkg_cwlcatchexception//:Sources_CwlCatchException",
        "@swiftpkg_swift_argument_parser//:Sources_ArgumentParser",
    ],
)

And the code which crashes is simply:

@main
struct ModuleWeights: AsyncParsableCommand {
   ...
}

[keith]

if it is the exact same case I never got around to finishing the fix https://github.com/apple/swift/pull/60636

[luispadron]

Thanks @keith! Did ya'll end up working around this somehow?

[keith]

If you use async behavior anywhere else in the code I think you sidestep it

[luispadron]

Do you know where exactly the other async code goes? This tool does import other async code and uses await in the same module.

I wonder if we can make this match whatever SPM is doing, this seems to work correctly there. I haven't been able to reproduce in a rules_swift example but this reproduces in our codebase the minute this @main target depends on another async capable swift_library

[luispadron]

Was able to track this down further, to specifically only happening when depending on Yams package. I was doing this via rules_swift_package_manager which would cause a crash, consuming the BUILD file for Yams directly via WORKSPACE and depending on that seems to resolve the crash.

Will see if i can repro in rules_swift_package_manager and open an issue there

Edit: Never mind, that seems to have fixed it for the build locally, CI still failed for us. I also cleared all my caches and re-built locally and see the same failure now

[keith]

It's a race condition so I'm not surprised it's in and out. I can't recall if I could repro with swiftpm or not, you'd have to pass the flag mentioned in my fix PR to do so, which swiftpm does not by default.

[luispadron]

Heres a swift-frontend call using SPM for the same code/deps:

/Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/Sources/ModuleWeights/ModuleGraph+LinesOfCodeSummary.swift /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/Sources/ModuleWeights/ModuleLinesOfCodeSummary.swift /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/Sources/ModuleWeights/Tool.swift -emit-dependencies-path /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/arm64-apple-macosx/debug/ModuleWeights.build/ModuleGraph+LinesOfCodeSummary.d -emit-reference-dependencies-path /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/arm64-apple-macosx/debug/ModuleWeights.build/ModuleGraph+LinesOfCodeSummary.swiftdeps -target arm64-apple-macosx12.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -stack-check -sdk /Applications/Xcode-14.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -I /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/arm64-apple-macosx/debug -I /Applications/Xcode-14.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -F /Applications/Xcode-14.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -color-diagnostics -enable-testing -g -module-cache-path /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/arm64-apple-macosx/debug/ModuleCache -swift-version 5 -Onone -D SWIFT_PACKAGE -D DEBUG -new-driver-path /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver -entry-point-function-name ModuleWeights_main -empty-abi-descriptor -resource-dir /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -Xcc -fmodule-map-file=/Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/checkouts/Yams/Sources/CYaml/include/module.modulemap -Xcc -I -Xcc /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/checkouts/Yams/Sources/CYaml/include -Xcc -fmodule-map-file=/Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/arm64-apple-macosx/debug/CwlCatchExceptionSupport.build/module.modulemap -Xcc -I -Xcc /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/checkouts/CwlCatchException/Sources/CwlCatchExceptionSupport/include -module-name ModuleWeights -disable-clang-spi -target-sdk-version 13.1 -o /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/arm64-apple-macosx/debug/ModuleWeights.build/ModuleGraph+LinesOfCodeSummary.swift.o -index-store-path /Users/lpadron/Development/cash-ios/Tools/ModuleWeights/.build/arm64-apple-macosx/debug/index/store -index-system-modules

And the same Bazel command:

 /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -c -primary-file Tools/ModuleWeights/Sources/ModuleWeights/ModuleGraph+LinesOfCodeSummary.swift -primary-file Tools/ModuleWeights/Sources/ModuleWeights/ModuleLinesOfCodeSummary.swift -primary-file Tools/ModuleWeights/Sources/ModuleWeights/Tool.swift -target arm64-apple-macos12.0 -Xllvm -aarch64-use-tbi -enable-objc-interop -stack-check -sdk /Applications/Xcode-14.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -I bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception -I bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_glob -I bazel-out/darwin_arm64-dbg/bin/external/yams -I bazel-out/darwin_arm64-dbg/bin/Tools/Frameworks/ModuleGraph -I bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_swift_argument_parser -enable-testing -g -module-cache-path bazel-out/darwin_arm64-dbg/bin/_swift_module_cache -static -Onone -D DEBUG -D SWIFT_PACKAGE -D SWIFT_PACKAGE -debug-prefix-map /Applications/Xcode-14.2.0.app/Contents/Developer=DEVELOPER_DIR -file-prefix-map /Applications/Xcode-14.2.0.app/Contents/Developer=DEVELOPER_DIR -debug-prefix-map /private/var/tmp/_bazel_lpadron/9ae9416857eb79bb978de35a53d54970/execroot/cash-ios=. -file-prefix-map /private/var/tmp/_bazel_lpadron/9ae9416857eb79bb978de35a53d54970/execroot/cash-ios=. -new-driver-path /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver -no-clang-module-breadcrumbs -no-serialize-debugging-options -color-diagnostics -enable-bare-slash-regex -empty-abi-descriptor -resource-dir /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -Xcc -Iexternal/swiftpkg_cwlcatchexception/Sources/CwlCatchExceptionSupport/include -Xcc -Ibazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception/Sources/CwlCatchExceptionSupport/include -Xcc -Ibazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception/CwlCatchExceptionSupport -Xcc -iquoteexternal/swiftpkg_cwlcatchexception -Xcc -iquotebazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception -Xcc -iquoteexternal/yams -Xcc -iquotebazel-out/darwin_arm64-dbg/bin/external/yams -Xcc -iquote. -Xcc -iquotebazel-out/darwin_arm64-dbg/bin -Xcc -isystemexternal/yams/Sources/CYaml/include -Xcc -isystembazel-out/darwin_arm64-dbg/bin/external/yams/Sources/CYaml/include -Xcc -DSWIFT_PACKAGE -Xcc -DSWIFT_PACKAGE=1 -Xcc -fmodule-map-file=bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception/Sources_CwlCatchExceptionSupport.swift.modulemap -Xcc -fmodule-map-file=bazel-out/darwin_arm64-dbg/bin/external/swiftpkg_cwlcatchexception/Sources_CwlCatchExceptionSupport_modulemap.swift.modulemap -Xcc -fmodule-map-file=bazel-out/darwin_arm64-dbg/bin/external/yams/CYaml.swift.modulemap -Xcc -O0 -Xcc -DDEBUG=1 -Xcc -fstack-protector -Xcc -fstack-protector-all -module-name ModuleWeights -disable-clang-spi -target-sdk-version 13.1 -parse-as-library -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/ModuleGraph+LinesOfCodeSummary.swift.o -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/ModuleLinesOfCodeSummary.swift.o -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/Tool.swift.o

To @jpsim comment about -parse-as-library I am seeing Bazel add this while SPM does not:

-parse-as-library -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/ModuleGraph+LinesOfCodeSummary.swift.o -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/ModuleLinesOfCodeSummary.swift.o -o bazel-out/darwin_arm64-dbg/bin/Tools/ModuleWeights/ModuleWeights.library_objs/Sources/ModuleWeights/Tool.swift.o

[luispadron]

If I remove the -no-clang-module-breadcrumbs argument here: https://github.com/bazelbuild/rules_swift/blob/dba1542ff53bc1ade1ad24a38f1bc9c0287e4039/swift/internal/compiling.bzl#L290-L293

After a clean and re-build the target compiles successfully.

[luispadron]

That would also explain why I dont see this crash in release builds

[luispadron]

Disabling cahcheable_swiftmodules features seems to be working locally, testing in CI now:

features = ["-swift.cacheable_swiftmodules"]

[keith]

hrm im not sure why flipping cacheable_swiftmodules would help, maybe this is a different bug

[luispadron]

From your issue in Swift repo it looks like compiler doesnt respect: -no-clang-module-breadcrumbs. rules_swift seems to add this flag when swift.cacheable_swiftmodules is enabled. I'm not sure what the real solution is here tbf but it doesn't seem like SPM passes that flag by default

Edit: Maybe just Xcode 14 bug from this quote:

It looks like the Xcode 14 toolchain is producing clang modules by default for implicit modules now, and they still contain breadcrumbs even when passing -no-clang-module-breadcrumbs:

[keith]

Yea that flag is required to make hermetic swiftmodules since otherwise they contain absolute paths to debug info. So swiftpm doesn't care about that.