Memory corruption in type checking from async entrypoint/dispatch

[koliyo]

In my Hylo-LSP prototype I have an async main entrypoint and I noticed that I am now getting memory corruption (SIGBUS) when running the type checker in hylo.

I managed to make a minimal application that demonstrates the problem. It is 100% reproducible on my machine. Here is the code:

https://github.com/hylo-lang/hylo/commit/c60d2dd2e8c8f4f74c9fdc68820e9554f7634267

Specifically the main entrypoint, which looks like this:

@main
struct MyApp {

    static func buildProgram() async throws {
      var diagnostics: DiagnosticSet = DiagnosticSet()
      let ast = try Host.hostedLibraryAST.get()
      let _ = try TypedProgram(
        annotating: ScopedProgram(ast), inParallel: false,
        reportingDiagnosticsTo: &diagnostics,
        tracingInferenceIf: nil)
    }

    static func main() async throws {
      print("async main!")
      try await MyApp.buildProgram()
      print("Built standard library")
    }
}

When running this, I get a crash during type checking:

❯ swift run hylo-async-crash
Building for debugging...
[1/1] Write swift-version--58304C5D6DBC2206.txt
Build complete! (0.16s)
async main!
fish: Job 1, 'swift run hylo-async-crash' terminated by signal SIGBUS (Misaligned address error)

Note that I only get the crash with the two step async dispatch pattern in MyApp.

If I place the hylo code directly in the async main function the code completes without crashing.

[dabrahams]

1. How would you place hylo code directly in a Swift function?
2. Is there any way this can possibly be a bug in Hylo? It sounds like a mis-compilation of Swift code.

[koliyo]

1. With hylo code, I mean the hylo compiler code, skipping the extra layer of async indirection. If I take the content of the buildProgram function and put it directly in the main function, I no longer get the crash.

2. I am very curious to the cause of this bug, and I think it is worth investigating as it could potentially be some "hidden" problem in the hylo compiler that might turn up at some other time(?). It is very possible it is some misconfiguration on my part, but the code for reproducing the bug is very minimal. It is just the main branch of the hylo repo, with the changeset linked above (https://github.com/hylo-lang/hylo/commit/c60d2dd2e8c8f4f74c9fdc68820e9554f7634267). It is just an additional executable declared in the Package.swift, and the short main entry point shown above.

As a start it would be interesting to hear if anyone else can reproduce the bug. As I said, on my machine it happens 100% of the time.

[kyouko-taiga]

Sounds like the spooky bug we had with parallel type checking. See here https://github.com/hylo-lang/hylo/issues/1162

[koliyo]

Ah interesting! Yeah that makes sense, blowing the stack, and the dispatch threads having a smaller stack to work with.