Segfaults with GHC 9.6

[9999years]

Your environment

Since upgrading to GHC 9.6.1, we've been seeing a lot of segfaults in haskell-language-server. This has persisted to the latest master commit 3bc4196d6a9f93bab4b2e4658869f97710d0fa30.

We've been seeing a lot of segfaults with GHC 9.6.1 in general. Our compiler engineer @wavewave has worked on and rebased this patch to GHC 9.6.1, which has improved the situation somewhat but hasn't completely ameliorated the issue. I had been hopeful the patch would fix the haskell-language-server instability, but haven't had luck so far. There's also a chance this is a GHC bug and not a haskell-language-server bug.

• Which OS do you use? macOS
• Which version of GHC do you use and how did you install it? 9.6.1 from nixpkgs
• How is your project built (alternative: link to the project)? cabal
• Which LSP client (editor/plugin) do you use? Neovim
• Which version of HLS do you use and how did you install it? Built from the latest master (3bc4196d6a9f93bab4b2e4658869f97710d0fa30). Installed with Nix. We're also building the other packages in this repo from the latest master, including ghcide and hie-compat.

• Have you configured HLS in any way (especially: a hie.yaml file)? Yes.

  cradle:
  multi:
    - path: "./docs"
      config: { cradle: { none:  } }
    - path: "./scripts"
      config: { cradle: { direct: { arguments: [] } } }
    - path: "./.packages"
      config: { cradle: { none: } }
    - path: "./Setup.hs"
      config: { cradle: { none: } }
    - path: "./nix/packages/mercury"
      config: { cradle: { direct: { arguments: [] } } }
    - path: "./"
      config:
        cradle:
          cabal:
            - path: "./src"
              component: "lib:test-dev"
            - path: "./test"
              component: "lib:test-dev"
            - path: "./app"
              component: "exe:mwb"
            - path: "./one-off-task"
              component: "exe:one-off-tasks"
            - path: "./benchmark"
              component: "benchmark:benchmark"
            - path: "./test-main"
              component: "test:test"

Steps to reproduce

1. Open the editor and let it load our Haskell backend (~5700 modules). Unfortunately, our codebase is proprietary and can't be shared. I'll see if I can reproduce on an open-source codebase.
2. Make an edit and save the file. I typically add a definition like foo :: (); foo = ().
3. haskell-language-server segfaults immediately.

Expected behaviour

haskell-language-server should reindex and continue to provide language intelligence.

Actual behaviour

haskell-language-server segfaults and exits unceremoniously.

Debug information

There are two bits of good news with this bug. The first is that on macOS, a crash report is automatically written to ~/Library/Logs/DiagnosticReports. The other is that it is very consistently reproducible.

Crashed Thread:        32  ghc_worker

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000040
Exception Codes:       0x0000000000000001, 0x0000000000000040

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [61666]

VM Region Info: 0x40 is not in any region.  Bytes before following region: 105553518919616
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      MALLOC_NANO (reserved)   600018000000-600020000000 [128.0M] rw-/rwx SM=NUL  ...(unallocated)

...

Thread 32 Crashed:: ghc_worker
0   ???                                    0x1046a3bd0 updateThunk + 392
1   ???                                    0x10469caa8 schedule + 1188
2   ???                                    0x10469caa8 schedule + 1188
3   libsystem_pthread.dylib                0x18a64ada0 thread_start + 8

It seems to always crash at updateThunk + 392.

[9999years]

I've filed an issue upstream in the GHC repository here.

[9999years]

This was indeed a GHC bug. We'll have a merge request open upstream soon.