Segfault using TemplateHaskell (NixOs, MacOs for now)

[fiadliel]

In a simple project with polysemy included, the addition of the makeSem call into Template Haskell causes HLS to crash reliably (segmentation fault). With the makeSem line commented out, the plugin starts correctly.

e.g.

{-# LANGUAGE TemplateHaskell #-}

module Main where

import Polysemy
import Polysemy.Reader

data MyEffect m a where
  Test :: m a -> MyEffect m a

makeSem ''MyEffect

A repository that should demonstrate the issue is at https://github.com/fiadliel/effects

[lukel97]

I can't reproduce this with HLS built from source on mac, going to try with the 0.2.2 binary

[lukel97]

I can't reproduce it with the 0.2.2 binary either, with this file:

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE TemplateHaskell #-}

module Main where

import Polysemy
import Polysemy.Reader

data MyEffect m a where
  Test :: m a -> MyEffect m a

makeSem ''MyEffect

and with polysemy-1.3.0.0 on ghc-8.10.1. What version of GHC are you using? Can you pass on any of the logs too?

[fiadliel]

This is ghc 8.4.4 (from a haskell.nix setup). There was a previous bug about Template Haskell incompatibilities at https://github.com/input-output-hk/haskell.nix/issues/400 but that's been fixed for a while.

If you have nix installed, you should be able to create the environment with nix-shell. You would probably want to use the cachix cache earnestresearch-public.cachix.org (unless you like compiling a lot).

I can also look at getting something from lldb, though it's been a while (and different language) ago 😅

If the answer is that haskell-language-server doesn't support this use case, it's not too bad as long as ghcide continues to work (this is the extant working system that we can fall back to); in this case, ghcide is compiled against the nix environment's ghc.

---

Basic info from lldb (not knowing what I'm doing):

(lldb) cont
Process 15466 resuming
Process 15466 stopped
* thread #27, stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
    frame #0: 0x0000000110da766f
->  0x110da766f: movq   0x5(%rbx), %rbx
    0x110da7673: movq   %rbx, %rcx
    0x110da7676: movq   %rax, %rbx
    0x110da7679: movq   %rcx, 0x8(%rbp)
Target 0: (haskell-language-server-macOS-8.8.4) stopped.
(lldb) bt
* thread #27, stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x0000000110da766f
    frame #1: 0x61000000000012cf

---

Log messages:

❯ ~/Downloads/haskell-language-server-macOS-8.8.4 --debug
haskell-language-server version: 0.2.1.0 (GHC: 8.8.4) (PATH: /Users/gcoady/Downloads/haskell-language-server-macOS-8.8.4) (GIT hash: a6f4aba69e45dfb48bc5721537fef120971d3899)
(haskell-language-server)Ghcide setup tester in /Users/gcoady/src/github.com/fiadliel/effects.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Step 1/4: Finding files to test in /Users/gcoady/src/github.com/fiadliel/effects
Found 2 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
[INFO] Consulting the cradle for "/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/Users/gcoady/src/github.com/fiadliel/effects", cradleOptsProg = CradleAction: Cabal}
> cabal: Unknown target
> '/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs'.
> The package servant-with-effects has no file target 'Setup.hs'.
>
>
File:     /Users/gcoady/src/github.com/fiadliel/effects/Setup.hs
Hidden:   no
Range:    1:0-2:0
Source:   cradle
Severity: DsError
Message:
  Failed to parse result of calling cabal

  cabal: Unknown target
  '/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs'.
  The package servant-with-effects has no file target 'Setup.hs'.

[INFO] Consulting the cradle for "/Users/gcoady/src/github.com/fiadliel/effects/src/Main.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/Users/gcoady/src/github.com/fiadliel/effects", cradleOptsProg = CradleAction: Cabal}
> Warning: No remote package servers have been specified. Usually you would have
> one specified in the config file.
> Resolving dependencies...
> Build profile: -w ghc-8.8.4 -O1
> In order, the following will be built (use -v for more details):
>  - servant-with-effects-0.1.0.0 (exe:servant-with-effects) (configuration changed)
> Configuring executable 'servant-with-effects' for servant-with-effects-0.1.0.0..
> Warning: The 'license-file' field refers to the file 'LICENSE' which does not
> exist.
> Preprocessing executable 'servant-with-effects' for servant-with-effects-0.1.0.0..
[INFO] Using interface files cache dir: /Users/gcoady/.cache/ghcide/main-1c5eedb9d24916147b5cef09d100a08962c59f40
[INFO] Making new HscEnv[main]
[INFO] Consulting the cradle for "/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/Users/gcoady/src/github.com/fiadliel/effects", cradleOptsProg = CradleAction: Cabal}
> cabal: Unknown target
> '/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs'.
> The package servant-with-effects has no file target 'Setup.hs'.
>
>
zsh: segmentation fault  ~/Downloads/haskell-language-server-macOS-8.8.4 --debug

[fiadliel]

And tried with 0.2.2, same error; version is:

haskell-language-server version: 0.2.2.0 (GHC: 8.8.4) (PATH: /Users/gcoady/Downloads/haskell-language-server-macOS-8.8.4-1) (GIT hash: e44f618c11b7978264a94beeee00d1f014867f6d)

[lukel97]

Hm those logs don't look good, not sure why its setting up a cradle multiple times for the same file. I've never used nix before but I think its about time I install it to see what's going on...

[lukel97]

I just noticed your nix expression ends up downloading ghc 8.4.4 but your cabal constraints suggest ghc 8.8.4 This was to bootstrap 8.8.4 =

[lukel97]

Took a while to build without the caches, but I can recreate it now. Time to figure out what's going on

[Galagora]

I'm also facing the same problem on NixOS, using HLS 0.3.0.0 with GHC 8.8.3 (and 8.8.4). The source file is the same (hello world with makeSem).

logs

``` $ haskell-language-server haskell-language-server version: 0.3.0.0 (GHC: 8.8.3) (PATH: /nix/store/zly7m0p5sd0iv8qn1n97gxkx4lkbh00z-haskell-language-server-0.3.0.0/bin/haskell-language-server) (haskell-language-server)Ghcide setup tester in /home/ao/code/haskell-starter. Report bugs at https://github.com/haskell/haskell-language-server/issues Tool versions found on the $PATH cabal: 3.2.0.0 stack: Not found ghc: 8.8.3 Step 1/4: Finding files to test in /home/ao/code/haskell-starter Found 1 files Step 2/4: Looking for hie.yaml files that control setup Found 1 cradle Step 3/4: Initializing the IDE Step 4/4: Type checking the files [INFO] Consulting the cradle for "/home/ao/code/haskell-starter/src/Main.hs" Output from setting up the cradle Cradle {cradleRootDir = "/home/ao/code/haskell-starter", cradleOptsProg = CradleAction: Cabal} > Warning: No remote package servers have been specified. Usually you would have > one specified in the config file. > Resolving dependencies... > Build profile: -w ghc-8.8.3 -O1 > In order, the following will be built (use -v for more details): > - starter-0.1.0.0 (exe:starter) (configuration changed) > Configuring executable 'starter' for starter-0.1.0.0.. > Preprocessing executable 'starter' for starter-0.1.0.0.. [INFO] Using interface files cache dir: /home/ao/.cache/ghcide/main-97733fd591cce96508bbfeab608cd02ca28697c1 [INFO] Making new HscEnv[main] [1] 11008 segmentation fault (core dumped) haskell-language-server ```

[Galagora]

Same issue when using freer-simple's makeEffect (equivalent to makeSem) and motor's reflectEvents, both Template Haskell functions. Does HLS not support TH or is this merely a bug in the implementation?

[pepeiborra]

What type of cradle (hie.yaml) are you using?

There is a possibly related TH idempotency bug in ghcide that @maralorn is investigating.

[Galagora]

Just a single cradle with a single component (executable). The correct settings are auto-detected, but here's the hie.yaml:

cradle:
  cabal:
    - path: "./src/"
      component: "habit:exe:hab"

[pepeiborra]

Does your Cabal descriptor consist of only one executable stanza, or also a library stanza?

[fiadliel]

I've tried with the following setups:

Nix setups:

• ghc 8.8.4, single executable stanza, use ghc versions of ghc-included libs: crash
• ghc 8.8.4, two cabal stanzas (lib+executable), use ghc versions of ghc-included libs: crash
• ghc 8.8.4, single executable stanza, use cabal versions of ghc-included libs: crash
• ghc 8.8.4, two cabal stanzas (lib+executable), use ghc versions of ghc-included libs: crash
• ghc 8.10.1, all the above combinations: works fine

ghcup:

• ghc 8.8.4: works fine
• ghc 8.10.1: works fine

[pepeiborra]

Thanks for testing! We have previously found bugs with ghc 8.8.4 that didn't arise with 8.10. Hereby closing as won't fix

[Galagora]

I still get the segfault using GHC 8.10.2, with a single executable stanza, and any TH function invocation. cabal-install 3.2.0.0, if that's relevant.

$ haskell-language-server --version
haskell-language-server version: 0.4.0.0 (GHC: 8.10.2) (PATH: /nix/store/sy4l8aqvvgzcc0av7zvx3dj2k8pg979r-haskell-language-server-0.4.0.0/bin/haskell-language-server)

habit.cabal

``` cabal-version: 2.4 name: habit version: 0.1.0.0 executable hab main-is: Main.hs build-depends: base ^>= 4.14.1.0 , text , haskell-say , freer-simple hs-source-dirs: src default-language: Haskell2010 ```

src/Main.hs

```haskell {-# LANGUAGE DataKinds #-} {-# LANGUAGE GADTs #-} {-# LANGUAGE TypeOperators #-} {-# LANGUAGE LambdaCase #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE TemplateHaskell #-} module Main where import HaskellSay (haskellSay) import Data.Text (Text) import Control.Monad.Freer (Eff, Member, send) import Control.Monad.Freer.TH data Console r where PutTextLn :: Text -> Console () makeEffect ''Console -- putTextLn :: Member Console effs => Text -> Eff effs () -- putTextLn = send . PutTextLn main :: IO () main = haskellSay "Hello, Haskell! You're using a function from another module!" ```

Please reopen the issue.

[jneira]

I will reopen, i think we have to tackle the problems with template haskell as a whole, there is several issues about

[jneira]

@galagora for completeness: the error you got with that minimal case is using NixOs and the same output as https://github.com/haskell/haskell-language-server/issues/277#issuecomment-671039968?

[jneira]

ghcide issues involving th and segfaults:

• https://github.com/haskell/ghcide/issues/444
• https://github.com/haskell/ghcide/issues/422
• https://github.com/haskell/ghcide/issues/775

[Galagora]

Yes, except the last line is:

[1]    5849 segmentation fault (core dumped)  haskell-language-server

instead of

zsh: segmentation fault  ~/Downloads/haskell-language-server-macOS-8.8.4 --debug

Which is probably just a platform difference.

[Galagora]

OK, so, for me at least, the bug seems to have nothing to do with ghcide; the executable for that package seems to run fine, while HLS segfaults. It also doesn't seem to have anything to do with the length function in Data.Text.Short, as suggested here. Rather, it occurs when using any Template Haskell at all. See this ultraminimal repro case:

$ haskell-language-server
haskell-language-server version: 0.4.0.0 (GHC: 8.10.2) (PATH: /nix/store/7j0ivnv2nmri3idp7nm60kailxp5a0xq-haskell-language-server-0.4.0.0/bin/haskell-language-server)
(haskell-language-server)Ghcide setup tester in /home/ao/code/hello.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Tool versions found on the $PATH
cabal:      3.2.0.0
stack:      2.3.3
ghc:        8.10.2

Step 1/4: Finding files to test in /home/ao/code/hello
Found 2 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
[INFO] Consulting the cradle for "/home/ao/code/hello/src/TH.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/home/ao/code/hello", cradleOptsProg = CradleAction: Cabal}
> Resolving dependencies...
> Build profile: -w ghc-8.10.2 -O0
> In order, the following will be built (use -v for more details):
>  - hello-0.1.1 (exe:hello) (configuration changed)
> Configuring executable 'hello' for hello-0.1.1..
> Preprocessing executable 'hello' for hello-0.1.1..
[INFO] Using interface files cache dir: /home/ao/.cache/ghcide/main-3563f69436b8a2c444891c780d1baa056f3e3edc
[INFO] Making new HscEnv[main]
[1]    14843 segmentation fault (core dumped)  haskell-language-server

ghcide logs

``` $ ghcide ghcide version: 0.3.0 (GHC: 8.10) (PATH: /nix/store/qmwdfrx9bkdr4knyxwi04d1xlik50ap5-ghcide-exe-ghcide-0.3.0/bin/ghcide) Ghcide setup tester in /home/ao/code/hello. Report bugs at https://github.com/digital-asset/ghcide/issues Step 1/4: Finding files to test in /home/ao/code/hello Found 2 files Step 2/4: Looking for hie.yaml files that control setup Found 1 cradle (/home/ao/code/hello/hie.yaml) Step 3/4: Initializing the IDE Step 4/4: Type checking the files [INFO] Consulting the cradle for "/home/ao/code/hello/src/TH.hs" Output from setting up the cradle Cradle {cradleRootDir = "/home/ao/code/hello", cradleOptsProg = CradleAction: Cabal} > Resolving dependencies... > Build profile: -w ghc-8.10.2 -O0 > In order, the following will be built (use -v for more details): > - hello-0.1.1 (exe:hello) (configuration changed) > Configuring executable 'hello' for hello-0.1.1.. > Preprocessing executable 'hello' for hello-0.1.1.. [INFO] Using interface files cache dir: /home/ao/.cache/ghcide/main-3563f69436b8a2c444891c780d1baa056f3e3edc [INFO] Making new HscEnv[main] Completed (2 files worked, 0 files failed) ```

hello.cabal

``` cabal-version: 2.4 name: hello version: 0.1.1 executable hello main-is: Main.hs other-modules: TH hs-source-dirs: src ghc-options: -Wall -Wwarn -fwarn-tabs build-depends: base , template-haskell default-language: Haskell2010 ```

src/TH.hs

```haskell {-#LANGUAGE TemplateHaskell#-} module TH ( thLit -- , thFunc -- , thFTyped ) where import Language.Haskell.TH thLit :: Q Exp thLit = [| 15 |] -- thFunc :: Q Exp -- thFunc = [| \x -> x + 1|] -- thFTyped :: Q (TExp (Integer -> Integer)) -- thFTyped = [|| \y -> y + 1 ||] ```

src/Main.hs

```haskell {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE TemplateHaskell #-} module Main where import Debug.Trace import TH main :: IO () main = do print ($(thLit) :: Integer) ```

hie.yaml

```yaml cradle: cabal: - path: "./src/Main.hs" component: "hello:exe:hello" - path: "./src/TH.hs" component: "hello:exe:hello" ```

That's pretty much the smallest piece of Template Haskell you can write.

[Galagora]

@pepeiborra Why is a ghcide submodule used? Which changes are made to it? What would it take to build with the master branch?

[pepeiborra]

https://github.com/haskell/ghcide/issues/663

[jneira]

@galagora wow, many thanks for minimize the repro case and report all the logs

[Galagora]

@jneira Yeah, no problem. Two things though:

1. Would it be possible to get this issue pinned, and perhaps prioritized? I feel not segfaulting in the face of TH is a big issue (you can't use the TH features of any extensible effects library, lens, aeson, etc., which can make some things really difficult (or impossible)). You can probably unpin (and close) the Nix issue, which has been solved (latest version is in Nixpkgs).

2. How would you begin to solve this problem? Given that haskell-language-server uses ghcide at the core, should I look for the problem there? In that case, if I can't reproduce the issue with ghcide 3.0.0, should merging the upstream (at least theoretically) solve the issue? Otherwise, which modules should I look for?

Currently, I'm trying to build and test the version of ghcide in the HLS submodule. I'll also try gdb, but that seems too low-level.

[jneira]

Yeah, the issue probably belongs to ghcide. The branch of ghcide used in hls is supposed to be merged onto ghcide master and it has not much diverged commits so i would test ghcide master first, it would isolate the cause in those commits.

Agree in pinning the issue so the use of TH is persasive, not sure it it does not work for all cases.

sure @wz1000 and @pepeiborra could help you to find the root cause better than me though :wink:

[Galagora]

The issue is actually fixed as of the latest ghcide submodule revision bump; the only thing left is place the new version in Nixpkgs, which requires resolving #404. Feel free to close this issue.

[jneira]

@fiadliel i am gonna close this one, feel free to reopen if you continue experiencing the issue

[HugoPeters1024]

@galagora I'm also getting a Segmentation fault (core dumped) from hls-8.10.4 (installed with ghcup) after starting to work with template haskell. Has the fix been released yet? Or should I do something manual to fix this?

[wz1000]

With ghc-8.10.{3,4}, the issue is likely is a ghc bug, you should either downgrade to ghc-8.10.2 or wait for ghc-8.10.5

See https://gitlab.haskell.org/ghc/ghc/-/issues/19417

[HugoPeters1024]

Thanks for the quick response. I downgraded, but the error is exactly the same :(

[HugoPeters1024]

Also for the record, the build (cabal build) succeeds just fine for all versions, leading me to believe that it is not a ghc bug.

[wz1000]

You can reproduce it with ghc-8.10.2? Is this code public? What operating system are you using?

You can also try using the dynamic linker to compile ghcide (see https://cabal.readthedocs.io/en/3.4/cabal-project.html?highlight=dynamic#cfg-field-executable-dynamic).

Also for the record, the build (cabal build) succeeds just fine for all versions, leading me to believe that it is not a ghc bug.

That doesn't really say much, the bug is a race condition which is triggered by large programs being compiled in parallel (which is exactly the strategy ghcide uses).

[HugoPeters1024]

It's a school project so dear professor, if you read this please don't hit me with them plagiarism, https://git.science.uu.nl/h.a.peters/afp-querybuilder. I think master might me broken, but I'll try to fix it first thing tomorrow

[Phavion]

Master is now fixed in the above project! I am working on the same project as @HugoPeters1024