ghcmod checks are broken if GHC built for dynamic linking

[marmistrz]

Cross-reporting from: https://github.com/alanz/vscode-hie-server/issues/56

Open the following file:

-- {-# LANGUAGE NoMonomorphismRestriction #-}
import Test.QuickCheck hiding((===))
import Control.Monad(liftM2)
import Control.Applicative((<$>),(<*>))

under Arch Linux. Then the following false positive is reported:

file: 'file:///path/to/file/Test.hs'
severity: 'Error'
message: 'Could not find module ‘Test.QuickCheck’
Use -v to see a list of the files searched for.'
at: '2,1'
source: 'ghcmod'
code: 'undefined'

QuickCheck is installed

$ pacman -Ql haskell-quickcheck
haskell-quickcheck /usr/
haskell-quickcheck /usr/lib/
haskell-quickcheck /usr/lib/ghc-8.2.2/
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/All.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Arbitrary.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Exception.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Function.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Gen.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Gen/
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Gen/Unsafe.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Modifiers.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Monadic.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Poly.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Property.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Random.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/State.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Test.dyn_hi
haskell-quickcheck /usr/lib/ghc-8.2.2/site-local/QuickCheck-2.10.1/Test/QuickCheck/Text.dyn_hi
haskell-quickcheck /usr/lib/libHSQuickCheck-2.10.1-pyTUKjDV2C3TaEubSGdV9-ghc8.2.2.so
haskell-quickcheck /usr/share/
haskell-quickcheck /usr/share/doc/
haskell-quickcheck /usr/share/doc/haskell-quickcheck/
haskell-quickcheck /usr/share/haskell/
haskell-quickcheck /usr/share/haskell/register/
haskell-quickcheck /usr/share/haskell/register/haskell-quickcheck.sh
haskell-quickcheck /usr/share/haskell/unregister/
haskell-quickcheck /usr/share/haskell/unregister/haskell-quickcheck.sh
haskell-quickcheck /usr/share/licenses/
haskell-quickcheck /usr/share/licenses/haskell-quickcheck/
haskell-quickcheck /usr/share/licenses/haskell-quickcheck/LICENSE

[alanz]

How do you build your project? using stack? cabal? Does it build ok using one of those two?

[marmistrz]

@alanz, I'm using neither stack nor cabal. I have just a couple of hs files and execute the tests with runhaskell

[alanz]

Ok, I just tested it on Debian, and it works.

But I have installed that stuff using cabal install, and I think that is what ghc-mod is expecting.

I am not sure how the arch linux runhaskell script works

[marmistrz]

I'll check if installing it manually with cabal helps. Well, it's just as though if I were compiling it manually with ghc.

[marmistrz]

The problem persists when quickcheck was installed by cabal. The resulting .so file is located in

~/.cabal/lib/x86_64-linux-ghc-8.2.2/libHSQuickCheck-2.11.3-19mdm5lhmxMFf4AANg83KE-ghc8.2.2.so

I think the problem may be the fact that Arch Linux links Haskell dynamically. If cabal install is invoked without , the cabal install quickcheck fails.

$ cabal install quickcheck            
Resolving dependencies...
Downloading QuickCheck-2.11.3...
Configuring QuickCheck-2.11.3...
Building QuickCheck-2.11.3...
Failed to install QuickCheck-2.11.3
Build log ( /home/marcin/.cabal/logs/ghc-8.2.2/QuickCheck-2.11.3-19mdm5lhmxMFf4AANg83KE.log ):
cabal: Entering directory '/tmp/cabal-tmp-30442/QuickCheck-2.11.3'
Configuring QuickCheck-2.11.3...
Preprocessing library for QuickCheck-2.11.3..
Building library for QuickCheck-2.11.3..
[ 1 of 15] Compiling Test.QuickCheck.Exception ( Test/QuickCheck/Exception.hs, dist/build/Test/QuickCheck/Exception.o )

Test/QuickCheck/Exception.hs:7:8: error:
    Could not find module ‘Prelude’
    There are files missing in the ‘base-4.10.1.0’ package,
    try running 'ghc-pkg check'.
    Use -v to see a list of the files searched for.
  |
7 | module Test.QuickCheck.Exception where
  |        ^^^^^^^^^^^^^^^^^^^^^^^^^

Test/QuickCheck/Exception.hs:15:1: error:
    Could not find module ‘Control.Exception’
    There are files missing in the ‘base-4.10.1.0’ package,
    try running 'ghc-pkg check'.
    Use -v to see a list of the files searched for.
   |
15 | import qualified Control.Exception as E
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cabal: Leaving directory '/tmp/cabal-tmp-30442/QuickCheck-2.11.3'
cabal: Error: some packages failed to install:
QuickCheck-2.11.3-19mdm5lhmxMFf4AANg83KE failed during the building phase. The
exception was:
ExitFailure 1

When one invokes cabal install --enable-shared --enable-executable-dynamic --disable-library-vanilla quickcheck everything is fine.

$ cabal install --enable-shared --enable-executable-dynamic --disable-library-vanilla quickcheck
Resolving dependencies...
Configuring QuickCheck-2.11.3...
Building QuickCheck-2.11.3...
Installed QuickCheck-2.11.3

Maybe HIE should provide an opportunity to use custom cabal flags?

[alanz]

I think there may be a way to pass them in.

But it might also make sense to have a flag seomthing like --arch-linux which does all that stuff.

[marmistrz]

--dynamic may be a better name. If you have any PoC, let me know, I'll test if it's a fix.

[alanz]

@DanielG, do you know if ghc-mod supports a way of passing flags to cabal? It would probably have to go through cabal-helper too.

[marmistrz]

I'm wondering too if this could be fixed upstream in cabal - I could specify these options in the config, which is ignored by cabal. https://github.com/haskell/cabal/issues/5207

[marmistrz]

I managed to set the configuration options in the ~/.cabal/config - at first I didn't notice that -- in the config file indicate a comment. Still, even though manually running cabal from the command line works perfectly, this is no fix for haskell-ide-engine. I presume that cabal-helper doesn't read ~/.cabal/config, @DanielG - is this correct?

[marmistrz]

There are two (very poor but working) workarounds:

1. Use stack for your projects.
2. Use ghc-pristine and ghc-static, adapt the GHCBIN in the hie wrapper, and manually install every single external dependency (such as mtl) with

   cabal --with-compiler=/usr/share/ghc-pristine/bin/ghc install <package>

In the second approach you probably need to have dynamic linking disabled for cabal, so your cabal install for default ghc will be utterly broken (as it is with vanilla installation) but YMMV (please check). IIRC having dynamic linking enabled for cabal messed with the HIE build process itself, but I'm not sure.

ghc-static itself takes up about 1GB of space, stack is a disk space hog too.

Using HIE with a pure cabal project seems to be utterly broken. (on Arch)

[DanielG]

@marmistrz cabal-helper doesn't read .cabal/config because it doesn't have to. caba-helper is really just a way to reliably read the information written by cabal configure. So as long as you re-build all your dependencies after changing that configuration I don't see why things shouldn't just work. Could you provide some more details on what exactly isn't working with that config set?

[marmistrz]

@DanielG let me explain more thoroughly how my test case looks like.

1. Create a new directory, say ~/proj/haskell
2. Create a single file ~/proj/haskell/main.hs with the following contents:

   import Test.QuickCheck
   main :: IO ()
   main = return ()

3. Then building with ghc -dynamic main.hs --make succeeds, note that building with ghc main.hs --make will fail (Could not find module ‘Test.QuickCheck’ because quickcheck is a dynamically linked module, this is expected).
4. Open VSCode with the HIE addon installed. Then ghcmod will start complaining about Could not find module ‘Test.QuickCheck’. (you probably need to save first)
5. Close VS Code.
6. Add the following options to ~/.cabal/config:

   library-vanilla: False
   shared: True
   executable-dynamic: True

   as described in the Arch Wiki: https://wiki.archlinux.org/index.php/Haskell#Problems_with_linking

7. Open VS Code once again. The same error will appear

[marmistrz]

I have also tried creating a full-fledged cabal project. Then the following error appears:

hie: <command line>: cannot satisfy -package-id QuickCheck-2.10.1-pyTUKjDV2C3TaEubSGdV9
    (use -v for more information)

even though cabal run succeeds. The exact steps:

0. Make sure that the haskell-quickcheck package is installed: pacman -Qs haskell-quickcheck
1. Create a new cabal project with cabal init -n --is-executable
2. Add QuickCheck to dependencies
3. Change Main.hs to:

   
   module Main where
   import Test.QuickCheck

main :: IO () main = putStrLn "Hello, Haskell!"

4. Launch VS Code 
5. The error: 

hie: : cannot satisfy -package-id QuickCheck-2.10.1-pyTUKjDV2C3TaEubSGdV9 (use -v for more information)

[mr-ubik]

Is there any update on this issue?

[jneira]

hie do not use ghcmod anymore, but i dont know if it can handle correctly ghc with dynamic linker @mr-ubik @marmistrz could you try again to check if the error is reproduced?

[PythonNut]

I believe this issue is still relevant. Here's a recipe to reproduce the problem

File: Main.hs

module Main where

main :: IO ()
main = putStrLn "Hello World!"

System: Arch Linux w/ haskell-ide-engine-git and w/o ghc-static. Note that this means static compilation does not work.

$ ghc Main.hs -o main
[1 of 1] Compiling Main             ( Main.hs, Main.o ) [Prelude changed]

Main.hs:1:8: error:
    Could not find module ‘Prelude’
    There are files missing in the ‘base-4.12.0.0’ package,
    try running 'ghc-pkg check'.
    Use -v to see a list of the files searched for.
  |
1 | module Main where
  |        ^^^^

But we can fix this by passing -dynamic.

$ ghc -dynamic Main.hs -o main
$ ./main
Hello World!

Oh the HIE side of things, we get messages like the following if we try to run lsp-haskell on Main.hs, indicating that HIE is trying to compile statically.

[Trace - 02:02:47 PM] Received notification 'textDocument/publishDiagnostics'.
Params: {
  "diagnostics": [
    {
      "message": "",
      "source": "bios",
      "range": {
        "end": {
          "character": 0,
          "line": 1
        },
        "start": {
          "character": 0,
          "line": 0
        }
      },
      "severity": 1
    },
    {
      "message": "Could not find module ‘Prelude’\nThere are files missing in the ‘base-4.12.0.0’ package,\ntry running 'ghc-pkg check'.\nUse -v to see a list of the files searched for.",
      "source": "bios",
      "range": {
        "end": {
          "character": 11,
          "line": 0
        },
        "start": {
          "character": 7,
          "line": 0
        }
      },
      "severity": 1
    }
  ],
  "uri": "file:///home/pythonnut/Documents/Main.hs"
}

It's not super clear to me if this should still be considered the same issue, or if this should be closed and potentially a new issue created?