Nix / Nixos support

[alanz]

See https://github.com/NixOS/nixpkgs/issues/31697

[mrkgnao]

Here are two logs with hie --lsp --vomit. In both cases, the essential error is

real-hie: <command line>: cannot satisfy -package-id base-4.10.0.0

Behaviour observed

In both cases, the following behaviour occurs:

• VSCode starts, tons of HLint warnings are shown.
• Hovering produces a Loading... tooltip that never resolves.
• Fixing the offending code does not dismiss the HLint warning.
• Refactoring does not work.

My environment

I am using NixOS (latest stable, 17.09).

I generated a hie executable from the Nix derivation in the linked thread, using nix-build, and put it on my $PATH as real-hie. I then made a wrapper script called hie:

#!/bin/sh
real-hie -d -l /tmp/hie.log --vomit $@ | tee /tmp/hie-wire.log

The logs

The first one is from a nix-shell+Cabal project, where VSCode was started from inside a Nix shell.

info: Found Cabal project at: /home/mrkgnao/code/sans/sound-and-complete
info: Using Cabal project at: /home/mrkgnao/code/sans/sound-and-complete
Using hoogle db at: /home/mrkgnao/code/sans/sound-and-complete/hiehoogledb.hoo
info: Found Cabal project at: /home/mrkgnao/code/sans/sound-and-complete
info: Using Cabal project at: /home/mrkgnao/code/sans/sound-and-complete
DEBUG: reading cache: /home/mrkgnao/code/sans/sound-and-complete/dist/setup-config.ghc-mod.cabal-components
DEBUG: resolveEntrypoint:
       ["-i","-idist/build/sound-and-complete/sound-and-complete-tmp","-isrc","-idist/build/sound-and-complete/autogen","-idist/build/global-autogen","-Idist/build/sound-and-complete/autogen","-Idist/build/global-autogen","-Idist/build/sound-and-complete/sound-and-complete-tmp","-optP-include","-optPdist/build/sound-and-complete/autogen/cabal_macros.h"]
Warning: resolveModule "/home/mrkgnao/code/sans/sound-and-complete/src/Main.md":
         cannot compile this file to desired target:
/home/mrkgnao/code/sans/sound-and-complete/src/Main.md
DEBUG: resolveEntrypoint: []
DEBUG: making sure autogen files exist
DEBUG: autogen files out of sync
DEBUG: writing Cabal autogen files
DEBUG: reading cache: dist/setup-config.ghc-mod.resolved-components
DEBUG: Could not find a component assignment, falling back to
       picking library component in cabal file.
VOMIT: Using the following mapped files:
VOMIT: Initializing GHC session with following options: "-fbuilding-cabal-package" "-O" "-outputdir" "dist/build/sound-and-complete/sound-and-complete-tmp" "-odir" "dist/build/sound-and-complete/sound-and-complete-tmp" "-hidir" "dist/build/sound-and-complete/sound-and-complete-tmp" "-stubdir" "dist/build/sound-and-complete/sound-and-complete-tmp" "-i" "-idist/build/sound-and-complete/sound-and-complete-tmp" "-isrc" "-idist/build/sound-and-complete/autogen" "-idist/build/global-autogen" "-Idist/build/sound-and-complete/autogen" "-Idist/build/global-autogen" "-Idist/build/sound-and-complete/sound-and-complete-tmp" "-optP-include" "-optPdist/build/sound-and-complete/autogen/cabal_macros.h" "-hide-all-packages" "-Wmissing-home-modules" "-package-id" "base-4.10.0.0" "-package-id" "mtl-2.2.1-19EL8AGBsN3DnnOhrC9xY3" "-package-id" "transformers-0.5.2.0" "-package-id" "safe-0.3.15-5SmTamu8dz8KnTA9MfNcRD" "-package-id" "lens-4.15.4-4q970k0XvkV7aqAQ5eBtP3" "-package-id" "prettyprinter-1.1.1-2QNu42iMc4MGCD9K0ZJaq" "-package-id" "hspec-2.4.4-FJuRtB7WMYqKF0brZ7icE5" "-package-id" "containers-0.5.10.2" "-package-id" "adjunctions-4.3-18NVfKnieoxAHaA9U5pxMh" "-package-id" "profunctors-5.2.1-1lOUsE8aiaZLtdvF0ROsOX" "-package-id" "text-1.2.2.2-5RuPYjcevF59B6YENNKuiP" "-package-id" "bytestring-0.10.8.2" "-package-id" "hashable-1.2.6.1-BqrB4iRJk0C8e42ef63nR4" "-package-id" "unordered-containers-0.2.8.0-KsuUwaupoRi1cUuuci0Zxb" "-XHaskell2010" "-pgmL" "markdown-unlit" "-O0" "-Wall"
DEBUG: initSession: Session not initialized, creating new one
real-hie: <command line>: cannot satisfy -package-id base-4.10.0.0
    (use -v for more information)

The second one is a new Stack project with nix: {enable: true} in stack.yaml (I think all this does is use Nix for GHC and zlib).

info: Found Stack project at: /home/mrkgnao/code/testproj
Warning: 'dist/setup-config' exists, ignoring Stack project
info: Found Cabal project at: /home/mrkgnao/code/testproj
info: Using Cabal project at: /home/mrkgnao/code/testproj
Using hoogle db at: /home/mrkgnao/.hoogle/default-haskell-5.0.13.hoo
info: Found Stack project at: /home/mrkgnao/code/testproj
Warning: 'dist/setup-config' exists, ignoring Stack project
info: Found Cabal project at: /home/mrkgnao/code/testproj
info: Using Cabal project at: /home/mrkgnao/code/testproj
DEBUG: reading cache: /home/mrkgnao/code/testproj/dist/setup-config.ghc-mod.cabal-components
DEBUG: resolveEntrypoint:
       ["-i","-idist/build/testproj/testproj-tmp","-isrc","-idist/build/testproj/autogen","-idist/build/global-autogen","-Idist/build/testproj/autogen","-Idist/build/global-autogen","-Idist/build/testproj/testproj-tmp","-optP-include","-optPdist/build/testproj/autogen/cabal_macros.h"]
DEBUG: resolveEntrypoint: []
DEBUG: making sure autogen files exist
DEBUG: autogen files out of sync
DEBUG: writing Cabal autogen files
DEBUG: reading cache: dist/setup-config.ghc-mod.resolved-components
VOMIT: Using the following mapped files: "/home/mrkgnao/code/testproj/src/Main.hs"
VOMIT: Initializing GHC session with following options: "-fbuilding-cabal-package" "-O" "-outputdir" "dist/build/testproj/testproj-tmp" "-odir" "dist/build/testproj/testproj-tmp" "-hidir" "dist/build/testproj/testproj-tmp" "-stubdir" "dist/build/testproj/testproj-tmp" "-i" "-idist/build/testproj/testproj-tmp" "-isrc" "-idist/build/testproj/autogen" "-idist/build/global-autogen" "-Idist/build/testproj/autogen" "-Idist/build/global-autogen" "-Idist/build/testproj/testproj-tmp" "-optP-include" "-optPdist/build/testproj/autogen/cabal_macros.h" "-hide-all-packages" "-Wmissing-home-modules" "-package-id" "base-4.10.0.0" "-XHaskell2010" "-O0" "-Wall"
DEBUG: initSession: Session not initialized, creating new one
real-hie: <command line>: cannot satisfy -package-id base-4.10.0.0
    (use -v for more information)

[alanz]

Please show the contents of /tmp/hie.log

[mrkgnao]

So the problem in the Stack project turned out to be the old ghc-mod dist/setup-config thing, and deleting the dist/ folder fixed it.

The Nix project, on the other hand, seemed to be an incompatibility in GHC versions. (The output about Main.md was irrelevant and had to do with a markdown-unlit literate file, which I've changed back into a plain .hs for simplicity.)

I compiled hie from source using stack-8.2.1.yaml. Here's the output from VSCode with --vomit:

info: Found Cabal project at: /home/mrkgnao/code/sans/sound-and-complete
info: Using Cabal project at: /home/mrkgnao/code/sans/sound-and-complete
info: Found Cabal project at: /home/mrkgnao/code/sans/sound-and-complete
info: Using Cabal project at: /home/mrkgnao/code/sans/sound-and-complete
DEBUG: reading cache: /home/mrkgnao/code/sans/sound-and-complete/dist/setup-config.ghc-mod.cabal-components
DEBUG: resolveEntrypoint:
       ["-i","-idist/build/sound-and-complete/sound-and-complete-tmp","-isrc","-idist/build/sound-and-complete/autogen","-idist/build/global-autogen","-Idist/build/sound-and-complete/autogen","-Idist/build/global-autogen","-Idist/build/sound-and-complete/sound-and-complete-tmp","-optP-include","-optPdist/build/sound-and-complete/autogen/cabal_macros.h"]
DEBUG: resolveEntrypoint: []
DEBUG: making sure autogen files exist
DEBUG: autogen files out of sync
DEBUG: writing Cabal autogen files
DEBUG: reading cache: dist/setup-config.ghc-mod.resolved-components
DEBUG: Could not find a component assignment, falling back to
       picking library component in cabal file.
VOMIT: Using the following mapped files:
VOMIT: Initializing GHC session with following options: "-fbuilding-cabal-package" "-O" "-outputdir" "dist/build/sound-and-complete/sound-and-complete-tmp" "-odir" "dist/build/sound-and-complete/sound-and-complete-tmp" "-hidir" "dist/build/sound-and-complete/sound-and-complete-tmp" "-stubdir" "dist/build/sound-and-complete/sound-and-complete-tmp" "-i" "-idist/build/sound-and-complete/sound-and-complete-tmp" "-isrc" "-idist/build/sound-and-complete/autogen" "-idist/build/global-autogen" "-Idist/build/sound-and-complete/autogen" "-Idist/build/global-autogen" "-Idist/build/sound-and-complete/sound-and-complete-tmp" "-optP-include" "-optPdist/build/sound-and-complete/autogen/cabal_macros.h" "-hide-all-packages" "-Wmissing-home-modules" "-package-id" "base-4.10.0.0" "-package-id" "mtl-2.2.1-19EL8AGBsN3DnnOhrC9xY3" "-package-id" "transformers-0.5.2.0" "-package-id" "safe-0.3.15-5SmTamu8dz8KnTA9MfNcRD" "-package-id" "lens-4.15.4-4q970k0XvkV7aqAQ5eBtP3" "-package-id" "prettyprinter-1.1.1-2QNu42iMc4MGCD9K0ZJaq" "-package-id" "hspec-2.4.4-FJuRtB7WMYqKF0brZ7icE5" "-package-id" "containers-0.5.10.2" "-package-id" "adjunctions-4.3-18NVfKnieoxAHaA9U5pxMh" "-package-id" "profunctors-5.2.1-1lOUsE8aiaZLtdvF0ROsOX" "-package-id" "text-1.2.2.2-5RuPYjcevF59B6YENNKuiP" "-package-id" "bytestring-0.10.8.2" "-package-id" "hashable-1.2.6.1-BqrB4iRJk0C8e42ef63nR4" "-package-id" "unordered-containers-0.2.8.0-KsuUwaupoRi1cUuuci0Zxb" "-XHaskell2010" "-pgmL" "markdown-unlit" "-O0" "-Wall"
DEBUG: initSession: Session not initialized, creating new one
real-hie: <command line>: cannot satisfy -package-id mtl-2.2.1-19EL8AGBsN3DnnOhrC9xY3
    (use -v for more information)

[Fresheyeball]

I have but a bounty on this issue. https://www.bountysource.com/issues/51713360-nix-nixos-support would love to see this resolved ASAP.

[domenkozar]

@fresheyeball @imalsogreg do you want to have IDE support for projects using nix-shell/nix-build or is stack+nix enough?

[imalsogreg]

@domenkozar Preferably nix-shell aware; most of my work is on projects with dependencies managed by nix.

[Fresheyeball]

@domenkozar I don't use stack. I'm in the same boat as @imalsogreg.

[shlevy]

https://github.com/NixOS/nixpkgs/pull/32947 fixes this I think.

[alanz]

Does anything need to change in https://github.com/haskell/haskell-ide-engine#installation-with-nix because of this?

[shlevy]

@alanz Those instructions are still reasonable for building and installing a local checkout with stack + nix, but end-users will probably want to install from nixpkgs which they can do with my PR. If it would be welcome, I'd be happy to also add nix expressions to make it possible to do a self-contained (not via stack) nix build from a nix checkout.

[shlevy]

Note that there may be some confusion on my part if this addresses the issue here. The idea with that PR is that your nix environment (nix-shell, nix-buffer, whatever) would build haskell-ide-engine with the same haskell package set as your package is using, and that would fix the version mismatch here. But @Fresheyeball is testing to see if that's enough.

[Fresheyeball]

@shlevy would you mind reaching out on slack?

[shlevy]

@Fresheyeball Which slack?

[Fresheyeball]

Idk, I am on fpchat.

[shlevy]

@imalsogreg @Fresheyeball The nixpkgs PR is now merged, testing appreciated 😄

[domenkozar]

I have prepared hie executables for GHC 8.0.2 and GHC 8.2.2 with Nix in reproducible mannner: https://github.com/domenkozar/hie-nix - let me know what you think

[shlevy]

@domenkozar This seems very hard to integrate in with nixpkgs haskell package sets

[michalrus]

@domenkozar, so if I add import-from-derivation this ghc-8.2.nix to my project’s default.nix/shell.nix… will it see correct dependencies declared in that same default.nix (e.g. by callCabal2Nix ./.)?

@shlevy is this what you mean, that it won’t?

[domenkozar]

@shlevy why would you need that?

[shlevy]

@domenkozar Because we're defining package environments for our projects based on nixpkgs (plus our own overlays), and we want (well, need) to have the same dependencies (not just the same GHC version!) to use hie with our projects.

[shlevy]

So I want to make an env that contains "the hie appropriate for this project" and launch that hie per-project.

[michalrus]

^ Exactly. What’s the point otherwise?

[domenkozar]

Afaik HIE does this at runtime using ghc-mod, so you'd have to run it with nix-shell of your environment.

[michalrus]

It does; working v. well. =) Thank you, @domenkozar!

---

In general, when developing Haskell programs on Nix, it is nice to enter a Nix shell with the required packages

Also, if you can specify path to a HIE wrapper in your editor, you can use a wrapper like https://github.com/michalrus/kornel/blob/8e0d466e4f23a8479a268eaff5940537a335e08d/hie.sh to not have to run the whole editor inside an impure nix-shell (crazy!).

(This -A hie from the wrapper is defined in https://github.com/michalrus/kornel/blob/8e0d466e4f23a8479a268eaff5940537a335e08d/default.nix#L40-L48 — a bit clunky, but this definition survives GC!)

• Emacs: https://github.com/michalrus/dotfiles/blob/bdc726eb8847a9f70275587001d37fb489a9b059/dotfiles/emacs/.emacs.d/init.d/080-proglang-haskell.el#L32-L34.

• vscode: https://github.com/alanz/vscode-hie-server/issues/48.

[domenkozar]

@Fresheyeball @imalsogreg could you provide a Nix project you'd like to see HIE working with? I think with hie-nix it should be trivial, but I want to make sure we agree that's the acceptance criteria for this to pass :)

[srid]

Here's a skeleton Nix project: https://github.com/ElvishJerricco/reflex-project-skeleton

The frontend package can be compiled using ghcjs, although for editor integration purposes one could compile it using ghc. It even targets android and iOS.

Should be a good candidate to test HIE support?

[alanz]

Once this thing settles down to best practice, can someone make a PR on the README, for future NIX users?

[domenkozar]

@alanz https://github.com/haskell/haskell-ide-engine#installation-with-nix

@srid @ElvishJerricco made it work, but not sure he posted the steps (although they are as in README with usual caveats).

Is this considered then done? :)

[alanz]

Well, the link is good, but the instructions at https://github.com/domenkozar/hie-nix/blob/master/README.md don't work.

We now seem to have

• default.nix
• ghc-8.0.nix
• ghc-8.2.nix

Rather than hie-nix.nix

[domenkozar]

It did work (I've tested locally), but I see your point, it was confusing.

What about this: https://github.com/domenkozar/hie-nix#install

[alanz]

Yep, that works.

And a pretty neat way of doing it.

Thanks.

[alanz]

@domenkozar I presume at some stage this will end up on a binary cache somewhere?

My install (admittedly for a fresh nixpkgs install) is building the world.

[domenkozar]

Yes, that's the plan :)

[Camsbury]

Writing here to thank @michalrus again. Coming back to this months later, and your strategy has worked for me.