Add HLS to dev shell

[samhh]

Installing it per-project is preferred as HLS is bound quite tightly to specific versions of GHC.

I had a binary available for download from CI with this. Is that also the case on macOS? I wouldn't want to merge this if it needs to be built from source.

[OliverJAsh]

I had a binary available for download from CI with this. Is that also the case on macOS? I wouldn't want to merge this if it needs to be built from source.

How do I test this? I checked out the branch but it didn't look like anything was installed/built by Nix.

My .envrc has use nix.

[samhh]

My .envrc has use nix.

On master now, and with this PR, I think it should instead be use flake.

How do I test this? I checked out the branch but it didn't look like anything was installed/built by Nix.

Possibly because of the above. I'd guess your direnv has you stuck in an old shell.

Once that's fixed you'll either download ~200MB and be on your way, or you'll download and then start building HLS, which would take a while.

The idea is that you remove any global HLS (probably installed via ghcup) and use the one defined in Nix dev shells on a per-project basis. Though in theory you shouldn't need to remove any other HLS, the one installed here should take priority in $PATH. Probably which haskell-language-server-wrapper will suffice to check, see if the path is from /nix/store/ or somewhere else.

[OliverJAsh]

On master now, and with this PR, I think it should instead be use flake.

Ah, okay. I just tried this on master and it seems like it's building stuff. Once that's done I'll try switching to this branch to see if it just downloads HSL or whether it needs to build it as well.

[OliverJAsh]

I just switched branch and it seems to be building. I'll keep it running and let you know how long it takes.

[OliverJAsh]

Still going… do you want me to keep waiting?

[OliverJAsh]

Finished after 2h 19m 49s.

[samhh]

Oh... okay this might be a problem more broadly with macOS on master which would need addressing first.

Here's something else to test. Switch to master and run the following:

$ nix flake update

This updates our flake's inputs, most notably nixpkgs. For me, the result is a ~30MB download of nixpkgs followed by a few seconds of processing and then a ~360MB download of updated project dependencies. Once these have finished downloading there's nothing to build (significant enough to be slower than downloading, anyway) and I'm in the shell with the likes of ghci accessible.

Is your experience the same, or does it start slowly building a bunch of stuff?

[OliverJAsh]

Is your experience the same, or does it start slowly building a bunch of stuff?

It seems to be building:

[OliverJAsh]

Finished after 24m. I have changes in my flake.lock as well. Do I commit those?

[samhh]

No need to commit, that's just a test to see if you have binaries available on Darwin on a fresh nixpkgs revision. (In the future if you wanted to bump nixpkgs, you'd run that command and then commit the lockfile, similar to Yarn.)

$ nix flake update --override-input nixpkgs github:NixOS/nixpkgs/e06bd4b

I was going to ask you to try this, the latest commit build for Darwin at time of writing, but that's also building for me. :angry:

It may be that haskell.packages.<version> isn't ever cached in Hydra CI, maybe? These are secondary to the primary versions at ghc, haskellPackages, etc, which for the nixpkgs revision on master targets GHC 9.0.

Let's try one more thing. This patch has us target the "main" version of everything Haskell in our nixpkgs revision. It's no good for master right now as it'd bump us up to GHC 9.0, but I'm hoping at least on Darwin you'll see binaries for everything here.

diff --git a/flake.nix b/flake.nix
index ed4e28c..b562d30 100644
--- a/flake.nix
+++ b/flake.nix
@@ -12,9 +12,10 @@
           buildInputs = with pkgs; [
             git

-            haskell.compiler.ghc8107
-            haskell.packages.ghc8107.cabal-install
-            haskell.packages.ghc8107.hspec-golden
+            cabal-install
+            ghc
+            haskell-language-server
+            haskellPackages.hspec-golden

             # For typechecking golden output
             nodejs

[OliverJAsh]

On master:

$ pbpaste | git apply
warning: Git tree '/Users/oliverash/Development/unsplash/intlc' is dirty
error: Package ‘compiler-rt-libc-7.1.0’ in /nix/store/qdsi66bin3h3yimbn35fxbj7fwdpi739-source/pkgs/development/compilers/llvm/7/compiler-rt/default.nix:96 is marked as broken, refusing to evaluate.

       a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowBroken = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowBroken = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
warning: Git tree '/Users/oliverash/Development/unsplash/intlc' is dirty

[samhh]

Oh yeah, that's a separate issue we documented back in https://github.com/unsplash/intlc/pull/162.

Try commenting out haskell-language-server. That doesn't solve this PR, but at least it should mean no building anything, and ghci et al in scope.

[OliverJAsh]

I tried that and it looked like nothing needed to build. 😄

[samhh]

One more thing I want to check to round this out, because I'm wondering if HLS is still built with the same M1-unfriendly LLVM package on a newer nixpkgs.

So, on a fresh master...:

diff --git a/flake.nix b/flake.nix
index ed4e28c..b562d30 100644
--- a/flake.nix
+++ b/flake.nix
@@ -12,9 +12,10 @@
           buildInputs = with pkgs; [
             git

-            haskell.compiler.ghc8107
-            haskell.packages.ghc8107.cabal-install
-            haskell.packages.ghc8107.hspec-golden
+            cabal-install
+            ghc
+            haskellPackages.hspec-golden

             # For typechecking golden output
             nodejs

Then:

$ nix flake update --override-input nixpkgs github:NixOS/nixpkgs/e06bd4b

And finally add haskell-language-server back to the dev shell.

[OliverJAsh]

warning: Git tree '/Users/oliverash/Development/unsplash/intlc' is dirty
error: Package ‘compiler-rt-libc-7.1.0’ in /nix/store/xc6s7iccz5nkm8vl9nybz698nq2ngxlc-source/pkgs/development/compilers/llvm/7/compiler-rt/default.nix:96 is marked as broken, refusing to evaluate.

       a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowBroken = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowBroken = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
warning: Git tree '/Users/oliverash/Development/unsplash/intlc' is dirty

[samhh]

:unamused:

Okay, so we could:

• Use mainline GHC et al packages so they should be in the binary cache for all platforms. As per #162 we'll need to check these versions are compatible with ghcup's build matrix.
• Conditionally only include HLS in the shell on Linux, and instruct macOS/M1 contributors to source HLS externally such as via ghcup.

There are more exotic solutions like haskell.nix but then folks have to start modifying their global Nix configs for access to these other binary caches. I suppose this doesn't preclude anyone from simply sourcing the dependencies externally and forgoing Nix anyway.

It really would be nicer to have HLS via Nix though for all platforms. I'm going to dig some more.