search

input-output-hk / haskell.nix

Alternative Haskell Infrastructure for Nixpkgs
https://input-output-hk.github.io/haskell.nix
Apache License 2.0
564 stars 237 forks source link

Use Nix flakes rather than Niv to pin dependencies #1674

Closed yvan-sraka closed 2 years ago

yvan-sraka commented 2 years ago

nix-tools use old-style default.nix/shell.nix files (pinning dependencies with niv), rather than flakes (and flake-compat retro-compatibility) as haskell.nix does.

Since default.nix is already quite straightforward and pure, it should not be hard to update it!

michaelpj commented 2 years ago

We've also discussed merging nix-tools into haskell.nix, since they're extremely coupled in practice and it's a big pain to update. Could we do that instead?

michaelpj commented 2 years ago

https://github.com/input-output-hk/haskell.nix/issues/957

hamishmack commented 2 years ago

We are thinking we will move he nix-tools/cabal.project to cabal.project and set up the flake.nix so that nix develop in the root of the haskell.nix repo will start a shell for working on nix-tools.

denizdogan commented 2 years ago

Did this subtly break something? I'm new to Haskell.nix, and I'm getting this weird error which seems to stem from this change:

error: cannot fetch input 'path:./nix-tools?lastModified=1&narHash=sha256-pqNsBhQkO%2fnT991ASoSXsn07P7lNqpTP0wIV85wft30=' because it uses a relative path
hamishmack commented 2 years ago

@denizdogan it may have. Is there a way I can reproduce that error?

denizdogan commented 2 years ago

@hamishmack Maybe you can, I'm not sure. Try and see if you can with the latest commit here: https://github.com/denizdogan/dhall-haskell/tree/denizdogan/use-haskell.nix-flakes

~/Code ❯❯❯ cd dhall-haskell/dhall-json
direnv: loading ~/Code/dhall-haskell/dhall-json/.envrc
direnv: using flake
trace: No index state specified for haskell-project, using the latest index state that we know about (2022-10-02T00:00:00Z)!
evaluating derivation 'git+file:///Users/deniz/Code/dhall-haskell?dir=dhall-json#devShells.aarch64-darwin.default'direnv: ([/nix/store/62nw2lg44mnzpl84k3pv9l41m79dgxrr-direnv-2.32.1/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
trace: No index state specified for cabal-install, using the latest index state that we know about (2022-10-02T00:00:00Z)!
trace: No index state specified for haskell-language-server, using the latest index state that we know about (2022-10-02T00:00:00Z)!
trace: No index state specified for hlint, using the latest index state that we know about (2022-10-02T00:00:00Z)!
error: cannot fetch input 'path:./nix-tools?lastModified=1&narHash=sha256-pqNsBhQkO%2fnT991ASoSXsn07P7lNqpTP0wIV85wft30=' because it uses a relative path
(use '--show-trace' to show detailed location information)
direnv: nix-direnv: renewed cache
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +LD_DYLD_PATH +MACOSX_DEPLOYMENT_TARGET +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_aarch64_apple_darwin +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_aarch64_apple_darwin +NIX_CFLAGS_COMPILE +NIX_COREFOUNDATION_RPATH +NIX_DONT_SET_RPATH +NIX_DONT_SET_RPATH_FOR_BUILD +NIX_ENFORCE_NO_NATIVE +NIX_GHC_LIBDIR +NIX_HARDENING_ENABLE +NIX_IGNORE_LD_THROUGH_GCC +NIX_INDENT_MAKE +NIX_LDFLAGS +NIX_NO_SELF_RPATH +NIX_STORE +NM +PATH_LOCALE +RANLIB +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +__darwinAllowLocalNetworking +__impureHostDeps +__propagatedImpureHostDeps +__propagatedSandboxProfile +__sandboxProfile +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +enableDWARF +exactDeps +installPhase +mesonFlags +name +nativeBuildInputs +out +outputs +packageSetupDeps +patches +phases +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system +withHaddock ~PATH ~XDG_DATA_DIRS

As far as I know, dhall-haskell doesn't have any previous dependency on nix-tools, it's only from Haskell.nix.

hamishmack commented 2 years ago

I tried nix develop in that dir and it worked. I am using a very recent nix version 2.12.0pre20220921_f704c27, but I am wondering if the use of relative paths in flake inputs might be newish. What version of nix are you using?

hamishmack commented 2 years ago

I think the nix-tools path should be interpreted as relative to its parent. Here is what I see when I run nix flake info.

$ nix flake info
warning: 'nix flake info' is a deprecated alias for 'nix flake metadata'
warning: Git tree '/Users/hamish/iohk/dhall-haskell' is dirty
Resolved URL:  git+file:///Users/hamish/iohk/dhall-haskell?dir=dhall-json
Locked URL:    git+file:///Users/hamish/iohk/dhall-haskell?dir=dhall-json
Description:   A very basic flake
Path:          /nix/store/3j3zwcg8jcz3w188lrv9a5akvmcljcyy-source
Last modified: 2022-10-04 23:38:10
Inputs:
├───flake-utils: github:numtide/flake-utils/c0e246b9b83f637f4681389ecabcb2681b4f3af0
├───haskellNix: github:input-output-hk/haskell.nix/e49b15fefce296f3fb359b43eb5c2534d9d810a4
│   ├───HTTP: github:phadej/HTTP/9bc0996d412fef1787449d841277ef663ad9a915
│   ├───cabal-32: github:haskell/cabal/48bf10787e27364730dd37a42b603cee8d6af7ee
│   ├───cabal-34: github:haskell/cabal/942639c18c0cd8ec53e0a6f8d120091af35312cd
│   ├───cabal-36: github:haskell/cabal/f27667f8ec360c475027dcaee0138c937477b070
│   ├───cardano-shell: github:input-output-hk/cardano-shell/9392c75087cb9a3d453998f4230930dea3a95725
│   ├───flake-compat: github:input-output-hk/flake-compat/eca47d3377946315596da653862d341ee5341318
│   ├───flake-utils: github:numtide/flake-utils/3cecb5b042f7f209c56ffd8371b2711a290ec797
│   ├───ghc-8.6.5-iohk: github:input-output-hk/ghc/95713a6ecce4551240da7c96b6176f980af75cae
│   ├───hackage: github:input-output-hk/hackage.nix/e83134e6ed9e1d9a112457dda1f5b99d4bbe1ed7
│   ├───hpc-coveralls: github:sevanspowell/hpc-coveralls/14df0f7d229f4cd2e79f8eabb1a740097fdfa430
│   ├───hydra: github:NixOS/hydra/28b682b85b7efc5cf7974065792a1f22203a5927
│   │   ├───nix: github:NixOS/nix/a1cd7e58606a41fcf62bf8637804cf8306f17f62
│   │   │   ├───lowdown-src: github:kristapsdz/lowdown/d2c2b44ff6c27b936ec27358a2653caaef8f73b8
│   │   │   ├───nixpkgs: github:NixOS/nixpkgs/82891b5e2c2359d7e58d08849e4c89511ab94234
│   │   │   └───nixpkgs-regression: github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2
│   │   └───nixpkgs follows input 'haskellNix/hydra/nix/nixpkgs'
│   ├───nix-tools: path:./nix-tools?lastModified=1&narHash=sha256-IVlPOoFDRn7b9gfvNb8agit97toR2GzAwU8HDvu3crY=
│   │   ├───flake-utils: github:numtide/flake-utils/c0e246b9b83f637f4681389ecabcb2681b4f3af0
│   │   ├───haskellNix: github:input-output-hk/haskell.nix/5990fee152c54a325665659cdcc51cb833e69960
│   │   │   ├───HTTP: github:phadej/HTTP/9bc0996d412fef1787449d841277ef663ad9a915
│   │   │   ├───cabal-32: github:haskell/cabal/48bf10787e27364730dd37a42b603cee8d6af7ee
│   │   │   ├───cabal-34: github:haskell/cabal/942639c18c0cd8ec53e0a6f8d120091af35312cd
│   │   │   ├───cabal-36: github:haskell/cabal/f27667f8ec360c475027dcaee0138c937477b070
│   │   │   ├───cardano-shell: github:input-output-hk/cardano-shell/9392c75087cb9a3d453998f4230930dea3a95725
│   │   │   ├───flake-compat: github:input-output-hk/flake-compat/eca47d3377946315596da653862d341ee5341318
│   │   │   ├───flake-utils: github:numtide/flake-utils/3cecb5b042f7f209c56ffd8371b2711a290ec797
│   │   │   ├───ghc-8.6.5-iohk: github:input-output-hk/ghc/95713a6ecce4551240da7c96b6176f980af75cae
│   │   │   ├───hackage: github:input-output-hk/hackage.nix/916b0ed20a0c4b0648bb9375b9d2a0d7bd914617
│   │   │   ├───hpc-coveralls: github:sevanspowell/hpc-coveralls/14df0f7d229f4cd2e79f8eabb1a740097fdfa430
│   │   │   ├───hydra: github:NixOS/hydra/28b682b85b7efc5cf7974065792a1f22203a5927
│   │   │   │   ├───nix: github:NixOS/nix/a1cd7e58606a41fcf62bf8637804cf8306f17f62
│   │   │   │   │   ├───lowdown-src: github:kristapsdz/lowdown/d2c2b44ff6c27b936ec27358a2653caaef8f73b8
│   │   │   │   │   ├───nixpkgs: github:NixOS/nixpkgs/82891b5e2c2359d7e58d08849e4c89511ab94234
│   │   │   │   │   └───nixpkgs-regression: github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2
│   │   │   │   └───nixpkgs follows input 'haskellNix/nix-tools/haskellNix/hydra/nix/nixpkgs'
│   │   │   ├───nix-tools: path:./nix-tools?lastModified=1&narHash=sha256-pqNsBhQkO%2fnT991ASoSXsn07P7lNqpTP0wIV85wft30=
│   │   │   ├───nixpkgs follows input 'haskellNix/nix-tools/haskellNix/nixpkgs-unstable'
│   │   │   ├───nixpkgs-2003: github:NixOS/nixpkgs/1db42b7fe3878f3f5f7a4f2dc210772fd080e205
│   │   │   ├───nixpkgs-2105: github:NixOS/nixpkgs/046ee4af7a9f016a364f8f78eeaa356ba524ac31
│   │   │   ├───nixpkgs-2111: github:NixOS/nixpkgs/573603b7fdb9feb0eb8efc16ee18a015c667ab1b
│   │   │   ├───nixpkgs-2205: github:NixOS/nixpkgs/549d82bdd40f760a438c3c3497c1c61160f3de55
│   │   │   ├───nixpkgs-unstable: github:NixOS/nixpkgs/65fae659e31098ca4ac825a6fef26d890aaf3f4e
│   │   │   ├───old-ghc-nix: github:angerman/old-ghc-nix/af48a7a7353e418119b6dfe3cd1463a657f342b8
│   │   │   └───stackage: github:input-output-hk/stackage.nix/7a4b8115809eb17cc33c1af71b878f34a2cc876c
│   │   └───nixpkgs follows input 'haskellNix/nix-tools/haskellNix/nixpkgs-unstable'
│   ├───nixpkgs follows input 'haskellNix/nixpkgs-unstable'
│   ├───nixpkgs-2003: github:NixOS/nixpkgs/1db42b7fe3878f3f5f7a4f2dc210772fd080e205
│   ├───nixpkgs-2105: github:NixOS/nixpkgs/022caabb5f2265ad4006c1fa5b1ebe69fb0c3faf
│   ├───nixpkgs-2111: github:NixOS/nixpkgs/eabc38219184cc3e04a974fe31857d8e0eac098d
│   ├───nixpkgs-2205: github:NixOS/nixpkgs/309faedb8338d3ae8ad8f1043b3ccf48c9cc2970
│   ├───nixpkgs-unstable: github:NixOS/nixpkgs/e14f9fb57315f0d4abde222364f19f88c77d2b79
│   ├───old-ghc-nix: github:angerman/old-ghc-nix/af48a7a7353e418119b6dfe3cd1463a657f342b8
│   └───stackage: github:input-output-hk/stackage.nix/f1abb218b94cc1e94bc52185dab499f6705b6edd
└───nixpkgs follows input 'haskellNix/nixpkgs-unstable'
michaelpj commented 2 years ago

I suspect this may be dependent on the Nix version that you're using...

michaelpj commented 2 years ago

Possibly dumb question: do we need to have a separate flake for nix-tools? Can it not be part of the main flake?

denizdogan commented 2 years ago

For what it's worth, I am using Nix 2.11.1. I'm still new to Nix so I'm not super keen on trying to update to a non-stable version :)

hamishmack commented 2 years ago

That seems like it should be plenty new enough. What happens if you just run nix develop in that dir (do you get the error)? Also what platform are you on (I have been testing on an M1 using x86_64-darwin). I did limit the systems to just that as well, like this:

diff --git a/dhall-json/flake.nix b/dhall-json/flake.nix
index 44030e21..ccadff27 100644
--- a/dhall-json/flake.nix
+++ b/dhall-json/flake.nix
@@ -4,7 +4,7 @@
   inputs.nixpkgs.follows = "haskellNix/nixpkgs-unstable";
   inputs.flake-utils.url = "github:numtide/flake-utils";
   outputs = { self, nixpkgs, flake-utils, haskellNix }:
-    flake-utils.lib.eachSystem [ "x86_64-linux" "x86_64-darwin" "aarch64-darwin" ] (system:
+    flake-utils.lib.eachSystem [ "x86_64-darwin" ] (system:
     let
       overlays = [ haskellNix.overlay
         (final: prev: {

Can you reproduce the issue with just one system in there?

denizdogan commented 2 years ago

@hamishmack When I remove use flake from .envrc and then run nix develop from a fresh shell, everything seems fine, I don't get any errors. I'm not sure how caching could affect all of this and I'm honestly scared I'll have to build GHC from source again if I mess around with it. :)

Anyway, I'm on M1 with nix-darwin, so it'd be aarch64-darwin for me. I first tried using x86_64-darwin as suggested, and I have configured extra-platforms = x86_64-darwin aarch64-darwin because I saw that being suggested elsewhere some time ago.

Here's the output for x86_64-darwin:

~/C/d/dhall-json ❯❯❯ vi .envrc
[...editing...]
direnv: loading ~/Code/dhall-haskell/dhall-json/.envrc
direnv: using flake
warning: Git tree '/Users/deniz/Code/dhall-haskell' is dirty
error: flake 'git+file:///Users/deniz/Code/dhall-haskell?dir=dhall-json' does not provide attribute 'devShells.aarch64-darwin.default', 'devShell.aarch64-darwin', 'packages.aarch64-darwin.default' or 'defaultPackage.aarch64-darwin'
warning: Git tree '/Users/deniz/Code/dhall-haskell' is dirty
error: cannot fetch input 'path:./nix-tools?lastModified=1&narHash=sha256-pqNsBhQkO%2fnT991ASoSXsn07P7lNqpTP0wIV85wft30=' because it uses a relative path
(use '--show-trace' to show detailed location information)
direnv: nix-direnv: renewed cache
direnv: export ~XDG_DATA_DIRS

Here's the output for aarch64-darwin:

~/C/d/dhall-json ❯❯❯ vi flake.nix
direnv: loading ~/Code/dhall-haskell/dhall-json/.envrc
direnv: using flake
warning: Git tree '/Users/deniz/Code/dhall-haskell' is dirty
trace: No index state specified for haskell-project, using the latest index state that we know about (2022-10-02T00:00:00Z)!
trace: No index state specified for cabal-install, using the latest index state that we know about (2022-10-02T00:00:00Z)!
trace: No index state specified for haskell-language-server, using the latest index state that we know about (2022-10-02T00:00:00Z)!
evaluating derivation 'git+file:///Users/deniz/Code/dhall-haskell?dir=dhall-json#devShells.aarch64-darwin.default'direnv: ([/nix/store/62nw2lg44mnzpl84k3pv9l41m79dgxrr-direnv-2.32.1/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
trace: No index state specified for hlint, using the latest index state that we know about (2022-10-02T00:00:00Z)!
warning: Git tree '/Users/deniz/Code/dhall-haskell' is dirty
error: cannot fetch input 'path:./nix-tools?lastModified=1&narHash=sha256-pqNsBhQkO%2fnT991ASoSXsn07P7lNqpTP0wIV85wft30=' because it uses a relative path
(use '--show-trace' to show detailed location information)
direnv: nix-direnv: renewed cache
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +LD_DYLD_PATH +MACOSX_DEPLOYMENT_TARGET +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_aarch64_apple_darwin +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_aarch64_apple_darwin +NIX_CFLAGS_COMPILE +NIX_COREFOUNDATION_RPATH +NIX_DONT_SET_RPATH +NIX_DONT_SET_RPATH_FOR_BUILD +NIX_ENFORCE_NO_NATIVE +NIX_GHC_LIBDIR +NIX_HARDENING_ENABLE +NIX_IGNORE_LD_THROUGH_GCC +NIX_INDENT_MAKE +NIX_LDFLAGS +NIX_NO_SELF_RPATH +NIX_STORE +NM +PATH_LOCALE +RANLIB +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +__darwinAllowLocalNetworking +__impureHostDeps +__propagatedImpureHostDeps +__propagatedSandboxProfile +__sandboxProfile +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +enableDWARF +exactDeps +installPhase +mesonFlags +name +nativeBuildInputs +out +outputs +packageSetupDeps +patches +phases +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system +withHaddock ~PATH ~XDG_DATA_DIRS
hamishmack commented 2 years ago

How can we check what version of nix nix-direnv is using? Does it link in nix or run it as an external process? If the later, is there a way to enable verbose logging so we can see what nix commands it runs?

hamishmack commented 2 years ago

Oh, that nix-tools input is not actually used anywhere (see #1736).

hamishmack commented 2 years ago

@denizdogan please let us know if #1736 fixes this.

denizdogan commented 2 years ago

@hamishmack Sorry for taking a while to get back. It seems to work now! Thanks!