how to `unbreak` haskellPackages.ip ?

teto commented 5 years ago

Issue description

I am kind of confused by some packages in the haskellPackages on nixos-unstable 4dd5c93998da55002fdec1c715c680531420381c

More specifically, I am trying to use ip http://hackage.haskell.org/package/ip. First of all there are 2 versions in https://raw.githubusercontent.com/NixOS/nixpkgs/master/pkgs/development/haskell-modules/hackage-packages.nix , ip and ip_1_5_0 . How come two packages are generated ? I've seen pkgs/development/haskell-modules/configuration-hackage2nix.yaml but couldn't find the technical/political reason why ? Similarly when are they going to converge again ?

Similarly we have hspec and hspec_2_7_1. hspec_2_7_1 expects hspec as input.

I've tried to alias hspec to hspec_2_7_1 etc in my shell.nix:

let
  config = {
    packageOverrides = pkgs: rec {
      haskellPackages = pkgs.haskellPackages.override {
        overrides = haskellPackagesNew: hold: rec {
          hspec = hold.hspec_2_7_1;
          hspec-core = hold.hspec-core_2_7_1;
          hspec-discover = hold.hspec-discover_2_7_1;
          QuickCheck = hold.QuickCheck_2_13_1;
          # QuickCheck = haskellPackagesOld.QuickCheck_2_13_1;
          # ip_1_5_0 = haskellPackagesOld.ip_1_5_0.override { };
        };
      };
    };
    allowBroken = true;
  };

  compiler = pkgs.haskell.packages.ghc864;
  # inherit config;
  pkgs = import <nixpkgs> {  inherit config; };

  # some custom neovim configuration generator
  my_nvim = pkgs.genNeovim  [ ] { withHaskell = true; };

in
  with pkgs;

  compiler.shellFor {
  # the dependencies of packages listed in `packages`, not the
  packages = p: with p; [
    (import ./. { inherit compiler;})
  ]
  ;
  withHoogle = true;
  nativeBuildInputs = [
    all-hies.versions.ghc864
    haskellPackages.ip_1_5_0 

    haskellPackages.cabal-install
    # haskellPackages.bytestring-conversion
    # haskellPackages.gutenhasktags
    # haskellPackages.haskdogs # seems to build on hasktags/ recursively import things
    haskellPackages.hasktags
  ];

  # export HIE_HOOGLE_DATABASE=$NIX_GHC_DOCDIR as DOCDIR doesn't exist it won't work
  # or an interesting
  # shellHook = "eval $(grep export ${ghc}/bin/ghc)";
  # echo "importing a custom nvim ${my_nvim}"
  shellHook = ''

    # check if it's still needed ?
    export HIE_HOOGLE_DATABASE="$NIX_GHC_LIBDIR/../../share/doc/hoogle/index.html"
    # export runghc=" "
    source ./run_daemon

    echo "to regenerate C bindings"
    echo "make headers_install in the linux kernel"
    echo "cabal clean"
    echo "cabal configure --package-db /home/teto/netlink-hs/dist/package.conf.inplace --extra-include-dirs=/home/teto/mptcp2/build/usr/include -v3"

    echo "to run the daemon "
    echo "buildNrun"
  '';

  }

but it fails with nixos-unstable 4dd5c93998da55002fdec1c715c680531420381c

error: while evaluating anonymous function at /home/teto/mptcpnetlink/hs/default.nix:1:1, called from /home/teto/mptcpnetlink/hs/shell.nix:56:6:
while evaluating 'doHaddock' at /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:71:15, called from /home/teto/mptcpnetlink/hs/default.nix:5:1:
while evaluating 'overrideCabal' at /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:37:24, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:71:20:
while evaluating 'callCabal2nix' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:213:32, called from /home/teto/mptcpnetlink/hs/default.nix:5:24:
while evaluating 'callCabal2nixWithOptions' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:198:66, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:213:38:
while evaluating 'overrideCabal' at /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:37:24, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:209:10:
while evaluating 'callPackageKeepDeriver' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:157:33, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:209:25:
while evaluating 'overrideCabal' at /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:37:24, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:158:5:
while evaluating 'callPackage' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:114:22, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:158:20:
while evaluating 'callPackageWithScope' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:74:37, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:114:28:
while evaluating 'makeOverridable' at /home/teto/nixpkgs/lib/customisation.nix:67:24, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:97:8:
while evaluating 'drvScope' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:87:18, called from /home/teto/nixpkgs/lib/customisation.nix:69:12:
while evaluating the attribute 'nativeBuildInputs' of the derivation 'cabal2nix-netlink-pm' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:126:7:
while evaluating the attribute 'buildInputs' of the derivation 'cabal2nix-2.14.3' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'language-nix-2.1.0.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'lens-4.17.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'generic-deriving-1.12.4' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'hspec-2.7.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'hspec-core-2.7.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'hspec-meta-2.6.0' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'QuickCheck-2.13.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'splitmix-0.0.2' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'async-2.2.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'hashable-1.2.7.0' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
infinite recursion encountered, at undefined position

Steps to reproduce

run nix-shell on https://github.com/teto/netlink_pm/blob/v5/hs/shell.nix on unstable 4dd5c93998da55002fdec1c715c680531420381c

Technical details

Please run nix-shell -p nix-info --run "nix-info -m" and paste the results.

infinisil commented 5 years ago

As far as I understand, those alternate version packages come from the extra-packages field here: https://github.com/NixOS/nixpkgs/blob/4dd5c93998da55002fdec1c715c680531420381c/pkgs/development/haskell-modules/configuration-hackage2nix.yaml#L2355. Because ip_1_5_0 isn't there though, it might be a dependency of one of those, unsure about that.

Ping @peti, I am sometimes also quite lost regarding the Haskell package stuff.

In any case if you want to unbreak/fix Haskell packages, this should go to the haskell-updates branch.

peti commented 5 years ago

First of all there are 2 versions in hackage-packages.nix: ip and ip_1_5_0. How come two packages are generated ?

The default-package-overrides section of configuration-hackage2nix.yaml constrains ip to version 1.4.x as per Stackage LTS 13.x. Therefore, that is our default version and the newer version is added with a version suffix attached to make it available for people who need the latest. The same applies to hspec et al.

peti commented 5 years ago

        overrides = haskellPackagesNew: hold: rec {
          hspec = hold.hspec_2_7_1;
          hspec-core = hold.hspec-core_2_7_1;
          hspec-discover = hold.hspec-discover_2_7_1;
          QuickCheck = hold.QuickCheck_2_13_1;
          # QuickCheck = haskellPackagesOld.QuickCheck_2_13_1;
          # ip_1_5_0 = haskellPackagesOld.ip_1_5_0.override { };
        };

If you override ip = hold.ip_1_5_0, then the build should succeed. hspec is a difficult package to replace because it consists of a whole zoo of sub-packages which have mutually recursive dependencies on each other. configuration-common.nix fixes those issues for the default versions but not for the latest ones.

teto commented 5 years ago

Thanks for the tip. I am trying and it is still downloading packages. Meanwhile, I've added a maintainers/scripts/update-hackage script at https://github.com/teto/nixpkgs/blob/nixos-unstable/maintainers/scripts/update-hackage so as to avoid cloning cabal2nix to update hackage-packages.nix.

I've been doing some tests in a nix-shell but hackage2nix seems to dislike some changes I've made for an upcoming PR

  meta = with stdenv.lib; {
    platforms = platforms.linux;
    requiredKernelConfig = [ (kernel.isEnabled "OPENVSWITCH") ];  # this line disturbs hackage2nix
    description = "A multilayer virtual switch";

as it fails with

error: value is a list while a set was expected, at /home/teto/nixpkgs/pkgs/os-specific/linux/openvswitch/default.nix:62:31
hackage2nix: user error (Error in $: Failed reading: not a valid json value)

(run via NIX_PATH="nixpkgs-overlays=/var/empty:$NIX_PATH" hackage2nix --nixpkgs=$HOME/nixpkgs)

teto commented 5 years ago

Btw ip = hold.ip_1_5_0 fails with the same error :'(

error: while evaluating anonymous function at /home/teto/netlink_pm/hs/default.nix:1:1, called from /home/teto/netlink_pm/hs/shell.nix:62:6:
while evaluating 'doHaddock' at /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:71:15, called from /home/teto/netlink_pm/hs/default.nix:5:1:
while evaluating 'overrideCabal' at /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:37:24, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:71:20:
while evaluating 'callCabal2nix' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:213:32, called from /home/teto/netlink_pm/hs/default.nix:5:24:
while evaluating 'callCabal2nixWithOptions' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:198:66, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:213:38:
while evaluating 'overrideCabal' at /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:37:24, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:209:10:
while evaluating 'callPackageKeepDeriver' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:157:33, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:209:25:
while evaluating 'overrideCabal' at /home/teto/nixpkgs/pkgs/development/haskell-modules/lib.nix:37:24, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:158:5:
while evaluating 'callPackage' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:114:22, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:158:20:
while evaluating 'callPackageWithScope' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:74:37, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:114:28:
while evaluating 'makeOverridable' at /home/teto/nixpkgs/lib/customisation.nix:67:24, called from /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:97:8:
while evaluating 'drvScope' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:87:18, called from /home/teto/nixpkgs/lib/customisation.nix:69:12:
while evaluating the attribute 'nativeBuildInputs' of the derivation 'cabal2nix-netlink-pm' at /home/teto/nixpkgs/pkgs/development/haskell-modules/make-package-set.nix:126:7:
while evaluating the attribute 'buildInputs' of the derivation 'cabal2nix-2.14.3' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'language-nix-2.1.0.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'lens-4.17.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'generic-deriving-1.12.4' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'hspec-2.7.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'hspec-core-2.7.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'hspec-meta-2.6.0' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'QuickCheck-2.13.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'splitmix-0.0.2' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'propagatedBuildInputs' of the derivation 'async-2.2.1' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
while evaluating the attribute 'buildInputs' of the derivation 'hashable-1.2.7.0' at /home/teto/nixpkgs/pkgs/development/haskell-modules/generic-builder.nix:245:3:
infinite recursion encountered, at undefined position

teto commented 5 years ago

I checked the ip.cabal to see that it didn't require hspec >=2.7 contrary to what the error messages said so I guessed it was due to a test dependency. I disabled the tests for it and it worked. Before reaching this conclusion, I looked into nixpkgs-stackage, played with hackage2nix etc.

I've put up a PR to update the hacakge-package without having to clone cabal2nix repo: https://github.com/NixOS/nixpkgs/pull/62105

NixOS / nixpkgs