Error while indexing: cannot satisfy -package-id

[langston-barrett]

I built this project with the following Nix file:

{ pkgsOld   ? import <nixpkgs> { }
, compiler  ? "ghc844"
}:

let
  config = {
    packageOverrides = pkgs: rec {
      haskellPackages =
        # builtins.trace (builtins.concatStringsSep "\n" (builtins.attrNames pkgs.haskell.packages))
        pkgs.haskell.packages."${compiler}".override {
        overrides = haskellPackagesNew: haskellPackagesOld: {
          cabal-helper = pkgs.haskell.lib.doJailbreak haskellPackagesOld.cabal-helper;
        };
      };
    };
  };
  pkgs = import <nixpkgs> { inherit config; };
in with pkgs; haskellPackages.callPackage ./haskell-code-explorer.nix { }

where haskell-code-explorer was generated by cabal2nix.

I then dropped into a nix-shell for the project I want to index, and called

./result/bin/haskell-code-indexer -p ../llvm-pretty-bc-parser --dist ../llvm-pretty-bc-parser/dist

and got:

2019-01-16 10:25:18.166980006 PST : [info] haskell-code-indexer version 0.1.0.0, GHC version 8.4.4.0                                                         
2019-01-16 10:25:18.694597024 PST : [info] Indexing llvm-pretty-bc-parser-0.4.0.0                                                                            
2019-01-16 10:25:18.786759929 PST : [error] Error while indexing component lib : <command line>: cannot satisfy -package-id array-0.5.3.0                    
    (use -v for more information)

But ghc-pkg lists it:

ghc-pkg list | grep array
    array-0.5.3.0

Any idea why it can't be found? cabal build works just fine.

(One thing that might be helpful in debugging this is if the debug logging actually printed the full GHC command line rather than just the flags.

[alexwl]

Thank you for the issue!

It looks like a GHC version mismatch.

• ghc-8.6.3 comes with array-0.5.3.0
• ghc-8.4.4 comes with array-0.5.2.0

(from https://www.haskell.org/platform/contents.html)

It seems that llvm-pretty-bc-parser is compiled with ghc-8.6 (because it uses array-0.5.3.0).

haskell-code-indexer is compiled with ghc-8.4.4:

2019-01-16 10:25:18.166980006 PST : [info] haskell-code-indexer version 0.1.0.0, GHC version 8.4.4.0

ghc-8.4.4 cannot find array-0.5.3.0.

Can you compile haskell-code-indexer with ghc-8.6.3? It might solve the problem.

One thing that might be helpful in debugging this is if the debug logging actually printed the full GHC command line rather than just the flags.

Good idea! I will add the GHC command line to the debug output.

[langston-barrett]

Thanks for the quick response! I can't believe I didn't catch that. Unfortunately, it doesn't seem that cabal-helper builds with GHC 8.6: https://github.com/DanielG/cabal-helper/issues/62. I'll try getting my project and haskell-code-explorer into the same build process with GHC 8.4*.

[langston-barrett]

After doing so, I'm seeing the following:

Preprocessing library for llvm-pretty-bc-parser-0.4.0.0..
Building library for llvm-pretty-bc-parser-0.4.0.0..
2019-01-16 16:50:01.102666452 PST : [info] haskell-code-indexer version 0.1.0.0, GHC version 8.4.3.0                                                         
2019-01-16 16:50:02.665394607 PST : [info] Indexing llvm-pretty-bc-parser-0.4.0.0                                                                            
Loaded package environment from /home/siddharthist/code/llvm-pretty-bc-parser/.ghc.environment.x86_64-linux-8.4.3                                            
2019-01-16 16:50:02.737419101 PST : [error] Error while indexing component setup : haskell-code-indexer: can't find a package database at /home/siddharthist/code/llvm-pretty-bc-parser/dist-newstyle/packagedb/ghc-8.4.3
2019-01-16 16:50:02.75733468 PST : [error] Error while indexing component lib : <command line>: cannot satisfy -package-id cereal-0.5.5.0-GeIcJ9Ke1T17EnoSBNrADY
    (use -v for more information)
2019-01-16 16:50:02.786510762 PST : [error] Error while indexing component exe-llvm-disasm : <command line>: cannot satisfy -package-id fgl-visualize-0.1.0.1-9anIwt2ydLg9fPF4tZN13s
    (use -v for more information)
2019-01-16 16:50:02.817692825 PST : [error] Error while indexing component exe-regression-test : <command line>: cannot satisfy -package-id foldl-1.4.3-AUrHQPGpv62GX4t8lj9o8Y
    (use -v for more information)
2019-01-16 16:50:03.126830897 PST : [info] Finished
> ghc-pkg list | grep cereal
    cereal-0.5.5.0

Perhaps this package shouldn't look for identical hashes, but just identical versions?

[alexwl]

Unfortunately, it doesn't seem that cabal-helper builds with GHC 8.6

cabal-helper from haskell-code-explorer repository: https://github.com/alexwl/haskell-code-explorer/tree/master/vendor/cabal-helper-0.8.1.2 builds with GHC 8.6.3. The fix for 'Could not deduce (Control.Monad.Fail.MonadFail m)' is here: https://github.com/alexwl/haskell-code-explorer/commit/e6d0b7bf0954f941357d77e8158aa52bb1c96686#diff-2574ae577de5ab7486769680da74a2c9 . (stack build command successfully builds cabal-helper from vendor/cabal-helper-0.8.1.2, haskell-code-indexer and haskell-code-server with GHC 8.6.3.). It seems that cabal-helper from Hackage doesn't support GHC 8.6.3 yet.

Is it possible to use cabal-helper from haskell-code-explorer/vendor/cabal-helper-0.8.1.2 for the Nix build?

[alexwl]

After doing so, I'm seeing the following:

Preprocessing library for llvm-pretty-bc-parser-0.4.0.0..
Building library for llvm-pretty-bc-parser-0.4.0.0..
2019-01-16 16:50:01.102666452 PST : [info] haskell-code-indexer version 0.1.0.0, GHC version 8.4.3.0                                                         
2019-01-16 16:50:02.665394607 PST : [info] Indexing llvm-pretty-bc-parser-0.4.0.0                                                                            
Loaded package environment from /home/siddharthist/code/llvm-pretty-bc-parser/.ghc.environment.x86_64-linux-8.4.3                                            
2019-01-16 16:50:02.737419101 PST : [error] Error while indexing component setup : haskell-code-indexer: can't find a package database at /home/siddharthist/code/llvm-pretty-bc-parser/dist-newstyle/packagedb/ghc-8.4.3
2019-01-16 16:50:02.75733468 PST : [error] Error while indexing component lib : <command line>: cannot satisfy -package-id cereal-0.5.5.0-GeIcJ9Ke1T17EnoSBNrADY
    (use -v for more information)
2019-01-16 16:50:02.786510762 PST : [error] Error while indexing component exe-llvm-disasm : <command line>: cannot satisfy -package-id fgl-visualize-0.1.0.1-9anIwt2ydLg9fPF4tZN13s
    (use -v for more information)
2019-01-16 16:50:02.817692825 PST : [error] Error while indexing component exe-regression-test : <command line>: cannot satisfy -package-id foldl-1.4.3-AUrHQPGpv62GX4t8lj9o8Y
    (use -v for more information)
2019-01-16 16:50:03.126830897 PST : [info] Finished

It would be helpful to see command line arguments for haskell-code-indexer and the output of haskell-code-indexer with -v option.

[langston-barrett]

Thanks again for taking a look! I'll see if I can get Nix building the vendored version. Any chance those changes will be upstreamed?

Here's the CLI:

haskell-code-indexer -v -p . --dist dist/

and the log: log.txt

[alexwl]

I assume that llvm-pretty-bc-parser was built using cabal build command (not cabal new-build). It's strange that GHC tries to use a package database from llvm-pretty-bc-parser/dist-newstyle/packagedb/ghc-8.4.3 (dist-newstyle is a new-build-specific dist directory: https://github.com/haskell/cabal/issues/5731 ). llvm-pretty-bc-parser/dist-newstyle/packagedb/ghc-8.4.3 might be mentioned in a package environment file (.ghc.environment.x86_64-linux-8.4.3). Is it possible to remove that file?

Any chance those changes will be upstreamed?

It looks like the problem has already been fixed in master. The code that caused 'Could not deduce (Control.Monad.Fail.MonadFail m)' error while compiling cabal-helper-0.8.1.2 with GHC-8.6.3 (getPackageId function) was removed from the master branch of cabal-helper.

[langston-barrett]

Okay, after running

rm -rf dist*
rm .ghc*

I still get a very similar result: log.txt

[alexwl]

Can I get llvm-pretty-bc-parser/dist/setup-config file? (the first line of setup-config contains versions of GHC and Cabal)

I've compiled (using GHC-8.6.3, cabal-install 2.4.1.0 and cabal new-build command) and indexed the master branch from github.com/GaloisInc/llvm-pretty-bc-parser: llvm-pretty-bc-parser.tar.gz

You can unpack the archive and start the server: haskell-code-server -p PATH/llvm-pretty-bc-parser

[langston-barrett]

setup-config.zip

I imagine that whatever is going on has to do with the fact that I am getting the libraries involved via a nix-shell...

[alexwl]

I imagine that whatever is going on has to do with the fact that I am getting the libraries involved via a nix-shell...

This, indeed, seems to be the case.

It turns out that the path to a global package database is /nix/store/lxsr3smq4896r09pkjmdgjjnn0m109wd-ghc-8.4.3-with-packages/lib/ghc-8.4.3/package.conf.d (I got that path from the setup-config file using cabal-helper0.8.1.2-Cabal2.2.0.1 PROJ_DIR DIST_DIR print-lbi --human command), but /nix/store/lxsr3smq4896r09pkjmdgjjnn0m109wd-ghc-8.4.3-with-packages/lib/ghc-8.4.3/package.conf.d package database is not in the list of options for GHC that cabal-helper generates (haskell-code-indexer uses GHC options from cabal-helper to initialize GHC session). At the moment, I can't say if it is a bug in cabal-helper or not.

haskell-code-indexer allows to add arbitrary GHC options, so it might help to specify the package database manually:

haskell-code-indexer -v -p . --dist dist/ --ghc "-package-db /nix/store/lxsr3smq4896r09pkjmdgjjnn0m109wd-ghc-8.4.3-with-packages/lib/ghc-8.4.3/package.conf.d" (https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/packages.html#package-databases)

Of course, there is also an option to build and index llvm-pretty-bc-parser without Nix.

[langston-barrett]

@alexwl That worked like a charm! Thanks for all the help, I'm very excited to be able to use this.

Here was my Nix solution:

{ pkgs ? import ../../galois-haskell-nix/pinned-pkgs.nix { } }:

let name  = "llvm-pretty-bc-parser";
    gpkgs = import ../../galois-haskell-nix/default.nix { compiler = "ghc843"; };
    this  = gpkgs.haskellPackages.${name};
    ghcWH = gpkgs.haskellPackages.ghcWithHoogle (hpkgs: with hpkgs; [
      ghcid
    ] ++ this.buildInputs ++ this.propagatedBuildInputs);
    ghcPkgConf = ghcWH + "/lib/ghc-8.4.3/package.conf.d";
in with pkgs; stdenv.mkDerivation {
  inherit name;
  src = if lib.inNixShell then null else lib.sourceFilesBySuffices ../. [ ".cabal" ".hs" ];
  shellHook = ''
    alias haskell-code-indexer='haskell-code-indexer --ghc "-package-db ${ghcPkgConf}"'
  '';
  buildInputs = [
    ghcWH
    gpkgs.haskellPackages.haskell-code-explorer
    haskellPackages.cabal-install
  ];
}

[alexwl]

@siddharthist Awesome, thank you for the Nix file!