Error after installing with stack

[codygman]

$ stack install haskell-awk
$ hawk '[1..3]'
error: Won't compile:
    <no location info>:
    Could not find module ‘System.Console.Hawk.Runtime.Base’
    Use -v to see a list of the files searched for.

[gelisam]

Hawk is a bit unusual in that its executable needs to know, at runtime, where its companion library has been installed. This is unusual because usually this information is only required at compile time, while linking the library into the executable.

The reason is that Hawk interprets code written by the user, and that code is modified to call into functions from the haskell-awk library, and in order for those calls to succeed we must pass the correct --package-db flag to the Haskell interpreter at runtime. We already have code to detect whether the library has been installed into a sandbox, including older-style cabal-dev sandboxes, and it looks like we'll now have to support stack's style of sandbox, whatever that is.

[gelisam]

I learned today that stack creates a .stack-work directory, which is apparently similar to a project sandbox. I'll have to try it out myself to see whether it makes sense to point --package-db there.

[codygman]

There is a global package db as well in ~/.stack/global/.stack-work. Searching my .stack directory with find looks like this:

cody@cody-G46VW:~$ find .stack/ -iname "*hawk*"
.stack/global/.stack-work/install/x86_64-linux/lts-2.16/7.8.4/lib/x86_64-linux-ghc-7.8.4/haskell-awk-1.1.1/System/Console/Hawk
.stack/global/.stack-work/install/x86_64-linux/lts-2.16/7.8.4/bin/hawk

On Wed, Jul 15, 2015 at 1:38 PM, Samuel Gélineau notifications@github.com wrote:

I learned today https://www.reddit.com/r/haskell/comments/3dcqwj/are_sandboxes_relevant_to_stack/ that stack creates a .stack-work directory, which is apparently similar to a project sandbox. I'll have to try it out myself to see whether it makes sense to point --package-db there.

— Reply to this email directly or view it on GitHub https://github.com/gelisam/hawk/issues/150#issuecomment-121707763.

[gelisam]

Thanks, that's exactly the information I need to make it work.

[gelisam]

Progress: I managed to install stack and I can reproduce the problem.

[gelisam]

Hmm, when I install from the source (stack init; stack install), stack installs the library in a completely different place:

$ find ~/.stack ~/.local ~/.ghc ~/.cabal .stack-work -iname 'hawk*'
~/.local/bin/hawk
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/hawk
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/hawk/hawk
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/hawk/hawk-tmp
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/hawk/hawk-tmp/System/Console/Hawk
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/hawk/hawk-tmp/System/Console/Hawk.hi
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/hawk/hawk-tmp/System/Console/Hawk.o
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/System/Console/Hawk
.stack-work/install/x86_64-osx/lts-2.18/7.8.4/bin/hawk
.stack-work/install/x86_64-osx/lts-2.18/7.8.4/lib/x86_64-osx-ghc-7.8.4/haskell-awk-1.1.1/System/Console/Hawk

Since I'm looking for the library's path relative to the executable's location, I'd be able to find ../lib/x86_64-osx-ghc-7.8.4/ from the .stack-work/install/x86_64-osx/lts-2.18/7.8.4/bin/hawk executable, but from the ~/.local/bin/hawk, there are no clues at all as to where the library is actually installed.

[gelisam]

Good news! I forgot that I had already solved this problem before :)

Instead of looking at the executable's location, I am looking at the cached copy of the executable's path, the one generated by cabal in the autogenerated module Paths_haskell_awk. Since that path points to .stack-work/install/x86_64-osx/lts-2.18/7.8.4/bin/hawk, my relative path strategy should work just fine.

[gelisam]

That autogenerated module also caches the lib folder. Why am I not simply using that?

[gelisam]

Answer: because that's not the path I need. I need to give a path which --package-db will understand.

stack
bindir     = "/Users/gelisam/working/haskell/hawk/.stack-work/install/x86_64-osx/lts-2.18/7.8.4/bin"
libdir     = "/Users/gelisam/working/haskell/hawk/.stack-work/install/x86_64-osx/lts-2.18/7.8.4/lib/x86_64-osx-ghc-7.8.4/haskell-awk-1.1.1"
package-db = "/Users/gelisam/working/haskell/hawk/.stack-work/install/x86_64-osx/lts-2.18/7.8.4/pkgdb"

cabal install (global)
bindir     = "/Users/gelisam/.cabal/bin"
libdir     = "/Users/gelisam/.cabal/lib/x86_64-osx-ghc-7.8.4/haskell-awk-1.1.1"
package-db = "/Users/gelisam/.ghc/x86_64-darwin-7.8.4/package.conf.d"

cabal install (sandbox)
bindir     = "/Users/gelisam/working/haskell/hawk/.cabal-sandbox/bin"
libdir     = "/Users/gelisam/working/haskell/hawk/.cabal-sandbox/lib/x86_64-osx-ghc-7.8.4/haskell-awk-1.1.1"
package-db = "/Users/gelisam/working/haskell/hawk/.cabal-sandbox/x86_64-osx-ghc-7.8.4-packages.conf.d"

As the above example data demonstrates, bindir always ends with bin and libdir always ends with lib/x86_64-osx-ghc-7.8.4/haskell-awk-1.1.1. Stripped of these redundant pieces, they both contain the same information: the path to the folder which contains both bin and lib.

Unfortunately, that folder is not the package-db, and the transformation from that folder's path to the package-db varies depending on the sandbox. So that's why Sandbox.hs needs some logic to figure out which kind of sandbox was used in order to find the package-db. I now need to add some custom logic to accomodate stack.

[gelisam]

The logic to for stack is slightly different than for the other two. I think I'll create a small DSL to express path finding, this way I'll be able to describe the different logics without too much duplication.

[gelisam]

My new sandbox-detection code on branch issue-150 now correctly finds the package-db path .stack-work/install/x86_64-osx/lts-2.18/7.8.4/pkgdb, but Hawk still fails at runtime with the same Could not find module ‘System.Console.Hawk.Runtime.Base’ error.

Looking at that package database, I see something fishy:

$ ghc-pkg list --package-db .stack-work/install/x86_64-osx/lts-2.18/7.8.4/pkgdb
WARNING: there are broken packages.  Run 'ghc-pkg check' for more details.
.stack-work/install/x86_64-osx/lts-2.18/7.8.4/pkgdb
   haskell-awk-1.1.1
$ ghc-pkg check --package-db .stack-work/install/x86_64-osx/lts-2.18/7.8.4/pkgdb
There are problems in package haskell-awk-1.1.1:
  Warning: haddock-interfaces: /Users/gelisam/working/haskell/hawk/.stack-work/install/x86_64-osx/lts-2.18/7.8.4/doc/haskell-awk-1.1.1/haskell-awk.haddock doesn't exist or isn't a file
  dependency "stringsearch-0.3.6.6-ffe4de52ac1c1bda03645ffa8654d90c" doesn't exist

The following packages are broken, either because they have a problem
listed above, or because they depend on a broken package.
haskell-awk-1.1.1

[gelisam]

Ah! I need to pass both .stack-work/install/x86_64-osx/lts-2.18/7.8.4/pkgdb and ~/.stack/snapshots/x86_64-osx/lts-2.18/7.8.4/pkgdb.

[gelisam]

Now the logic for stack and the other sandboxes is even more divergent: stack requires two sandboxes, one of which is unrelated to the bindir.

Thankfully, @snoyberg explained that the sandboxes can be obtained via the HASKELL_PACKAGE_SANDBOXES environment variable, accessible at compile-time via TemplateHaskell's runIO. The logic to obtain that is even more different than the logic for the other sandboxes, but it's a much better solution: if all the sandboxing tools provided the information that way, I wouldn't have to add custom code to support each new one as they are introduced. So I'm now using HASKELL_PACKAGE_SANDBOXES if it exists, and falling back to the old method otherwise. It works great.

I wish cabal supported HASKELL_PACKAGE_SANDBOXES as well, then I could get rid of the old method entirely!

[gelisam]

@codygman, do you need a point release for this, or are you fine with using the source version until the next major release?