Wrong package.cache with Stack

[ruhatch]

I am running ghc-mod in a Stack project with a local version of ghc-mod installed with stack build.

When I run stack exec -- ghc-mod check file everything works fine, but when Atom invokes ghc-mod I get the following trace:

Atom Version: 1.11.2 Electron Version: 1.4.4 System: linux 4.8.2-1-ARCH Thrown From: haskell-ghc-mod package, v1.19.0

Stack Trace

Haskell-ghc-mod: ghc-mod command check failed with error Error

At caps: {"version":[5,6,0,0],"fileMap":true,"quoteArgs":true,"optparse":true,"typeConstraints":true,"browseParents":true,"interactiveCaseSplit":true}
URI: /home/rupert/haskell/music/src/furElise.hs
Args: --
message: Command failed: ghc-mod check /home/rupert/haskell/music/src/furElise.hs --
Warning: STACK_EXE set, preferring Stack project
ghc-mod: /home/rupert/.stack/programs/x86_64-linux/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d/package.cache: GHC.PackageDb.readPackageDb: inappropriate type (Not a valid Unicode code point!)

log:
-0.431s: "FurElise did-save triggered"
-0.43s: "FurElise updating"
-0.389s: "Running ghc-mod check,/home/rupert/haskell/music/src/furElise.hs,-- with opts = ",{"cwd":"/home/rupert/haskell/music","env":{"ATOM_HOME":"/home/rupert/.atom","CAML_LD_LIBRARY_PATH":"/home/rupert/.opam/4.02.3/lib/stublibs","XDG_VTNR":"1","MANPATH":":/home/rupert/.opam/system/man:/home/rupert/.opam/4.02.3/man","XDG_SESSION_ID":"c1","SHELL":"/bin/zsh","TERM":"rxvt-unicode-256color","PERL5LIB":"/home/rupert/perl5/lib/perl5:/home/rupert/.opam/4.02.3/lib/perl5:/home/rupert/perl5/lib/perl5:/home/rupert/.opam/system/lib/perl5:","WINDOWID":"33554441","PERL_MB_OPT":"--install_base \"/home/rupert/perl5\"","GTK_MODULES":"canberra-gtk-module","ZSH":"/home/rupert/.oh-my-zsh","OCAML_TOPLEVEL_PATH":"/home/rupert/.opam/4.02.3/lib/toplevel","USER":"rupert","LS_COLORS":"no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=41;33;01:ex=00;32:ow=0;41:*.cmd=00;32:*.exe=01;32:*.com=01;32:*.bat=01;32:*.btm=01;32:*.dll=01;32:*.tar=00;31:*.tbz=00;31:*.tgz=00;31:*.rpm=00;31:*.deb=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.lzma=00;31:*.zip=00;31:*.zoo=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.tb2=00;31:*.tz2=00;31:*.tbz2=00;31:*.avi=01;35:*.bmp=01;35:*.fli=01;35:*.gif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mng=01;35:*.mov=01;35:*.mpg=01;35:*.pcx=01;35:*.pbm=01;35:*.pgm=01;35:*.png=01;35:*.ppm=01;35:*.tga=01;35:*.tif=01;35:*.xbm=01;35:*.xpm=01;35:*.dl=01;35:*.gl=01;35:*.wmv=01;35:*.aiff=00;32:*.au=00;32:*.mid=00;32:*.mp3=00;32:*.ogg=00;32:*.voc=00;32:*.wav=00;32:*.patch=00;34:*.o=00;32:*.so=01;35:*.ko=01;31:*.la=00;33","GHC_PACKAGE_PATH":"/home/rupert/.stack/global-project/.stack-work/install/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/programs/x86_64-linux/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d","HASKELL_PACKAGE_SANDBOXES":"/home/rupert/.stack/global-project/.stack-work/install/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb:","PAGER":"less","MOZ_PLUGIN_PATH":"/usr/lib/mozilla/plugins","LSCOLORS":"Gxfxcxdxbxegedabagacab","HASKELL_PACKAGE_SANDBOX":"/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb","MAIL":"/var/spool/mail/rupert","PATH":"/home/rupert/haskell/music/.stack-work/install/x86_64-linux/lts-7.2/8.0.1/bin:/home/rupert/.stack/snapshots/x86_64-linux/lts-7.2/8.0.1/bin:/home/rupert/perl5/bin:/home/rupert/.opam/4.02.3/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/rupert/.local/bin","PWD":"/home/rupert","EDITOR":"vim","LANG":"en_GB.UTF-8","COLORFGBG":"15;default","HOME":"/home/rupert","SHLVL":"5","STACK_EXE":"/usr/bin/stack","XDG_SEAT":"seat0","_JAVA_AWT_WM_NONREPARENTING":"1","HASKELL_DIST_DIR":".stack-work/dist/x86_64-linux/Cabal-1.22.5.0","PERL_LOCAL_LIB_ROOT":"/home/rupert/perl5:/home/rupert/perl5","LESS":"-R","LOGNAME":"rupert","DBUS_SESSION_BUS_ADDRESS":"unix:path=/run/user/1000/bus","LC_CTYPE":"en_GB.UTF-8","WINDOWPATH":"1","DISPLAY":":0","XDG_RUNTIME_DIR":"/run/user/1000","PERL_MM_OPT":"INSTALL_BASE=/home/rupert/perl5","COLORTERM":"rxvt","XAUTHORITY":"/home/rupert/.Xauthority","_":"/usr/bin/nohup","GOOGLE_API_KEY":"AIzaSyAQfxPJiounkhOjODEO5ZieffeBv6yft2Q","CHROME_DESKTOP":"Electron.desktop","NODE_PATH":"/usr/lib/atom/exports","NODE_ENV":"production"},"encoding":"utf-8","maxBuffer":null}
-0.36s: "Running ghc-mod lint,/home/rupert/haskell/music/src/furElise.hs,-- with opts = ",{"cwd":"/home/rupert/haskell/music","env":{"ATOM_HOME":"/home/rupert/.atom","CAML_LD_LIBRARY_PATH":"/home/rupert/.opam/4.02.3/lib/stublibs","XDG_VTNR":"1","MANPATH":":/home/rupert/.opam/system/man:/home/rupert/.opam/4.02.3/man","XDG_SESSION_ID":"c1","SHELL":"/bin/zsh","TERM":"rxvt-unicode-256color","PERL5LIB":"/home/rupert/perl5/lib/perl5:/home/rupert/.opam/4.02.3/lib/perl5:/home/rupert/perl5/lib/perl5:/home/rupert/.opam/system/lib/perl5:","WINDOWID":"33554441","PERL_MB_OPT":"--install_base \"/home/rupert/perl5\"","GTK_MODULES":"canberra-gtk-module","ZSH":"/home/rupert/.oh-my-zsh","OCAML_TOPLEVEL_PATH":"/home/rupert/.opam/4.02.3/lib/toplevel","USER":"rupert","LS_COLORS":"no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=41;33;01:ex=00;32:ow=0;41:*.cmd=00;32:*.exe=01;32:*.com=01;32:*.bat=01;32:*.btm=01;32:*.dll=01;32:*.tar=00;31:*.tbz=00;31:*.tgz=00;31:*.rpm=00;31:*.deb=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.lzma=00;31:*.zip=00;31:*.zoo=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.tb2=00;31:*.tz2=00;31:*.tbz2=00;31:*.avi=01;35:*.bmp=01;35:*.fli=01;35:*.gif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mng=01;35:*.mov=01;35:*.mpg=01;35:*.pcx=01;35:*.pbm=01;35:*.pgm=01;35:*.png=01;35:*.ppm=01;35:*.tga=01;35:*.tif=01;35:*.xbm=01;35:*.xpm=01;35:*.dl=01;35:*.gl=01;35:*.wmv=01;35:*.aiff=00;32:*.au=00;32:*.mid=00;32:*.mp3=00;32:*.ogg=00;32:*.voc=00;32:*.wav=00;32:*.patch=00;34:*.o=00;32:*.so=01;35:*.ko=01;31:*.la=00;33","GHC_PACKAGE_PATH":"/home/rupert/.stack/global-project/.stack-work/install/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/programs/x86_64-linux/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d","HASKELL_PACKAGE_SANDBOXES":"/home/rupert/.stack/global-project/.stack-work/install/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb:","PAGER":"less","MOZ_PLUGIN_PATH":"/usr/lib/mozilla/plugins","LSCOLORS":"Gxfxcxdxbxegedabagacab","HASKELL_PACKAGE_SANDBOX":"/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb","MAIL":"/var/spool/mail/rupert","PATH":"/home/rupert/haskell/music/.stack-work/install/x86_64-linux/lts-7.2/8.0.1/bin:/home/rupert/.stack/snapshots/x86_64-linux/lts-7.2/8.0.1/bin:/home/rupert/perl5/bin:/home/rupert/.opam/4.02.3/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/rupert/.local/bin","PWD":"/home/rupert","EDITOR":"vim","LANG":"en_GB.UTF-8","COLORFGBG":"15;default","HOME":"/home/rupert","SHLVL":"5","STACK_EXE":"/usr/bin/stack","XDG_SEAT":"seat0","_JAVA_AWT_WM_NONREPARENTING":"1","HASKELL_DIST_DIR":".stack-work/dist/x86_64-linux/Cabal-1.22.5.0","PERL_LOCAL_LIB_ROOT":"/home/rupert/perl5:/home/rupert/perl5","LESS":"-R","LOGNAME":"rupert","DBUS_SESSION_BUS_ADDRESS":"unix:path=/run/user/1000/bus","LC_CTYPE":"en_GB.UTF-8","WINDOWPATH":"1","DISPLAY":":0","XDG_RUNTIME_DIR":"/run/user/1000","PERL_MM_OPT":"INSTALL_BASE=/home/rupert/perl5","COLORTERM":"rxvt","XAUTHORITY":"/home/rupert/.Xauthority","_":"/usr/bin/nohup","GOOGLE_API_KEY":"AIzaSyAQfxPJiounkhOjODEO5ZieffeBv6yft2Q","CHROME_DESKTOP":"Electron.desktop","NODE_PATH":"/usr/lib/atom/exports","NODE_ENV":"production"},"encoding":"utf-8","maxBuffer":null}
-0.335s: "Running ghc-mod browse,-d,-p,--,FurElise with opts = ",{"cwd":"/home/rupert/haskell/music","env":{"ATOM_HOME":"/home/rupert/.atom","CAML_LD_LIBRARY_PATH":"/home/rupert/.opam/4.02.3/lib/stublibs","XDG_VTNR":"1","MANPATH":":/home/rupert/.opam/system/man:/home/rupert/.opam/4.02.3/man","XDG_SESSION_ID":"c1","SHELL":"/bin/zsh","TERM":"rxvt-unicode-256color","PERL5LIB":"/home/rupert/perl5/lib/perl5:/home/rupert/.opam/4.02.3/lib/perl5:/home/rupert/perl5/lib/perl5:/home/rupert/.opam/system/lib/perl5:","WINDOWID":"33554441","PERL_MB_OPT":"--install_base \"/home/rupert/perl5\"","GTK_MODULES":"canberra-gtk-module","ZSH":"/home/rupert/.oh-my-zsh","OCAML_TOPLEVEL_PATH":"/home/rupert/.opam/4.02.3/lib/toplevel","USER":"rupert","LS_COLORS":"no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=41;33;01:ex=00;32:ow=0;41:*.cmd=00;32:*.exe=01;32:*.com=01;32:*.bat=01;32:*.btm=01;32:*.dll=01;32:*.tar=00;31:*.tbz=00;31:*.tgz=00;31:*.rpm=00;31:*.deb=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.lzma=00;31:*.zip=00;31:*.zoo=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.tb2=00;31:*.tz2=00;31:*.tbz2=00;31:*.avi=01;35:*.bmp=01;35:*.fli=01;35:*.gif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mng=01;35:*.mov=01;35:*.mpg=01;35:*.pcx=01;35:*.pbm=01;35:*.pgm=01;35:*.png=01;35:*.ppm=01;35:*.tga=01;35:*.tif=01;35:*.xbm=01;35:*.xpm=01;35:*.dl=01;35:*.gl=01;35:*.wmv=01;35:*.aiff=00;32:*.au=00;32:*.mid=00;32:*.mp3=00;32:*.ogg=00;32:*.voc=00;32:*.wav=00;32:*.patch=00;34:*.o=00;32:*.so=01;35:*.ko=01;31:*.la=00;33","GHC_PACKAGE_PATH":"/home/rupert/.stack/global-project/.stack-work/install/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/programs/x86_64-linux/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d","HASKELL_PACKAGE_SANDBOXES":"/home/rupert/.stack/global-project/.stack-work/install/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb:","PAGER":"less","MOZ_PLUGIN_PATH":"/usr/lib/mozilla/plugins","LSCOLORS":"Gxfxcxdxbxegedabagacab","HASKELL_PACKAGE_SANDBOX":"/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb","MAIL":"/var/spool/mail/rupert","PATH":"/home/rupert/haskell/music/.stack-work/install/x86_64-linux/lts-7.2/8.0.1/bin:/home/rupert/.stack/snapshots/x86_64-linux/lts-7.2/8.0.1/bin:/home/rupert/perl5/bin:/home/rupert/.opam/4.02.3/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/rupert/.local/bin","PWD":"/home/rupert","EDITOR":"vim","LANG":"en_GB.UTF-8","COLORFGBG":"15;default","HOME":"/home/rupert","SHLVL":"5","STACK_EXE":"/usr/bin/stack","XDG_SEAT":"seat0","_JAVA_AWT_WM_NONREPARENTING":"1","HASKELL_DIST_DIR":".stack-work/dist/x86_64-linux/Cabal-1.22.5.0","PERL_LOCAL_LIB_ROOT":"/home/rupert/perl5:/home/rupert/perl5","LESS":"-R","LOGNAME":"rupert","DBUS_SESSION_BUS_ADDRESS":"unix:path=/run/user/1000/bus","LC_CTYPE":"en_GB.UTF-8","WINDOWPATH":"1","DISPLAY":":0","XDG_RUNTIME_DIR":"/run/user/1000","PERL_MM_OPT":"INSTALL_BASE=/home/rupert/perl5","COLORTERM":"rxvt","XAUTHORITY":"/home/rupert/.Xauthority","_":"/usr/bin/nohup","GOOGLE_API_KEY":"AIzaSyAQfxPJiounkhOjODEO5ZieffeBv6yft2Q","CHROME_DESKTOP":"Electron.desktop","NODE_PATH":"/usr/lib/atom/exports","NODE_ENV":"production"},"encoding":"utf-8","maxBuffer":null}
-0.001s: "Warning: STACK_EXE set, preferring Stack project\nghc-mod: /home/rupert/.stack/programs/x86_64-linux/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d/package.cache: GHC.PackageDb.readPackageDb: inappropriate type (Not a valid Unicode code point!)\n"
-0.001s: "Running ghc-mod check,/home/rupert/haskell/music/src/furElise.hs,-- failed with ",{"killed":false,"code":1,"signal":null,"cmd":"ghc-mod check /home/rupert/haskell/music/src/furElise.hs --"}
0s: {"killed":false,"code":1,"signal":null,"cmd":"ghc-mod check /home/rupert/haskell/music/src/furElise.hs --"}

Error
    at /home/rupert/.atom/packages/haskell-ghc-mod/lib/util.coffee:60:26
    at ChildProcess.exithandler (child_process.js:225:5)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:877:16)
    at Socket.<anonymous> (internal/child_process.js:334:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at Pipe._handle.close [as _onclose] (net.js:493:12)

So it seems to be picking up the wrong package.cache for some reason. Any ideas?

[lierdakil]

It seems that A) you are running ghc-mod built with ghc 8.0 (lts-7.2 resolver) on a project that seemingly wants 6.19 resolver (ghc 7.10). It won't work this way due to changes in GHC API. you need to build ghc-mod with the same compiler (and usually resolver) as the project. B) you are running via 'stack exec'. This is considered a hack and is deprecated.

Caveat: lts-6.19 has only ghc-mod 5.5. Upgrading to 5.6 (optional, recommended) is kind of a drag (as upgrading any package without changing the resolver is). Be aware of this.

Tl;dr: please check your resolvers. If necessary, run 'stack build ghc-mod' in project directory. If you want ghc-mod 5.6 on ghc 7.10, consult stack manual.

[ruhatch]

Unfortunately neither of those are true! This project is using resolver lts-7.2 and I installed ghc-mod using stack build ghc-mod. There is no global ghc-mod, so there is definitely no conflict there. I am not running it using stack exec, but what makes you think that I was?

Thanks for getting back.

[lierdakil]

Unfortunately neither of those are true!

I seem to have misread the log output, and got a bit confused about what resolver goes where. See below for a more detailed explanation.

I am not running it using stack exec, but what makes you think that I was?

Basically this line:

Warning: STACK_EXE set, preferring Stack project

This environment variable is usually set by stack exec, so that's why I assumed that was the case.

Also, you have GHC_PACKAGE_PATH environment variable set (which is also usually set by stack exec)

"GHC_PACKAGE_PATH":"/home/rupert/.stack/global-project/.stack-work/install/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/programs/x86_64-linux/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d","HASKELL_PACKAGE_SANDBOXES":"/home/rupert/.stack/global-project/.stack-work/install/x86_64-linux/lts-6.19/7.10.3/pkgdb:/home/rupert/.stack/snapshots/x86_64-linux/lts-6.19/7.10.3/pkgdb:"

Note that it's pointing to lts-6.19.

At the same time, ghc-mod is probably installed in

/home/rupert/haskell/music/.stack-work/install/x86_64-linux/lts-7.2/8.0.1/bin:/home/rupert/.stack/snapshots/x86_64-linux/lts-7.2/8.0.1/bin

Note lts-7.2 here.

So yes, I've got those resolvers mixed up a bit (I was answering this while on a train from my phone, so that at least somewhat explains my inadequacy -- again, sorry about that)

But it still seems like you're running Atom via stack exec, or at least something else sets those environment variables. Not entirely sure what that might be. Definitely not ide-haskell packages -- at the moment it's only responsible for a bit of PATH manipulation.

I guess check your .bashrc/.bash_profile (or .zsh* counterparts, or whatever shell you're using)? Not sure at this point. At the very least, try to unset GHC_PACKAGE_PATH and run Atom from that environment?

P.S. Note that Atom runs ghc-mod directly, not via stack exec, so that's why stack exec -- ghc-mod ... seemingly works. But that is unsupported upstream and can break eventually.

P.P.S. Also just occurred to me, that you might be running your window manager with stack exec? I mean, you might be using XMonad or something like that, I guess... Graphical applications will usually inherit window manager's environment, so that might be it?.. This is pure speculation at this point, btw, just throwin' some ideas out there, that might help you diagnose. Sorry I can't be of more help.

[ruhatch]

XMonad!!! facepalm That's some nifty speculation!

I like having XMonad running through stack, but it's not worth the loss of ghc-mod. Do you know of any way to isolate Atom from the changes?

Thank you so much for that revelation, that could have taken a long time to track down!

[ruhatch]

I just saw this which should make things better! Do you know if it will be merged soon?

[lierdakil]

I was just about to link you there. I have no idea when or if it will be merged/released, however. +1 it or something while you're at it I guess?

For now, I guess you could unset GHC_PACKAGE_PATH in XMonad's startupHook. That will likely break xmonad --recompile though, so you'll have to change that to stack exec -- xmonad --recompile && stack exec -- xmonad --restart (I don't remember exact command line used in default M-q binding, so I might be a little bit off with this suggestion -- use your own judgement).

If you just want this to work with Atom, there's an easier route. You can add the following to your Atom init script (Edit → Init Script...):

delete process.env.GHC_PACKAGE_PATH

This will just unconditionally unset GHC_PACKAGE_PATH when Atom starts (only for Atom of course). Although there's a caveat that non-stack projects won't work anymore, but that's not an issue for you as far as I can tell?

[ruhatch]

Thanks again - I am using delete for now, but I'll keep an eye on the build scripts.