Unpredictable builds with ghcjs

[tomsmalley]

I'm getting very temperamental builds with the master branch, this wasn't happening in stack 1.4.0 (hence reporting here). The first build seems to work perfectly, but as soon as there is any change in the stack.yaml/snapshot file (e.g. removing an empty line) I'm getting errors like:

$ /Users/Tom/Projects/stack/.stack-work/install/x86_64-osx/lts-8.5/8.0.2/bin/stack build
Ignoring that the GHCJS boot package "aeson" has a different version, 1.1.1.0, than the resolver's wanted version, 1.0.2.1
blaze-builder-0.4.0.2: using precompiled package
cabal-doctest-1: using precompiled package
cryptohash-md5-0.11.100.1: using precompiled package
cryptohash-sha1-0.11.100.1: using precompiled package
Progress: 4/81Running /Users/Tom/.stack/programs/x86_64-osx/ghcjs-0.2.1.9008011_ghc-8.0.2/bin/ghcjs-pkg --user --no-user-package-db --package-db /Users/Tom/.stack/snapshots/x86_64-osx/lts-8.11/ghcjs-0.2.1.9008011_ghc-8.0.2/pkgdb describe --simple-output cryptohash-sha1 --expand-pkgroot exited with ExitFailure 1

ghcjs-pkg-0.2.1.9008011-8.0.2: cannot find package cryptohash-sha1

Running /Users/Tom/.stack/programs/x86_64-osx/ghcjs-0.2.1.9008011_ghc-8.0.2/bin/ghcjs-pkg --user --no-user-package-db --package-db /Users/Tom/.stack/snapshots/x86_64-osx/lts-8.11/ghcjs-0.2.1.9008011_ghc-8.0.2/pkgdb describe --simple-output cryptohash-md5 --expand-pkgroot exited with ExitFailure 1

ghcjs-pkg-0.2.1.9008011-8.0.2: cannot find package cryptohash-md5

Running /Users/Tom/.stack/programs/x86_64-osx/ghcjs-0.2.1.9008011_ghc-8.0.2/bin/ghcjs-pkg --user --no-user-package-db --package-db /Users/Tom/.stack/snapshots/x86_64-osx/lts-8.11/ghcjs-0.2.1.9008011_ghc-8.0.2/pkgdb describe --simple-output cabal-doctest --expand-pkgroot exited with ExitFailure 1

ghcjs-pkg-0.2.1.9008011-8.0.2: cannot find package cabal-doctest

Running /Users/Tom/.stack/programs/x86_64-osx/ghcjs-0.2.1.9008011_ghc-8.0.2/bin/ghcjs-pkg --user --no-user-package-db --package-db /Users/Tom/.stack/snapshots/x86_64-osx/lts-8.11/ghcjs-0.2.1.9008011_ghc-8.0.2/pkgdb describe --simple-output blaze-builder --expand-pkgroot exited with ExitFailure 1

ghcjs-pkg-0.2.1.9008011-8.0.2: cannot find package blaze-builder

This happens both with custom snapshots and the normal resolvers. stack clean --full doesn't help, I've only found completely cleaning ~/.stack (hence having to reboot ghcjs, taking around 2 hours) allows a build to work.

stack.yaml:

resolver: ../snapshot.yaml

compiler: ghcjs-0.2.1.9008011_ghc-8.0.2
compiler-check: match-exact
setup-info:
  ghcjs:
    source:
      ghcjs-0.2.1.9008011_ghc-8.0.2:
        url: https://github.com/matchwood/ghcjs-stack-dist/raw/master/ghcjs-0.2.1.9008011.tar.gz
        sha1: a72a5181124baf64bcd0e68a8726e65914473b3b

packages:
- .
- ../common

snapshot.yaml:

resolver: lts-8.11
name: snapshot
packages:

- prim-uniq-0.1.0.1
- ref-tf-0.4.0.1
- jsaddle-wkwebview-0.9.0.0
- dependent-sum-template-0.0.0.6
- dependent-map-0.2.4.0
- constraints-0.9.1
- jsaddle-0.9.2.1
- zenc-0.1.1

- eventful-core-0.1.3
- eventful-memory-0.1.3
- eventful-sqlite-0.1.3
- eventful-sql-common-0.1.3
- sum-type-boilerplate-0.1.0

- websockets-0.11.2.0

- git: https://github.com/reflex-frp/reflex
  commit: 50305c797c41a27660b74f543e204e902c086bbf
- git: https://github.com/reflex-frp/reflex-dom
  commit: a20e829214e5330a2d829bcf156046b1fe24a169
  subdirs:
  - reflex-dom
  - reflex-dom-core
- git: https://github.com/reflex-frp/reflex-dom-semui.git
  commit: d612750d37e57aeb89dc0dca5dd76826402e84d4
- git: https://github.com/ghcjs/jsaddle-dom
  commit: 80e4fdd381d57c6fd643815fcccef51b3f420e05
- git: https://github.com/ghcjs/ghcjs-dom
  commit: 6ee9b52b36245a0f42e3ac8ed2c73d15108ae990
  subdirs:
  - ghcjs-dom
  - ghcjs-dom-jsaddle
  - ghcjs-dom-jsffi

Note: The build I am using is actually from 20dc5ef04df20171e8f7b21303bed4e00873223f (relative path snapshots) but this also happens on 455c79fb7ba2c8880f284091f164decdd93e825b when using absolute snapshot path (or no custom snapshot).

[mgsloan]

Thanks for the report!

My guess is that something broke due to the changes to add implicit custom snapshots (more package sharing). Since GHC and GHCJS work very similarly, I suspected that it was just invoking ghc-pkg instead of using ghcjs-pkg, or using GHC_PACKAGE_PATH instead of GHCJS_PACKAGE_PATH.

Indeed, searching GHC_PACKAGE_PATH revealed a spot that the implicit snapshots change added a use of it without checking which compiler you're using. I've pushed a fix to that to master. Can you please check if this resolves the issue?

[mgsloan]

Pinging @tomsmalley , does updating to the master version of stack resolve this?

[tomsmalley]

Looks to be working, thanks!