Specify the precise nixpkgs commit

[mpickering]

Running nix-shell in code/exercises fails building one of the packages for my version of nixpkgs.

Could you perhaps make the build truly reproducible by also specifying which nixpkgs commit should be used (as in https://garbas.si/2015/reproducible-development-environments.html)?

[dalaing]

I'm already using a pinned version of reflex-platform, which contains a pinned version of nixpkgs.

It looks like it was some sloppy work on my part with the shell.nix, which I believe I have fixed now - could you try again and see how you go?

[gspia]

I did git pull a few minutes ago and can confirm that this works

[mpickering]

Still fails for me.

Preprocessing library directory-tree-0.12.1...
[1 of 1] Compiling System.Directory.Tree ( System/Directory/Tree.hs, dist/build/System/Directory/Tree.o )
Preprocessing test suite 'test' for directory-tree-0.12.1...
[1 of 2] Compiling System.Directory.Tree ( System/Directory/Tree.hs, dist/build/test/test-tmp/System/Directory/Tree.dyn_o )
[2 of 2] Compiling Main             ( Test.hs, dist/build/test/test-tmp/Main.dyn_o )

Test.hs:7:1: warning: [-Wunused-imports]
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
Linking dist/build/test/test ...
running tests
Running 1 test suites...
Test suite test: RUNNING...
-- The following tests will either fail with an error 
-- message or with an 'undefined' error
OK
test: writeDirectory returned a tree that didn't match
CallStack (from HasCallStack):
  error, called at Test.hs:33:13 in main:Main
Test suite test: FAIL
Test suite logged to: dist/test/directory-tree-0.12.1-test.log
0 of 1 test suites (0 of 1 test cases) passed.
builder for ‘/nix/store/37fzj0i8sq1kv95dqwdgldawn9njwza3-directory-tree-0.12.1.drv’ failed with exit code 1

with commit 02342587ec3462d5038feb44ac1d8360d59fd902

[mpickering]

The reflex nixpkgs appears to be pinned at 32833c010ecf868826aaa3b60d322bf697f37134

Looking in hackage-packages.nix (https://raw.githubusercontent.com/obsidiansystems/nixpkgs/32833c010ecf868826aaa3b60d322bf697f37134/pkgs/development/haskell-modules/hackage-packages.nix) I observe that the broken version of directory-tree is present at this commit -

  "directory-tree" = callPackage
    ({ mkDerivation, base, directory, filepath, process }:
     mkDerivation {
       pname = "directory-tree";
       version = "0.12.1";
       sha256 = "05z5ws58kky3wcwwwv6q16m9vs0lzj35qqs9v5acy9m2nfal8272";
       libraryHaskellDepends = [ base directory filepath ];
       testHaskellDepends = [ base directory filepath process ];
       homepage = "http://brandon.si/code/directory-tree-module-released/";
       description = "A simple directory-like tree datatype, with useful IO functions";
       license = stdenv.lib.licenses.bsd3;
       hydraPlatforms = stdenv.lib.platforms.none;
     }) {};

So I don't understand why I am the only one who has this package in the build plan as it is surely broken for everyone.

[mpickering]

The dependency on directory-tree comes from wai-middleware-static via depending on hpc-coveralls in the test suite.

[dalaing]

What operating system are you using?

[mpickering]

nixos

[dalaing]

Do you have the reflex binary caches set up?

I'm just trying to think of anything that might be obviously different between our systems.

[mpickering]

I have "https://nixcache.reflex-frp.org" if that is the one you mean.

[mpickering]

In your nix-shell environment is directory-tree installed?

[dalaing]

It's not turning up in ghc-pkg list, and neither is hpc-coveralls, although wai-middleware-static is in there.

Could it be something different in out ~/.cabal/config files?

[dalaing]

It's working for me with nix-shell --pure, although I don't know if that will help.

[mpickering]

I overrode wai-middleware-static and now everything works. I don't know what is causing you to not run the checkPhase by default. Is mkDerivation overloaded anywhere?

{ reflex-platform ? import ./reflex-platform.nix
, compiler   ? "ghcjs"
} :
let
  pkgs = reflex-platform.nixpkgs.pkgs;

  haskellPackages = reflex-platform.${compiler}.override {
                      overrides = self: super: {
                        wai-middleware-static = pkgs.haskell.lib.dontCheck (super.wai-middleware-static); }; };

  adjust-for-ghcjs = drv: {
    executableToolDepends = [pkgs.closurecompiler pkgs.zopfli];
    doHaddock = false;
    doCheck = false;
    postInstall = ''
      mkdir -p $out
      mkdir -p $out/css/reflex/basics-exercises
      cp -r ./css/exercises $out/css/reflex/basics-exercises/
      cp -r ./css/solutions $out/css/reflex/basics-exercises/
      mkdir -p $out/js/reflex/basics-exercises
      cp $out/bin/solutions.jsexe/all.js $out/js/reflex/basics-exercises/solutions.js
      cd $out/bin/solutions.jsexe
      closure-compiler all.js --compilation_level=ADVANCED_OPTIMIZATIONS --isolation_mode=IIFE --assume_function_wrapper --jscomp_off="*" --externs=all.js.externs > $out/js/reflex/basics-exercises/solutions.min.js
      rm -Rf $out/bin/solutions.jsexe
      rm -Rf $out/bin
      cd $out/js/reflex/basics-exercises
      zopfli -i1000 solutions.min.js
      rm -Rf $out/lib
      rm -Rf $out/nix-support
      rm -Rf $out/share
    '';
  };

  adjust = drv:
    if compiler == "ghcjs"
    then adjust-for-ghcjs drv
    else adjust-for-ghcjs drv;

  exercises = pkgs.haskell.lib.overrideCabal (haskellPackages.callPackage ./exercises.nix {}) adjust;
in
exercises

[dalaing]

That last line of adjust is pretty different from mine, which may or may not have been having an effect.

If that's not it, I'm happy to fold that override so that the exercises work for as many folks as possible.

[dalaing]

... and to answer your question, I don't have mkDerivation overriden anywhere - my setup is pretty simple.

[mpickering]

diff --git a/code/exercises/default.nix b/code/exercises/default.nix
diff --git a/code/exercises/default.nix b/code/exercises/default.nix
index 45b83b6..c6681be 100644
--- a/code/exercises/default.nix
+++ b/code/exercises/default.nix
@@ -4,6 +4,10 @@
 let
   pkgs = reflex-platform.nixpkgs.pkgs;

+  haskellPackages = reflex-platform.${compiler}.override {
+                      overrides = self: super: {
+                        wai-middleware-static = pkgs.haskell.lib.dontCheck (super.wai-middleware-static); }; };
+
   adjust-for-ghcjs = drv: {
     executableToolDepends = [pkgs.closurecompiler pkgs.zopfli];
     doHaddock = false;
@@ -36,6 +40,6 @@ let
     then adjust-for-ghcjs drv
     else drv;

-  exercises = pkgs.haskell.lib.overrideCabal (reflex-platform.${compiler}.callPackage ./exercises.nix {}) adjust;
+  exercises = pkgs.haskell.lib.overrideCabal (haskellPackages.callPackage ./exercises.nix {}) adjust;
 in
   exercises

These are the necessary changes, the other changes are irrelevant and just from experimenting.

[dalaing]

Cool - I've folded that in, thanks for persevering with this!

[dalaing]

Is this working for you now?

[mpickering]

nix-shell loads but #4 has resurfaced.

[mpickering]

Ah very frustratingly I had text-1.2.2.2 installed globally. I removed the relevant .ghc folder and now it works.