doesn't run on nixos

[avh4]

elm-test appears to rely on native binaries which don't work on nixos (and potentially other platforms).

$ mkdir tmp
$ cd tmp
$ npm install elm-test
$ ./node_modules/.bin/elm-test init
$ ./node_modules/.bin/elm-test
Success! Compiled 54 modules.                                       
Successfully generated /dev/null
Unhandled exception while running the tests: { Error: spawn /home/avh4/workspace/Apps/tmp/node_modules/elm-test/bin/elm-interface-to-json ENOENT
    at _errnoException (util.js:1024:11)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:190:19)
    at onErrorNT (internal/child_process.js:372:16)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
  code: 'ENOENT',
  errno: 'ENOENT',
  syscall: 'spawn /home/avh4/workspace/Apps/tmp/node_modules/elm-test/bin/elm-interface-to-json',
  path: '/home/avh4/workspace/Apps/tmp/node_modules/elm-test/bin/elm-interface-to-json',
  spawnargs: [ '--path', '/home/avh4/workspace/Apps/tmp/tests' ] }

[avh4]

Additional info:

$ /home/avh4/workspace/Apps/tmp/node_modules/elm-test/bin/elm-interface-to-json
bash: /home/avh4/workspace/Apps/tmp/node_modules/elm-test/bin/elm-interface-to-json: No such file or directory

$ file /home/avh4/workspace/Apps/tmp/node_modules/elm-test/bin/elm-interface-to-json 
/home/avh4/workspace/Apps/tmp/node_modules/elm-test/bin/elm-interface-to-json: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=c80df72c1b55ad3d2ac0ca5d4a6fef751c2bbe0b, stripped

$ ldd /home/avh4/workspace/Apps/tmp/node_modules/elm-test/bin/elm-interface-to-json
        linux-vdso.so.1 (0x00007fff48738000)
        librt.so.1 => /nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49/lib/librt.so.1 (0x00007f5c1f7c0000)
        libutil.so.1 => /nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49/lib/libutil.so.1 (0x00007f5c1f5bd000)
        libdl.so.2 => /nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49/lib/libdl.so.2 (0x00007f5c1f3b9000)
        libgmp.so.10 => not found
        libm.so.6 => /nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49/lib/libm.so.6 (0x00007f5c1f0a6000)
        libpthread.so.0 => /nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49/lib/libpthread.so.0 (0x00007f5c1ee88000)
        libgcc_s.so.1 => /nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49/lib/libgcc_s.so.1 (0x00007f5c1ec72000)
        libc.so.6 => /nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49/lib/libc.so.6 (0x00007f5c1e8d3000)
        /lib64/ld-linux-x86-64.so.2 => /nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49/lib64/ld-linux-x86-64.so.2 (0x00007f5c1f9c8000)

[kamilchm]

It would be helpful if we know how to build elm-interface-to-json and node-test-runner all from sources. It can be then properly ported to nix. @rtfeldman is there a way to install node-test-runner without hitting bintray?

[manveru]

I ran elm-interface-to-json for some time with patchelf, but today tried to make a package for it. Unfortunately my Haskell-fu is weak, and it failed to build with a strange error:

building path(s) '/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0'
building '/nix/store/fn097xbm0wyk41wfz76yrc8cr3fvqc7j-elm-interface-to-json-0.1.0.0.drv'...
setupCompilerEnvironmentPhase
Build with /nix/store/d7pa43zz28cxmmbnk8aw1fsdlwhizcmm-ghc-8.0.2.
unpacking sources
unpacking source archive /nix/store/fgv17ngf3rdfahm0y24zxbbh3f1jpy10-elm-interface-to-json-9884c1c
source root is elm-interface-to-json-9884c1c
patching sources
Run jailbreak-cabal to lift version restrictions on build inputs.
compileBuildDriverPhase
setupCompileFlags: -package-db=/build/package.conf.d -j8 -threaded
[1 of 1] Compiling Main             ( Setup.hs, /build/Main.o )
Linking Setup ...
direnv: ([direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
direnv: ([/nix/store/yqiw3ai6fg7v5l6pfcn31nyl89qil10a-direnv-2.13.1-bin/bin/direnv apply_dump /dev/fd/63]) is taking a while to execute. Use CTRL-C to give up.
configuring
configureFlags: --verbose --prefix=/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0 --libdir=$prefix/lib/$compiler --libsubdir=$pkgid --with-gcc=gcc --package-db=/build/package.conf.d --ghc-option=-optl=-Wl,-rpath=/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0/lib/ghc-8.0.2/elm-interface-to-json-0.1.0.0 --ghc-option=-j8 --disable-split-objs --disable-library-profiling --disable-profiling --enable-shared --disable-coverage --enable-library-vanilla --enable-executable-dynamic --enable-tests --ghc-option=-split-sections
Configuring elm-interface-to-json-0.1.0.0...
Dependency aeson -any: using aeson-1.1.2.0
Dependency aeson-pretty -any: using aeson-pretty-0.8.5
Dependency base -any: using base-4.9.1.0
Dependency binary -any: using binary-0.8.3.0
Dependency bytestring -any: using bytestring-0.10.8.1
Dependency concatenative -any: using concatenative-1.0.1
Dependency containers -any: using containers-0.5.7.1
Dependency directory -any: using directory-1.3.0.0
Dependency either -any: using either-4.4.1.1
Dependency elm-compiler -any: using elm-compiler-0.18
Dependency filemanip -any: using filemanip-0.3.6.3
Dependency filepath -any: using filepath-1.4.1.1
Dependency indents -any: using indents-0.3.3
Dependency optparse-applicative -any: using optparse-applicative-0.13.2.0
Dependency parsec -any: using parsec-3.1.11
Dependency text -any: using text-1.2.2.2
Dependency transformers -any: using transformers-0.5.2.0
Warning: This package indirectly depends on multiple versions of the same
package. This is highly likely to cause a compile failure.
package elm-compiler-0.18 requires aeson-pretty-0.7.2
package elm-interface-to-json-0.1.0.0 requires aeson-pretty-0.8.5
Using Cabal-1.24.2.0 compiled by ghc-8.0
Using compiler: ghc-8.0.2
Using install prefix:
/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0
Binaries installed in:
/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0/bin
Libraries installed in:
/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0/lib/ghc-8.0.2/elm-interface-to-json-0.1.0.0
Dynamic libraries installed in:
/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0/lib/ghc-8.0.2/x86_64-linux-ghc-8.0.2
Private binaries installed in:
/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0/libexec
Data files installed in:
/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0/share/x86_64-linux-ghc-8.0.2/elm-interface-to-json-0.1.0.0
Documentation installed in:
/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0/share/doc/x86_64-linux-ghc-8.0.2/elm-interface-to-json-0.1.0.0
Configuration files installed in:
/nix/store/1ksfp9ac73f24l633qy1q011piyxd1f9-elm-interface-to-json-0.1.0.0/etc
No alex found
Using ar found on system at:
/nix/store/yha4q7zgm6y0ci0b57a8hbmzwyvv1w44-binutils-2.28.1/bin/ar
No c2hs found
No cpphs found
Using gcc version 6.4.0 given by user at:
/nix/store/yjh21h38fmw3i4h2b5r557p4ggbvrd0v-gcc-wrapper-6.4.0/bin/gcc
Using ghc version 8.0.2 found on system at:
/nix/store/d7pa43zz28cxmmbnk8aw1fsdlwhizcmm-ghc-8.0.2/bin/ghc
Using ghc-pkg version 8.0.2 found on system at:
/nix/store/d7pa43zz28cxmmbnk8aw1fsdlwhizcmm-ghc-8.0.2/bin/ghc-pkg
No ghcjs found
No ghcjs-pkg found
No greencard found
Using haddock version 2.17.3 found on system at:
/nix/store/d7pa43zz28cxmmbnk8aw1fsdlwhizcmm-ghc-8.0.2/bin/haddock
No happy found
Using haskell-suite found on system at: haskell-suite-dummy-location
Using haskell-suite-pkg found on system at: haskell-suite-pkg-dummy-location
No hmake found
Using hpc version 0.67 found on system at:
/nix/store/d7pa43zz28cxmmbnk8aw1fsdlwhizcmm-ghc-8.0.2/bin/hpc
Using hsc2hs version 0.68.1 found on system at:
/nix/store/d7pa43zz28cxmmbnk8aw1fsdlwhizcmm-ghc-8.0.2/bin/hsc2hs
No hscolour found
No jhc found
Using ld found on system at:
/nix/store/yjh21h38fmw3i4h2b5r557p4ggbvrd0v-gcc-wrapper-6.4.0/bin/ld
No lhc found
No lhc-pkg found
No pkg-config found
Using strip version 2.28 found on system at:
/nix/store/yha4q7zgm6y0ci0b57a8hbmzwyvv1w44-binutils-2.28.1/bin/strip
Using tar found on system at:
/nix/store/7lx6bha6l22vi5h9c31nri1p7rqs7v1d-gnutar-1.29/bin/tar
No uhc found
*** abort because of serious configure-time warning from Cabal
builder for '/nix/store/fn097xbm0wyk41wfz76yrc8cr3fvqc7j-elm-interface-to-json-0.1.0.0.drv' failed with exit code 1

Maybe someone else can make sense of this error, I have no clue what's going on.

In case you're curious about patchelf, that's how i did it:

let
  my-nixpkgs = import <nixpkgs> {};
  my-glibc = my-nixpkgs.pkgs.glibc;
  my-gcc = my-nixpkgs.gcc-unwrapped.lib;
  my-gmp = my-nixpkgs.gmp;
in stdenv.mkDerivation {
  name = "elm-test-shell";
  buildInputs = with elmPackages; [
    elm
    elm-compiler
    elm-make
    elm-package
    elm-reactor
    elm-repl
  ];

  shellHook = ''
    export PATH="$PATH:${toString ./node_modules/.bin }"

    patchelf --print-interpreter node_modules/elm-test/bin/elm-interface-to-json | grep -q nix || \
      patchelf \
        --debug \
        --set-interpreter ${my-glibc}/lib/ld-linux-x86-64.so.2 \
        --set-rpath "${concatStringsSep ":" (map (l: "${l}/lib") [my-gcc my-gmp my-glibc])}" \
        node_modules/elm-test/bin/elm-interface-to-json
  '';
}

[avh4]

Can you post your attempted elm-interface-to-json nix file? My guess from the errors is that the nix file isn't specifying all the haskell dependencies that interface-to-json needs to build.

[manveru]

@avh4 Yeah, it's just what cabal2nix gave me, I made a quick PR to nixpkgs to show you: https://github.com/NixOS/nixpkgs/pull/32194/files

[manveru]

@avh4 after taking another look at it, I figured it out. Updated the PR with a version that should build. I'm testing it right now.

[manveru]

It's merged into nixpkgs now, so should be available in nixpkgs-unstable shortly and then you can simply use elmPackages.elm-interface-to-json in your buildInputs 🎆

[helinko]

What's the status of this? I'm running NixOS unstable and still getting the error

[turboMaCk]

@manveru expression for elm-interface-to-json is working fine for me but I can't find it in nixpkgs repo for some reason. Was it removed?

Anyway, this issue seems to be really tricky. I'm trying to make this work together with node2nix and frankly, I had to make a lot of dirty workarounds in order to run it. I still don't know how can I make reproducible definitions for a project where I'm testing it. I ended up replacing symlink created during installation to make it work. Does anyone have some success story with this in the end? Is there any public project when I can look for inspiration?

[manveru]

Yes, it was removed with the upgrade to Elm 0.19. I'm still using 0.18 myself due to that (and things like the WebSockets package not having been ported when I tried it).

[turboMaCk]

sorry for hijacking this thread but what is your strategy even with 0.18 @manveru? are you using node2nix to generate elm-test dependencies with buildInputs (I'm doing this in the project itself and it doesn't work but probably it can be fixed by manually overwriting symlink in node_modules/elm-test/bin) or something else? I'm kind of trying to figure out some good workflow on nixos with elm so maybe I'm doing some dump things still:/

[manveru]

I don't use npm for it at all, since it's a Haskell library and only the binary happens to be distributed via npm, which of course means you'd have to patchelf it into submission. Here's my old derivation for it:

{ mkDerivation, aeson, aeson-pretty, base, binary, bytestring
, concatenative, containers, directory, either, elm-compiler
, filemanip, filepath, indents, optparse-applicative, parsec
, stdenv, text, transformers, fetchgit
}:
mkDerivation {
  pname = "elm-interface-to-json";
  version = "0.1.0.0";
  src = fetchgit {
    url = "https://github.com/stoeffel/elm-interface-to-json";
    sha256 = "1izc78w91m7nrc9i2b3lgy3kyjsy4d5mkkblx96ws0bp3dpm5f9k";
    rev = "9884c1c997a55f11cf7c3d99a8afa72cf2e97323";
  };
  isLibrary = false;
  isExecutable = true;
  jailbreak = true;
  executableHaskellDepends = [
    aeson aeson-pretty base binary bytestring concatenative containers
    directory either elm-compiler filemanip filepath indents
    optparse-applicative parsec text transformers
    alex c2hs cpphs happy hscolour pkgconfig uhc
  ];
  homepage = "https://github.com/githubuser/elm-interface-to-json#readme";
  license = stdenv.lib.licenses.bsd3;
}

[turboMaCk]

I've got this working. Problem is with elm-test and how to make it use this binary and not the one it pulls to its local bin. I guess I can patch it by symlink even though it seems hacky. Anyway, I don't want to pollute this issue with my specific question or waste your time. I guess I must just figure it out :/ Thanks for your reply. If you have time and want to help me you can ping me on elm slack (turbo_mack)

[bburdette]

If anyone would like to post a sequence of commands that would result in a working elm-test on nixos 18.09, that would be greeat. I installed into ~/.npm-packages but I have the bad elmi-to-json problem.

[mdevlamynck]

There is also https://github.com/NixOS/nixpkgs/issues/45280. I'm not good enough with nix yet to try to fix this, @turboMaCk @manveru do you think you can help with this?

[bburdette]

So, sources on slack tell me that we can expect an elm-test package very soon! In the meantime I was able to get elm-test to work by:

0) getting npm and stack/cabal/ghc installed on nixos. 1) creating a file ~/.npmrc, containing: prefix = ${HOME}/.npm-packages 2) install elm-test with npm, pretty sure the command was npm install -g elm-test@beta 3) clone github.com/stoeffel/elmi-to-json locally (NOT elm-interface-to-json which is for elm 0.18) 4) build it with stack build 5) copy the resulting elmi-to-json executable into where it needs to be, which for me was:

 cp ~/op-code/elmi-to-json/.stack-work/install/x86_64-linux-nix/lts-11.0/8.2.2/bin/elmi-to-json /home/bburdette/.npm-packages/lib/node_modules/elm-test/node_modules/elmi-to-json/unpacked_bin/

[mdevlamynck]

Wouldn't it make more sense to continue this on the nixpkgs issue https://github.com/NixOS/nixpkgs/issues/45280? I'll post there my progress on creating a package there.

[knuton]

Can I suggest adding an --elmi option for specifying the path to elm-interface-to-json/elmi-to-json? There is already a --compiler option.

This makes it easy for people on unusual platforms to bypass the bundled binary, without messing with node_modules post-install.

[turboMaCk]

Sorry I haven't posted this earlier here (I couldn't find the issue).

Please use this instead of NPM install https://github.com/turboMaCk/nix-elm-tools

I'll be working together with this and other upstreams including nixpkgs to find the best way for all the projects.

[harrysarson]

@turboMaCk as elm-test now works on nixos using your tooling, can this issue be closed?

[turboMaCk]

@harrysarson I've packaged elm-test in nixpkgs nix-env -i elmPackages.elm-test so I think this can be closed. I don't think npm installation will work but NixOS user will probably don't want to use it anyway.

[harrysarson]

Brilliant, thanks @turboMaCk