Static binary release for 0.7.1 possible?

[webframp]

Hi! 👋

I'm not very familiar with the current state of haskell builds with stack, but was curious if you think it's possible to get a static 64bit linux binary on the releases?

Or maybe you know pointers/current blockers for doing this myself? Thanks!

[duijf]

I think it can be done, but might be cumbersome if this blog post is anything to go by: https://vadosware.io/post/static-binaries-for-haskell-a-convoluted-approach/

I don't think there would be blockers specifically with vaultenv, but I might be really wrong about that.

What is your usecase for static binaries? If it's common enough, we can maybe help

[duijf]

(Just to be sure we're talking about the same thing: if you just want a binary, you can build one with the command stack build, the only thing that these link dynamically is libgmp and libc)

[webframp]

My reason for a static binary is using vaultenv in docker containers. For the smallest possible container I'd like to be able to create a layer using FROM scratch or at least alpine linux, instead of having the external lib dependencies and a larger base container image.

[duijf]

Alpine isn't officially supported by Stack yet https://github.com/commercialhaskell/stack/issues/2387, but reportedly people have gotten this to work for GHC 8.0.2. Vaultenv uses GHC 8.2 through stack, but it still compiles successfully if you change the resolver to lts-9.20 in stack.yaml.

The foundation Haskell library builds against musl: https://github.com/haskell-foundation/foundation/issues/393 so building Haskell on Alpine against musl can be done. Don't know about vaultenv, though, but it's worth a shot

Some instructions that I found on this:

• https://www.reddit.com/r/haskell/comments/5lk33p/struggling_building_a_static_binary_for_aws/
• https://gist.github.com/erebe/9611b2c8f1d26cc91cc3fb33bbae0eb9

[webframp]

I'm still experimenting, here's another gist found during some research:

• https://gist.github.com/jkachmar/4828bfe0f585bec93878ea893c3373ee

[naushadh]

@webframp check out https://github.com/naushadh/hello-world

[webframp]

Thanks for the tip @naushadh

I experimented but the resolver is different between the hello-world example and vaultenv, no success yet but I'll keep at it as my time allows

[naushadh]

@webframp perhaps try bumping down the vaultenv resolver to match what's in hello-world (9.21)? @duijf already appears to have had success with resolver 9.2.

Once https://github.com/alpinelinux/aports/pull/4255 gets merged, newer resolver should be possible in the alpine docker setup.

[ruuda]

I was able to build a static binary like so (based on this blog post):

$ cabal2nix --shell . > default.nix

Then insert the following lines:

  enableSharedExecutables = false;
  enableSharedLibraries = false;
  configureFlags = [
    "--ghc-option=-optl=-static"
    "--ghc-option=-optl=-pthread"
    "--ghc-option=-optl=-L${pkgs.gmp6.override { withStatic = true; }}/lib"
    "--ghc-option=-optl=-L${pkgs.zlib.static}/lib"
    "--ghc-option=-optl=-L${pkgs.glibc.static}/lib"
  ];

Remove the qualified Options.Applicative.Builder.Internal as OptParse import from src/Config.hs, it is unused and causes a -Werror failure. Then build the binary:

$ nix build
$ file result/bin/vaultenv
result/bin/vaultenv: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, stripped

[arianvp]

Statically linking libgmp I would advice against. It means you need to relicense vaultenv under the LGPL license.

However you could use 'integer-simple' instead of libgmp for Integers.

[ruuda]

Good point @arianvp.

[nh2]

Issue title:

Static binary release for 0.7.1 possible?

@webframp Do you mean 1.7.1?

[nh2]

@webframp Do you mean 1.7.1?

Ah sorry, you were talking about valutenv's version number, I thought you meant stack's.

[nh2]

I was able to build a static binary like so (based on this blog post):

@ruuda I tuned the idea of that blog post further, so that you can link fully statically with musl instead of glibc (glibc doesn't really support fully static linking):

https://github.com/nh2/static-haskell-nix

It can now build executables with many dependencies, such as stack (there's an example in the repo). You may be able to adapt this approach to build whatever you would like fully statically.

[agix]

According to the previous link and poking around on the internet, I successfully built a statically linked vaultenv following this simple process

git clone https://github.com/channable/vaultenv

edit package.yaml to add ld-options: -static in executables

executables:
  vaultenv:
    main: Main.hs
    source-dirs: app
    ld-options: -static
    dependencies:
      - vaultenv

docker run --rm -v $(pwd)/vaultenv:/app -it fpco/stack-build:lts-12.9 /bin/bash

with lts-12.9 equals to the one in stack.yaml

In the docker, go to /app and run the stack install

stack clean; stack install --split-objs --ghc-options="-fPIC -fllvm"

root@47eb6446173d:~# file /root/.local/bin/vaultenv 
/root/.local/bin/vaultenv: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=30bd59a4f83818d7ac1a26c08a09e043a2a4e9cd, stripped

vaultenv.zip

[nh2]

@agix Did you ensure that you don't link statically against glibc though?

glibc is the default libc on most Linux distributions, so if you just add -static flags, you are likely to use that, which creates a high probability that your program will crash at run time, like in this example.

The easiest way to get static executables is to add the project to Stackage because static-haskell-nix can already create statically linked executables out of the box for most programs on Stackage.

[agix]

Hi, yes I read some function may not work properly. At least it does the job for me and I can now use vaultenv in my docker images... Waiting for a better method.

[ruuda]

I was following along the developments in https://github.com/NixOS/nixpkgs/issues/43795; I think we should give static-haskell-nix a try, and if it works, make it the default at least for the releases we upload here. It would solve compatibility issues like this one.

[duijf]

I've started work on fully static binaries with Nix and have a working PoC in the nix-build branch.

The binaries currently aren't fit for distribution, as they statically against libgmp, which is a LGPL software. I hope to get a working build with integer-simple as well

[nh2]

@duijf Regarding https://github.com/channable/vaultenv/commit/1cce569f4bd8c1ede673fc4a4076507192ce31ea, did you see that you can also easily static-nixify stack projects, see https://github.com/nh2/static-haskell-nix/tree/master/static-stack2nix-builder-example ?

[ruuda]

@nh2, I tried that in 073511179c171864b2b589c85ca8e989733e86d9, but I ran into the following error:

...
+ cabal2nix --subpath . --system x86_64-linux --compiler ghc-8.4.4 cabal://zstd-0.1.0.0
+ cabal2nix --subpath . --system x86_64-linux --compiler ghc-8.4.4 cabal://zot-0.0.3
+ cabal2nix --subpath . --system x86_64-linux --compiler ghc-8.4.4 cabal://ztail-1.2.0.2
stack2nix: /tmp/stack2nix-output-dir.yfkiFc11m8/stack2nix-output.nix: commitBuffer: invalid argument (invalid character)

I can reproduce it reliably, but I haven’t looked more closely into what is going on.

[nh2]

@ruuda Ah, another instance of the horrible "Handle encoding guessing" problem. Try the suggestion in https://github.com/input-output-hk/stack2nix/issues/150#issuecomment-443221773, LANG=en_US.utf8, as a workaround.

Overall I expect that using stack2nix will be a much better working experience for most projects than pinning your version of packages to what nixpkgs has; so I think this is worth working out.

[nh2]

@ruuda Oh and of course please tell me what echo $LANG is for you normally, so I can see if I can reproduce it.

[ruuda]

I use LANG=en_GB.utf8. Setting LANG=en_US.utf8 when running ...-stack2nix-and-build-script.sh does not help, nor does adding export LANG=en_US.utf8 to that generated script. I also tried adding it deeper, just before the stack2nix call here, but that did not help either.

[duijf]

Overall I expect that using stack2nix will be a much better working experience for most projects than pinning your version of packages to what nixpkgs has; so I think this is worth working out.

I don't have much experience with either approach. What are the pros and cons in your opinion?

LANG=en_US.utf8, as a workaround.

I have LANG=en_US.utf8 set.

~/repos/channable/vaultenv static
❯ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

~/repos/channable/vaultenv static
❯ git rev-parse HEAD
073511179c171864b2b589c85ca8e989733e86d9

~/repos/channable/vaultenv static
❯ nix-build --no-link -A fullBuildScript                                   
/nix/store/iwkf1hyr8iqcskfb10d54pagvpkiisxv-stack2nix-and-build-script.sh

~/repos/channable/vaultenv static
❯ /nix/store/iwkf1hyr8iqcskfb10d54pagvpkiisxv-stack2nix-and-build-script.sh
+ /nix/store/0acd2zbljgc38f50xwfhhsc5fb6l5xmg-stack2nix-0.2.2/bin/stack2nix /home/duijf/repos/channable/vaultenv --stack-yaml stack.yaml --hackage-snapshot 2019-08-02T00:00:00Z -o /tmp/stack2nix-output-dir.tx4bkmDm8M/stack2nix-output.nix
/nix/store/cinw572b38aln37glr0zb8lxwrgaffl4-bash-4.4-p23/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Ensuring git version is >= 2 ...
Ensuring cabal version is >= 2 ...

+ cabal2nix --subpath . --system x86_64-linux --compiler ghc-8.4.4 cabal://ALUT-2.4.0.2
[...]
+ cabal2nix --subpath . --system x86_64-linux --compiler ghc-8.4.4 cabal://ztail-1.2.0.2
stack2nix: /tmp/stack2nix-output-dir.tx4bkmDm8M/stack2nix-output.nix: commitBuffer: invalid argument (invalid character)

I guess there is something to the warning about failing to set LC_ALL?

[nh2]

I don't have much experience with either approach. What are the pros and cons in your opinion?

With stack2nix you can continue using stack and stack.yaml, thus being able to switch the Stackage version at will and use different dependencies much more freely than with nixpkgs (which has only 1 Stackage version in it at any given point in time).

[nh2]

commitBuffer: invalid argument (invalid character)

Ah, I see. I think we're using a too old version, thus missing this fix https://github.com/input-output-hk/stack2nix/commit/cb05818ef8b58899f15641f50cb04e5473b4f9b0 that was released with stack2nix 0.2.3.

[duijf]

@nh2 I wanted to say thank you for all the effort you're putting into static Haskell builds and the help you've given us so far on this one :)

I'll try out the new changes in static-haskell-nix on vaultenv soon!

[duijf]

With https://github.com/channable/vaultenv/pull/82 this is fixed. We'll upload a static binary for vaultenv 0.11.0 :)