nh2 / static-haskell-nix

easily build most Haskell programs into fully static Linux executables
388 stars 36 forks source link

ghcWithPackages fails to build #109

Closed aveltras closed 3 years ago

aveltras commented 3 years ago

Hi,

First of all thanks for your work on this, that's very appreciated.

I've been trying to get static builds on a project at work now that static-haskell-nix is up to date with ghc 8.10.4. The project is using Bazel and I have been able to get a ghcWithPackages with all our dependencies in scope to build. However I'm facing issues down the line while trying to build our binaries statically. To debug this, I'm trying to setup a minimal repo to reproduce the issue but I'm then facing issues before reaching the same point.

Repo is available here: https://github.com/aveltras/haskell-bazel-nix/tree/static

The nix setup is the following:

let
  githubTarball = owner: repo: rev:
    builtins.fetchTarball { url = "https://github.com/${owner}/${repo}/archive/${rev}.tar.gz"; };

  compiler = "ghc8104";

  baseNixpkgs = githubTarball "NixOS" "nixpkgs" "21.05";
  haskellNixpkgs = githubTarball "NixOS" "nixpkgs" "d00b5a5fa6fe8bdf7005abb06c46ae0245aec8b5";
  staticHaskellNixpkgs = githubTarball "nh2" "static-haskell-nix" "bd66b86b72cff4479e1c76d5916a853c38d09837";

  haskellPkgsOverlay = (self: super: {
    haskell = super.haskell // {
      packages = super.haskell.packages // {
        "${compiler}" = super.haskell.packages."${compiler}".override {
          overrides = final: prev: 
            let
              effectfulSrc = githubTarball "arybczak" "effectful" "1d35ef1ddebe8d8689a0a9ba97c4cd6d6968691b";
            in {
              cabal2nix = self.haskell.lib.dontCheck prev.cabal2nix;
              effectful = self.haskell.lib.dontCheck (self.haskellPackages.callCabal2nix "effectful" effectfulSrc {});
            };
        };
      };
    };
  });

  staticHaskellPkgs = (import (staticHaskellNixpkgs + "/survey/default.nix") {
    inherit compiler;
    normalPkgs = import haskellNixpkgs { overlays = [haskellPkgsOverlay]; };
  }).approachPkgs;

  localOverlays = [

    (self: super: {

      # replace staticHaskell lndir with default one
      xorg = super.xorg // {
        lndir = super.xorg.lndir;
      };

      staticHaskell = staticHaskellPkgs.extend (selfSH: superSH: {
        ghc = (superSH.ghc.override {
          enableRelocatedStaticLibs = true;
          enableShared = false;
        }).overrideAttrs (oldAttrs: {
          preConfigure = ''
            ${oldAttrs.preConfigure or ""}
            echo "GhcLibHcOpts += -fPIC -fexternal-dynamic-refs" >> mk/build.mk
            echo "GhcRtsHcOpts += -fPIC -fexternal-dynamic-refs" >> mk/build.mk
          '';
        });
      });
    })

    # override ghc adding all project dependencies as toolchain packages
    (self: super: {
      compiler = super.staticHaskell.haskellPackages.ghcWithPackages (p: with p; [
        # aeson # uncommenting this breaks the build
        # effectful
        bytestring
      ]);
    })

    (self: super: {
      raw-haskell-base-image =
        let haskellBase = super.dockerTools.buildLayeredImage {
          name = "haskell-base-image";
          created = "now";
          contents = with super; [
            # gmp
            # libffi
          ];
        };
        in super.runCommand "haskell-base-image" { } ''
          mkdir -p $out
          gunzip -c ${haskellBase} > $out/image
        '';
    })
  ];

in args@{ overlays ? [], ... }:
    import baseNixpkgs (args // {
      overlays = localOverlays ++ overlays;
    })

I'm trying to build with nix-build -Q nixpkgs.nix -A compiler.

When adding dependencies like aeson in the ghcWithPackages section, it breaks with:

these derivations will be built:
  /nix/store/09za53mlnrdb66rq0j889lrsn5xkrdg6-lndir-1.0.3.drv
  /nix/store/nakzx1sqxpl757w62vd3mjzn92m504w8-ghc-8.10.4-with-packages.drv
building '/nix/store/09za53mlnrdb66rq0j889lrsn5xkrdg6-lndir-1.0.3.drv'...
builder for '/nix/store/09za53mlnrdb66rq0j889lrsn5xkrdg6-lndir-1.0.3.drv' failed with exit code 126; last 10 log lines:
  unpacking source archive /nix/store/rhwljq7yyq4gka6jan564ywahjjx8jxr-lndir-1.0.3.tar.bz2
  source root is lndir-1.0.3
  setting SOURCE_DATE_EPOCH to timestamp 1331185905 of file lndir-1.0.3/INSTALL
  patching sources
  updateAutotoolsGnuConfigScriptsPhase
  Updating Autotools / GNU config script to a newer upstream version: ./config.sub
  Updating Autotools / GNU config script to a newer upstream version: ./config.guess
  configuring
  configure flags: --disable-dependency-tracking --prefix=/nix/store/3r95ninc1sbm5agl9308zqc0kggb7gfa-lndir-1.0.3
  /nix/store/p61072rqa772qnwqhbn9xy9nbpcxw7cb-stdenv-linux/setup: ./configure: /bin/sh: bad interpreter: No such file or directory
cannot build derivation '/nix/store/nakzx1sqxpl757w62vd3mjzn92m504w8-ghc-8.10.4-with-packages.drv': 1 dependencies couldn't be built
error: build of '/nix/store/nakzx1sqxpl757w62vd3mjzn92m504w8-ghc-8.10.4-with-packages.drv' failed

When there's only bytestringin the dependencies, it builds fine.

The section:

replace staticHaskell lndir with default one
xorg = super.xorg // {
  lndir = super.xorg.lndir;
};

in the overlay is there be cause it seems to have fixed that kind of issue on our main project repo but not sure it's useful here.

I'm a bit puzzled here since the setup is nearly the same on our main repo, we just have some more overrides in the haskellPkgsOverlay (I've copy pasted the nixpkgs.nix in the demo repo and trimmed it a bit but that's all).

Any idea on how to solve this ?

aveltras commented 3 years ago

Seems the problems came from something else as I can't reproduce it today. Closing.