search

nh2 / static-haskell-nix

easily build most Haskell programs into fully static Linux executables
388 stars 36 forks source link

Tracking issue for GHC bootstrapping compilers being broken with musl in nixpkgs #99

Closed talw closed 3 years ago

talw commented 4 years ago

First of all, a big thanks for all the work on this big project!

I am trying to build a project statically with GHC 8.8.3 and I get the following error:

/nix/store/nrdv3qhafvxnvbxqksahnffw47ymafh7-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-pkg: /nix/store/pk12rld0n6g6k74s0c0j9wmr7glpfnrj-ncurses-6.2-abi5-compat/lib/libtinfo.so.5: no version information available (required by /nix/store/nrdv3qhafvxnvbxqksahnffw47ymafh7-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/../terminfo-0.4.1.2/libHSterminfo-0.4.1.2-ghc8.6.5.so)
Inconsistency detected by ld.so: dl-lookup.c: 111: check_match: Assertion `version->filename == NULL || ! _dl_name_match_p (version->filename, map)' failed!
make[1]: *** [ghc.mk:991: install_packages] Error 127
make: *** [Makefile:51: install] Error 2
builder for '/nix/store/4xb1js32jx9a1iwa1b4l3zb5vqm81cvy-ghc-8.6.5-binary.drv' failed with exit code 2
cannot build derivation '/nix/store/jkjqd7fwgv18mqlm8zrpm95cz17h313a-ghc-8.8.3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/p75l86x5qapq0grk7a3cj2lwi52db9qr-myProj-0.1.0.drv': 1 dependencies couldn't be built

The involved files are:

default.nix:

let
  compiler = "ghc883";

  pkgs = import ./nix/pkgs.nix compiler;

  static-haskell-nix = fetchGit (builtins.fromJSON (builtins.readFile ./nix/static-haskell-rev.json));
  static-pkgs = (import (static-haskell-nix + "/survey/default.nix") {
      pkgs = pkgs.pkgsMusl;
      inherit compiler;
      defaultCabalPackageVersionComingWithGhc = "Cabal_3_0_0_0";
    }).pkgs;

  haskPkgs = pkgs.haskell.packages.${compiler};

  gitignore = pkgs.nix-gitignore.gitignoreSourcePure [ ./.gitignore ];

  static-drv = static-pkgs.haskell.lib.overrideCabal
            (static-pkgs.haskell.packages.${compiler}.callPackage ./proj.nix {})
            (old: {
                  configureFlags = [
                    "-flambda"
                    "--ghc-option=-fPIC"
                    "--ghc-option=-optl-static"
                    "--ghc-option=-optc-static"
                    "--ghc-option=-optl-pthread"
                    "--ld-option=-static"
                    "--ld-option=-pthread"
                    "--extra-lib-dirs=${static-pkgs.gmp6.override { withStatic = true; }}/lib"
                    "--extra-lib-dirs=${static-pkgs.zlib.static}/lib"
                    "--extra-lib-dirs=${static-pkgs.libffi.overrideAttrs (old: { dontDisableStatic = true; })}/lib"
                    "--disable-executable-stripping"
                  ];
            });

in
  {
    inherit static-drv;
  }

proj.nix:

{ mkDerivation, aeson, amazonka, amazonka-core, amazonka-dynamodb
, amazonka-sqs, aws-lambda-haskell-runtime, base, bytestring
, composition, containers, either, generic-lens, iproute, microlens
, random, rio, stdenv, text, time, unordered-containers, uuid
, vector
}:
mkDerivation {
  pname = "myProj";
  version = "0.1.0";
  src = ./.;
  isLibrary = true;
  isExecutable = true;
  libraryHaskellDepends = [
    aeson amazonka amazonka-core amazonka-dynamodb amazonka-sqs
    aws-lambda-haskell-runtime base bytestring composition containers
    either generic-lens iproute microlens random rio text time
    unordered-containers uuid vector
  ];
  executableHaskellDepends = [
    amazonka aws-lambda-haskell-runtime base iproute
  ];
  license = "unknown";
  hydraPlatforms = stdenv.lib.platforms.none;
}

nix/pkgs-rev.json:

{
  "url": "git://github.com/NixOS/nixpkgs-channels",
  "ref": "nixos-unstable",
  "rev": "4cdd64d9078181e8fcb6132909dc2835059a09bd"
}

nix/static-haskell-rev.json:

{
  "url": "git://github.com/nh2/static-haskell-nix",
  "ref": "master",
  "rev": "dbce18f4808d27f6a51ce31585078b49c86bd2b5"
}

I tried many other permutations of options for configureFlags, as well as adding stuff to the cabal file like ld-options: -static and cc-options: -static.

ghc-options: -threaded -rtsopts -O2 -static -with-rtsopts=-N
ld-options: -static -pthread
cc-options: -static

While, this is not enough to reproduce the problem yourselves, If I could receive as much as a clue as to a possible direction I should head towards, or a thing I should try, I will do it and reply with the results, so feel free to suggest anything!

Why is GHC 8.6.5 involved in the building of this? Is this related to the version mismatch? I tried stating a build with compiler = "ghc865" It gave an error about a dependency time not found at the correct version for rebase, so I jailbroken it and continued building but again, once the last module for the project finishes compiling I get an error about that crtbeginT.o but this time the message is different:

/nix/store/x4hjqhnzfbzh4f333xh1sq31zak3dab1-binutils-2.31.1/bin/ld: /nix/store/p9q71abn3zxpbqrr1k4f35j3k6kx8ayk-gcc-9.3.0/lib/gcc/x86_64-unknown-linux-musl/9.3.0/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
/nix/store/x4hjqhnzfbzh4f333xh1sq31zak3dab1-binutils-2.31.1/bin/ld: /nix/store/p9q71abn3zxpbqrr1k4f35j3k6kx8ayk-gcc-9.3.0/lib/gcc/x86_64-unknown-linux-musl/9.3.0/crtend.o: relocation R_X86_64_32 against `.ctors' can not be used when making a shared object; recompile with -fPIC
/nix/store/x4hjqhnzfbzh4f333xh1sq31zak3dab1-binutils-2.31.1/bin/ld: final link failed: nonrepresentable section on output
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)

Anyone got a static build working with GHC 8.8.x?

nh2 commented 4 years ago 
check_match: Assertion `version->filename == NULL || ! _dl_name_match_p (version->filename, map)' failed!

This is a GHC-in-nixpkgs problem; googling that in conjuction with nixpkgs brings up this first result: https://github.com/NixOS/nixpkgs/issues/85924

The fix suggested by @bgamari at https://github.com/NixOS/nixpkgs/issues/85924#issuecomment-640277067 works for me (in https://github.com/nh2/nixpkgs/commit/d7f663fdf4ca183fe48ff2dba06d03e9c19530ed), I'm also using it in #90.

But nobody has PR'd that fix yet.

I think the main holdup with that is testing it and also fixing it for the non-x86_64 GHCs.

If you would like to help with that, start with my patch, also update i686-linux and aarch64-linux to the fitting Fedora releases in here, test that all the Haskell packages build, and PR the change against the haskell-updates nixpkgs branch, CC'ing the participants of https://github.com/NixOS/nixpkgs/issues/85924 and #90 on the PR. You might also want to join @peti's Haskell Twitch stream on Fridays where such PRs are reviewed together.

talw commented 4 years ago 
check_match: Assertion `version->filename == NULL || ! _dl_name_match_p (version->filename, map)' failed!

This is a GHC-in-nixpkgs problem; googling that in conjuction with nixpkgs brings up this first result: NixOS/nixpkgs#85924

The fix suggested by @bgamari at NixOS/nixpkgs#85924 (comment) works for me (in nh2/nixpkgs@d7f663f), I'm also using it in #90.

But nobody has PR'd that fix yet.

I think the main holdup with that is testing it and also fixing it for the non-x86_64 GHCs.

If you would like to help with that, start with my patch, also update i686-linux and aarch64-linux to the fitting Fedora releases in here, test that all the Haskell packages build, and PR the change against the haskell-updates nixpkgs branch, CC'ing the participants of NixOS/nixpkgs#85924 and #90 on the PR. You might also want to join @peti's Haskell Twitch stream on Fridays where such PRs are reviewed together.

Thank you very much! I googled a lot of stuff but did not come across that issue! From this I learn that one should search in nixpkgs' issues as well. Should this issue be closed as a duplicate of NixOS/nixpkgs#85924 ? Although it's an issue in nixpkgs, that makes me realize this issue perhaps belongs there as well.

Also, thanks for the guidance on how to proceed from here in order to fix this upstream!

nh2 commented 4 years ago

Should this issue be closed as a duplicate of NixOS/nixpkgs#85924 ?

We can keep it open, I renamed it accordingly.

Also, thanks for the guidance on how to proceed from here in order to fix this upstream!

Sure thing, that's probably the highest-impact work I can do :)

nomeata commented 3 years ago

@talw did you find out more about the crtbeginT.o issue? I am running into this trying to build statically as well (in my case building on top of haskell.nix, https://github.com/input-output-hk/haskell.nix/issues/914)

talw commented 3 years ago

@talw did you find out more about the crtbeginT.o issue? I am running into this trying to build statically as well (in my case building on top of haskell.nix, input-output-hk/haskell.nix#914)

@nomeata

Yes. I applied the patch by @bgamari as suggested by @nh2 and it fixed the issue. The fix is: https://github.com/NixOS/nixpkgs/issues/85924#issuecomment-640277067

nomeata commented 3 years ago

Thanks for applying that fix upstream. It seems I now have a new problem with the bootstrap compiler:

…
Patchelfing /nix/store/iz8azixwl6vzzd87q1p38vlsm3hb21aa-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/hp2ps
Patchelfing /nix/store/iz8azixwl6vzzd87q1p38vlsm3hb21aa-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/ghc-iserv
Patchelfing /nix/store/iz8azixwl6vzzd87q1p38vlsm3hb21aa-ghc-8.6.5-binary/lib/ghc-8.6.5/bin/runghc
Patchelfing /nix/store/iz8azixwl6vzzd87q1p38vlsm3hb21aa-ghc-8.6.5-binary/lib/ghc-8.6.5/ghc-boot-th-8.6.5/libHSghc-boot-th-8.6.5-ghc8.6.5.so
running install tests
/nix/store/6my73ym207a4ds2szcl9wnr1gby10q56-stdenv-linux/setup: line 1318:  7439 Segmentation fault      (core dumped) $out/bin/ghc --make main.hs
builder for '/nix/store/00x82zgp94rf4682gzq63d2nlldzvfmb-ghc-8.6.5-binary.drv' failed with exit code 1

Anyone seen that before?

Opened https://github.com/NixOS/nixpkgs/issues/118731 to track that.

sjakobi commented 3 years ago

Just a shot in the dark, but it might be related to https://gitlab.haskell.org/ghc/ghc/-/issues/17508?!

bgamari commented 3 years ago

Indeed every time I've seen a static build fail in the past it has been due to GHC#17508., which is itself due to a very unfortunate ld.gold bug. That being said, I vaguely remember nixpkgs having already worked around this by not using ld.gold in the static GHC build. Can you confirm, @nh2?

nomeata commented 3 years ago

@talw did you find out more about the crtbeginT.o issue? I am running into this trying to build statically as well (in my case building on top of haskell.nix, input-output-hk/haskell.nix#914)

@nomeata

Yes. I applied the patch by @bgamari as suggested by @nh2 and it fixed the issue. The fix is: NixOS/nixpkgs#85924 (comment)

Thanks! I now get similar error messages when trying nix-build -A pkgsMusl.buildPackages.haskell.compiler.ghc8102Binary on nixpkgs master, namely

Installing library in /nix/store/ffa8jpz98vnqyfsl23pqgp5j44c4rl78-ghc-8.10.2-binary/lib/ghc-8.10.2/ghc-8.10.2
"/nix/store/ffa8jpz98vnqyfsl23pqgp5j44c4rl78-ghc-8.10.2-binary/lib/ghc-8.10.2/bin/ghc-pkg" --force --global-package-db "/nix/store/ffa8jpz98vnqyfsl23pqgp5j44c4rl78-ghc-8.10.2-binary/lib/ghc-8.10.2/package.conf.d" update rts/dist/package.conf.install
/nix/store/ffa8jpz98vnqyfsl23pqgp5j44c4rl78-ghc-8.10.2-binary/lib/ghc-8.10.2/bin/ghc-pkg: /nix/store/s5xpnmnhibl76a47wd65cwrwgjclzrnc-ncurses-6.2/lib/libtinfo.so.6: no version information available (required by /nix/store/ffa8jpz98vnqyfsl23pqgp5j44c4rl78-ghc-8.10.2-binary/lib/ghc-8.10.2/bin/ghc-pkg)
/nix/store/ffa8jpz98vnqyfsl23pqgp5j44c4rl78-ghc-8.10.2-binary/lib/ghc-8.10.2/bin/ghc-pkg: /nix/store/s5xpnmnhibl76a47wd65cwrwgjclzrnc-ncurses-6.2/lib/libtinfo.so.6: no version information available (required by /nix/store/ffa8jpz98vnqyfsl23pqgp5j44c4rl78-ghc-8.10.2-binary/lib/ghc-8.10.2/bin/../terminfo-0.4.1.4/libHSterminfo-0.4.1.4-ghc8.10.2.so)
Inconsistency detected by ld.so: dl-lookup.c: 105: check_match: Assertion `version->filename == NULL || ! _dl_name_match_p (version->filename, map)' failed!
make[1]: *** [ghc.mk:973: install_packages] Error 127
make: *** [Makefile:51: install] Error 2
builder for '/nix/store/hnn4qhzk85vy5rhpi77zfcsykfaz6mmh-ghc-8.10.2-binary.drv' failed with exit code 2

but it seems that the patch by bgamari that you linked to is already applied there.

nh2 commented 3 years ago

We can close this issue once the latter is merged.

nh2 commented 3 years ago

Closing as I merged the fix: https://github.com/NixOS/nixpkgs/pull/129289

You will still need to wait a bit until it hits master; subscribe to https://github.com/NixOS/nixpkgs/issues/129247 for that.