`binary_closure` failing to find `main` when linking

[lunaris]

I'm trying to use binary_closure (commit ed9be32ef539ea3645847987e11609028d6491f5, master at time of writing) to create a closure of a haskell_binary, but when I do I'm getting:

ERROR: /.../BUILD:43:1: Linking of rule '//engine:test-closure-closure_wrapper' failed (Exit 1)
../sysdeps/x86_64/start.S:110: error: undefined reference to 'main'
collect2: error: ld returned 1 exit status

which presumably indicates that the binary's main function isn't being picked up correctly. For reference, the haskell_binary looks like:

haskell_binary(
    name = "test-bin",
    src_strip_prefix = "bin/test/test-bin/src",
    srcs = glob(['bin/test/test-bin/src/**/*.hs']),
    deps = [
        ":test-lib",
        ],
    prebuilt_dependencies = [
        "base",
        ],
    )

haskell_library(
    name = "test-lib",
    src_strip_prefix = "lib/test-lib/src",
    srcs = glob(['lib/test-lib/src/**/*.hs']),
    deps = [
        hazel_library("bifunctors"),
        hazel_library("constraints"),
        hazel_library("dlist"),
        hazel_library("hashable"),
        hazel_library("lens"),
        hazel_library("mmorph"),
        hazel_library("mono-traversable"),
        hazel_library("protolude"),
        hazel_library("tagged"),
        hazel_library("text"),
        ],
    prebuilt_dependencies = [
        "base",
        "bytestring",
        "containers",
        "transformers",
        ],
    )

while the closure is built with:

binary_closure(
    name = "test-closure",
    src = "test-bin",
    )

(If context needed, hazel_library is provided by FormationAI/hazel and creates external dependencies out of Hackage packages)

The binary works when built and run with bazel run //engine:test-bin. When I run bazel build --subcommands //engine:test-closure I see:

SUBCOMMAND: # //engine:test-closure-closure_wrapper [action 'Linking engine/test-closure-closure_wrapper']
(cd /.../.cache/bazel/_bazel_../540...ae3a/execroot/... && \
  exec env - \
    PATH=/../.nix-profile/bin:/.. \
    PWD=/proc/self/cwd \
  /.../.cache/bazel/_bazel_.../540...ae3a/external/gcc/bin/gcc -o bazel-out/k8-fastbuild/bin/engine/test-closure-closure_wrapper '-Wl,-rpath,$ORIGIN/../_solib_k8/_U_S_Sengine_Ctest-closure-closure_Uwrapper___Uengine_Sengine_UStest-closure-closure_UUsolib' -Lbazel-out/k8-fastbuild/bin/_solib_k8/_U_S_Sengine_Ctest-closure-closure_Uwrapper___Uengine_Sengine_UStest-closure-closure_UUsolib -Lengine_Stest-closure-closure_Usolib '-Wl,-rpath=$ORIGIN' bazel-out/k8-fastbuild/genfiles/engine/test-closure-closure-params.ld -Tbazel-out/k8-fastbuild/genfiles/engine/test-closure-closure-search_dirs.ld '-fuse-ld=gold' -Wl,-no-as-needed -Wl,-z,relro,-z,now -B/.../.cache/bazel/_bazel_.../540...ae3a/external/gcc/bin -B/usr/bin -pass-exit-codes -Wl,-S -Wl,@bazel-out/k8-fastbuild/bin/engine/test-closure-closure_wrapper-2.params)
ERROR: /.../BUILD:43:1: Linking of rule '//engine:test-closure-closure_wrapper' failed (Exit 1)
../sysdeps/x86_64/start.S:110: error: undefined reference to 'main'
collect2: error: ld returned 1 exit status
Target //engine:test-closure failed to build

test-closure_closure_wrapper-2.params contains:

-lHSrts_thr-ghc8.2.2
-lexternal_Shaskell_Utext_Slibtext-cbits
-lexternal_Shaskell_Uhashable_Slibhashable-cbits
-lexternal_Shaskell_Uprimitive_Slibprimitive-cbits
-lexternal_Shaskell_Utext_Stext-cbits_libtext-cbits
-lexternal_Shaskell_Uhashable_Shashable-cbits_libhashable-cbits
-lexternal_Shaskell_Uprimitive_Sprimitive-cbits_libprimitive-cbits
-lHSexternalZShaskellZUbifunctorsZSPaths-bifunctors-1.0.0-ghc8.2.2
-lHSexternalZShaskellZUbaseZUorphansZSbase-orphans-0.7-ghc8.2.2
-lHSexternalZShaskellZUsemigroupsZSsemigroups-0.18.4-ghc8.2.2
-lHSexternalZShaskellZUtransformersZUcompatZSPaths-transformers-compat-1.0.0-ghc8.2.2
-lHSexternalZShaskellZUtransformersZUcompatZStransformers-compat-0.5.1.4-ghc8.2.2
-lHSexternalZShaskellZUtaggedZStagged-0.8.5-ghc8.2.2
-lHSexternalZShaskellZUstmZSstm-2.4.5.0-ghc8.2.2
-lHSexternalZShaskellZUStateVarZSStateVar-1.1.1.0-ghc8.2.2
-lHSexternalZShaskellZUcontravariantZScontravariant-1.4.1-ghc8.2.2
-lHSexternalZShaskellZUdistributiveZSdistributive-0.5.3-ghc8.2.2
-lHSexternalZShaskellZUcomonadZScomonad-5.0.3-ghc8.2.2
-lHSexternalZShaskellZUthZUabstractionZSth-abstraction-0.2.8.0-ghc8.2.2
-lHSexternalZShaskellZUbifunctorsZSbifunctors-5.5.2-ghc8.2.2
-lHSexternalZShaskellZUtextZStext-1.2.3.0-ghc8.2.2
-lHSexternalZShaskellZUhashableZShashable-1.2.7.0-ghc8.2.2
-lHSexternalZShaskellZUmtlZSmtl-2.2.2-ghc8.2.2
-lHSexternalZShaskellZUconstraintsZSconstraints-0.10-ghc8.2.2
-lHSexternalZShaskellZUdlistZSdlist-0.8.0.4-ghc8.2.2
-lHSexternalZShaskellZUlensZSPaths-lens-1.0.0-ghc8.2.2
-lHSexternalZShaskellZUcallZUstackZSPaths-call-stack-1.0.0-ghc8.2.2
-lHSexternalZShaskellZUcallZUstackZScall-stack-0.1.0-ghc8.2.2
-lHSexternalZShaskellZUprofunctorsZSprofunctors-5.2.2-ghc8.2.2
-lHSexternalZShaskellZUunorderedZUcontainersZSunordered-containers-0.2.9.0-ghc8.2.2
-lHSexternalZShaskellZUsemigroupoidsZSsemigroupoids-5.2.2-ghc8.2.2
-lHSexternalZShaskellZUtransformersZUbaseZStransformers-base-0.4.4-ghc8.2.2
-lHSexternalZShaskellZUexceptionsZSexceptions-0.8.3-ghc8.2.2
-lHSexternalZShaskellZUfreeZSfree-5.0.2-ghc8.2.2
-lHSexternalZShaskellZUvoidZSvoid-0.7.2-ghc8.2.2
-lHSexternalZShaskellZUadjunctionsZSadjunctions-4.4-ghc8.2.2
-lHSexternalZShaskellZUkanZUextensionsZSkan-extensions-5.1-ghc8.2.2
-lHSexternalZShaskellZUparallelZSparallel-3.2.1.1-ghc8.2.2
-lHSexternalZShaskellZUreflectionZSreflection-2.1.3-ghc8.2.2
-lHSexternalZShaskellZUprimitiveZSprimitive-0.6.4.0-ghc8.2.2
-lHSexternalZShaskellZUvectorZSvector-0.12.0.1-ghc8.2.2
-lHSexternalZShaskellZUlensZSlens-4.16.1-ghc8.2.2
-lHSexternalZShaskellZUmmorphZSmmorph-1.1.2-ghc8.2.2
-lHSexternalZShaskellZUmonoZUtraversableZSPaths-mono-traversable-1.0.0-ghc8.2.2
-lHSexternalZShaskellZUsplitZSsplit-0.2.3.3-ghc8.2.2
-lHSexternalZShaskellZUvectorZUalgorithmsZSvector-algorithms-0.7.0.1-ghc8.2.2
-lHSexternalZShaskellZUmonoZUtraversableZSmono-traversable-1.0.8.1-ghc8.2.2
-lHSexternalZShaskellZUasyncZSasync-2.1.1.1-ghc8.2.2
-lHSexternalZShaskellZUprotoludeZSprotolude-0.2.2-ghc8.2.2
-lHSengineZStest-lib-1.0.0-ghc8.2.2
-ltest-bin
-lstdc++
-lm

So it looks like -ltest-bin is being linked against, but perhaps not in the right way? Not sure if there's any useful information I've omitted that might also help debug this; thanks!

[facundominguez]

Hello. The haskell_binary call is not passing "-pie" to ghc. The BUILD file shows how to do it, but not the example in the readme. I will mention it in the readme.

[mboes]

https://github.com/tweag/clodl#usage talks about the PIE requirement, but the binary_closure section should be made consistent with this later section.

[lunaris]

Hi both. Thanks very much for responding to this. So I've added -pie (and -dynamic and -threaded, though those too appear to already be in the generated command line) to both my haskell_binary and, when that failed, to the haskell_library it depends on, to no avail. I then set about seeing if perhaps I needed to play the bootstrap trick you are playing in this repository (in which you export a cc_library of GHC's RTS includes and make main a weak symbol), but before I did so thought I'd check in here to see whether or not that's an expected requirement.

[facundominguez]

The bootstrap bit shoudn't be necessary. At least bazel run clotestbin doesn't use it and works in CI.

If you have a repo to share (or a gist or whatever), I could have a look.

[lunaris]

I've created https://github.com/lunaris/bazel-example, which I think is a reproduction of the issue I'm seeing. It depends on a few common Haskell libraries to make sure there are a variety of cbits etc. floating around -- you can try and build a closure using bazel build //engine:test-closure. Many thanks!

[facundominguez]

Adding -rdynamic to compiler_flags fixes it for me. But I don't understand yet why it isn't necessary in the clodl repo.

[lunaris]

Interesting. Is it perhaps that your bootstrapping stuff references main in a way that means it is considered a "used" symbol whereas in my example that recognition is missing for some reason?

[facundominguez]

That would explain it for the cases where we use the bootstrap library. But it also works without it.

[facundominguez]

That would explain it for the cases where we use the bootstrap library. But it also works without it.

I was wrong. Linking bootstrap does have an effect here, and we are always linking bootstrap.

Linking bootstrap into hello-hs includes main in the table of dynamic symbols:

$ nm -D bazel-bin/hello-hs | grep main
                 U hs_main
                 U __libc_start_main
00000000000024d3 T main

But if we don't link bootstrap, main is missing.

$ nm -D bazel-bin/engine/test-bin | grep main
                 U hs_main
                 U __libc_start_main

We will have to update the documentation to account for this. We either ask -rdynamic, or -Wl,--dynamic-list=<file with a list of symbols including main>. Also, we can test if the input binary defines main when using binary_closure and suggest in the error message one of these alternatives.

[lunaris]

Thanks so much for helping out with this!

[facundominguez]

You're welcome. Thanks for reporting the issue and putting the testing repo together.