Darwin build error - ghc: loadArchive: neither an archive, nor a fat archive

[brendanhay]

This is more of a help request than a rules_haskell issue. (Apologies if this isn't the correct place to ask!)

I have a project that I'm converting to bazel but have encountered a head scratching error when building on macOS specifically (Linux builds fine.) 140 out of 141 packages build correctly, but one fails with the following error:

ERROR: /Users/administrator/amazonka/gen/BUILD.bazel:10:16: HaskellBuildLibrary //gen:lib failed: (Exit 1): ghc_wrapper failed: error executing command
  (cd /private/var/tmp/_bazel_administrator/bfc4aa19cc8105eb50a996b73cc646b3/sandbox/processwrapper-sandbox/860/execroot/amazonka && \
  exec env - \
    CC_WRAPPER_CC_PATH=external/local_config_cc/cc_wrapper.sh \
    CC_WRAPPER_CPU=darwin \
    CC_WRAPPER_PLATFORM=darwin \
    LANG=C.UTF-8 \
    RULES_HASKELL_DOCDIR_PATH=/nix/store/jhqn60bsm36lv9z9b155qy7izakylzin-ghc-8.10.7-doc/share/doc/ghc/html/libraries/base-4.14.3.0 \
    RULES_HASKELL_GHC_PATH=external/rules_haskell_ghc_nixpkgs/bin/ghc \
    RULES_HASKELL_GHC_PKG_PATH=external/rules_haskell_ghc_nixpkgs/bin/ghc-pkg \
    RULES_HASKELL_LIBDIR_PATH=/nix/store/8znjk0f8np09lvqi09lzdmw713lgldr4-ghc-8.10.7/lib/ghc-8.10.7 \
  bazel-out/host/bin/external/rules_haskell/haskell/ghc_wrapper bazel-out/darwin-fastbuild/bin/gen/compile_flags_lib_HaskellBuildLibrary bazel-out/darwin-fastbuild/bin/gen/extra_args_lib_HaskellBuildLibrary)
Execution platform: @io_tweag_rules_nixpkgs//nixpkgs/platforms:host

Use --sandbox_debug to see verbose messages from the sandbox
ghc: loadArchive: Neither an archive, nor a fat archive: `/nix/store/gsf62i0m9a3lbvnlvci9nlranvx7ncrw-clang-wrapper-7.1.0/bin/clang++'
ghc: panic! (the 'impossible' happened)
  (GHC version 8.10.7:
    loadArchive "/nix/store/gsf62i0m9a3lbvnlvci9nlranvx7ncrw-clang-wrapper-7.1.0/bin/clang++": failed

Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

This can be reproduced via:

git clone https://github.com/brendanhay/amazonka.git
cd amazonka
git checkout bh/the-numbers-all-go-to-eleven
nix-shell --run 'bazel build //gen'

I have other projects built with identical bazelrc/WORKSPACE/nixpkgs/etc. that work as expected - so at a wild guess I assume it's related to a dependency and ghc 16590? I'd rather not resort to patching ghc if possible, so, has anyone encountered this before or has advice how to debug this further?

[aherrmann]

Thanks for the detailed bug report! I haven't encountered that issue before.

Reading through ghc 16590 I see the following question being raised:

[...] it's searching for the prefixed and suffixed version of the library first. So why did libc++.dylib or libc++.a not find your library?

That's a good question. IIUC the issue should not occur if clang finds one of these libc++ first. I can see in your WORKSPACE that you are using the rules_nixpkgs provided C toolchain. On MacOS that one is configured to include libcxx in the library search path. You can add ghcopts = ["-v"] to the target to make GHC print out more verbose messages. Perhaps that reveals a bit more about what it is looking for in which order.

[brendanhay]

Thanks @aherrmann.

It looks like it's failing on double-conversion. I've attached the output at the bottom of this message since it's noisy - but the order in which cc_wrapper-python is invoked is:

libs=(
  'libc++.dylib'
  'liblibc++.dylib'
  'c++.lib'
  'libc++.lib'
  'libc++.dll.a'
  'c++.dll.a'
  'libc++.a'
  'c++.a'
  'libc++'
  'c++'
)

for lib in "${libs[@]}"; do
  bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python \
    -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib \
    --print-file-name "$lib"
done

# Outputs
libc++.dylib
liblibc++.dylib
c++.lib
libc++.lib
libc++.dll.a
c++.dll.a
libc++.a
c++.a
libc++
/nix/store/gsf62i0m9a3lbvnlvci9nlranvx7ncrw-clang-wrapper-7.1.0/bin/c++

Running otool on the library:

> otool -L bazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib/libHSdouble-conversion-2.0.2.0-ghc8.10.7.dylib
/nix/store/lnh4bn0wf59mqdzzb9w17g6a24vffnrw-libcxx-7.1.0/lib/libc++.1.0.dylib

Original output with -v:

Loading package texmath-0.12.3.1 ... linking ... done.
Loading package digest-0.0.1.3 ... linking ... done.
Loading package zip-archive-0.4.1 ... linking ... done.
Loading package pandoc-2.14.0.3 ... linking ... done.
Loading package haskell-src-exts-1.23.1 ... linking ... done.
Loading package clock-0.8.2 ... linking ... done.
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'libc++.dylib'
!!! systool:linker: finished in 0.58 milliseconds, allocated 0.064 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'liblibc++.dylib'
!!! systool:linker: finished in 0.50 milliseconds, allocated 0.063 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'c++.lib'
!!! systool:linker: finished in 0.52 milliseconds, allocated 0.062 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'libc++.lib'
!!! systool:linker: finished in 0.49 milliseconds, allocated 0.062 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'libc++.dll.a'
!!! systool:linker: finished in 0.50 milliseconds, allocated 0.063 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'c++.dll.a'
!!! systool:linker: finished in 0.49 milliseconds, allocated 0.062 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'libc++.a'
!!! systool:linker: finished in 0.48 milliseconds, allocated 0.062 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'c++.a'
!!! systool:linker: finished in 0.49 milliseconds, allocated 0.061 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'libc++'
!!! systool:linker: finished in 0.51 milliseconds, allocated 0.061 megabytes
*** systool:linker:
*** gcc:
bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -Bbazel-out/darwin-fastbuild/bin/gen/../external/stackage/double-conversion-2.0.2.0/_install/lib --print-file-name 'c++'
!!! systool:linker: finished in 0.50 milliseconds, allocated 0.061 megabytes
Loading package double-conversion-2.0.2.0 ... ghc: loadArchive: Neither an archive, nor a fat archive: `/nix/store/gsf62i0m9a3lbvnlvci9nlranvx7ncrw-clang-wrapper-7.1.0/bin/clang++'

[brendanhay]

Note that I've "fixed" this by surgically removing the double-conversion dependency - not particularly helpful if is someone else runs into this, but it was expedient.

[aherrmann]

Thanks for following up and glad to hear that you found a workaround.

It's still a bit puzzling to me that clang doesn't find libc++. I don't have a MacOS machine handy at the moment, but, I poked at clang on Linux and found the following.

$ nix-shell -p clang -p libcxx
$ clang++ -L/nix/store/wdk8c9qrcbyyyrj7f222yyb0ypm5qd6m-libcxx-7.1.0/lib --print-file-name=libc++.so
libc++.so
$ clang++ -B/nix/store/wdk8c9qrcbyyyrj7f222yyb0ypm5qd6m-libcxx-7.1.0/lib --print-file-name=libc++.so
/nix/store/wdk8c9qrcbyyyrj7f222yyb0ypm5qd6m-libcxx-7.1.0/lib/libc++.so

So, it looks like clang does not consult -L for --print-file-name but only -B. (gcc seems to behave the same way). So, perhaps another solution to this issue might be to add a -B flag to the clang wrapper.

Alternatively, you could perhaps import libcxx as a cc_library via nixpkgs_package and then add it to the dependencies of double-conversion using the extra_deps attribute of stack_snapshot.