search

tweag / rules_haskell

Haskell rules for Bazel.
https://haskell.build
Apache License 2.0
266 stars 80 forks source link

Examples do not build #1419

Open TomMD opened 4 years ago

TomMD commented 4 years ago

Describe the bug The examples in rules_haskell do not build.

To Reproduce On my OS-X machine

git clone https://github.com/tweag/rules_haskell
cd rules_haskell/examples
bazel build //cat_hs/exec/cat_hs

And observe

ERROR: /private/tmp/rules_haskell/examples/cat_hs/exec/cat_hs/BUILD.bazel:6:15 HaskellCabalLibrary @stackage//:unliftio-core failed (Exit 1) cabal_wrapper failed: error executing command bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:unliftio-core unliftio-core-0.1.2.0 true external/stackage/unliftio-core-0.1.2.0/Setup.hs external/stackage/unliftio-core-0.1.2.0 ... (remaining 7 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
INFO: Elapsed time: 3.009s, Critical Path: 2.85s
INFO: 20 processes: 20 darwin-sandbox.
FAILED: Build did NOT complete successfully

Expected behavior I thought this would build a binary that could be run.

Environment

Additional context Add any other context about the problem here.

TomMD commented 4 years ago

I also tried on linux:

docker run --rm -it ubuntu:20.04 bash
# follow bazel instructions here https://docs.bazel.build/versions/master/install-ubuntu.html
apt update && apt install -y git
git clone https://github.com/tweag/rules_haskell
cd rules_haskell/examples
bazel build //cat_hs/exec/cat_hs

Yielding:

# bazel build //cat_hs/exec/cat_hs
Starting local Bazel server and connecting to it...
DEBUG: /root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/private/versions.bzl:60:10: WARNING: bazel version is too recent. Supported versions range from 2.1.0 to 3.3.1, but found: 3.5.0
INFO: Repository rules_haskell_python_local instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule _config_python3_toolchain defined at:
  /root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl:598:44: in <toplevel>
ERROR: An error occurred during the fetch of repository 'rules_haskell_python_local':
   Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl", line 562, column 32, in _configure_python3_toolchain_impl
                python3_path = _find_python(repository_ctx)
        File "/root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl", line 555, column 40, in _find_python
                result = repository_ctx.execute([python, "--version"])
Error in execute: Argument 0 of execute is neither a path, label, nor string.
ERROR: Analysis of target '//cat_hs/exec/cat_hs:cat_hs' failed; build aborted: Argument 0 of execute is neither a path, label, nor string.
INFO: Elapsed time: 6.960s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (14 packages loaded, 8 targets configured)
    currently loading: @bazel_tools//tools/jdk ... (2 packages)
    Fetching @rules_haskell_sh_posix_local; fetching
    Fetching @local_config_cc; fetching
mboes commented 4 years ago

The latter looks like an instance of https://github.com/tweag/rules_haskell/issues/1416. Regarding macOS, could you run again with --verbose_failures?

TomMD commented 4 years ago

On linux with python 3 the problem now looks much more manageable (mismatched hashes):

# bazel build --verbose_failures //cat_hs/exec/cat_hs
DEBUG: /root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/private/versions.bzl:60:10: WARNING: bazel version is too recent. Supported versions range from 2.1.0
to 3.3.1, but found: 3.5.0
INFO: Repository rules_haskell_ghc_linux_amd64 instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule _ghc_bindist defined at:
  /root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl:362:31: in <toplevel>
INFO: Repository 'rules_haskell_ghc_linux_amd64' used the following cache hits instead of downloading the corresponding file.
 * Hash 'c419fd0aa9065fe4d2eb9a248e323860c696ddf3859749ca96a84938aee49107' for https://downloads.haskell.org/~ghc/8.6.5/ghc-8.6.5-x86_64-deb8-linux.tar.xz
If the definition of 'rules_haskell_ghc_linux_amd64' was updated, verify that the hashes were also updated.
ERROR: An error occurred during the fetch of repository 'rules_haskell_ghc_linux_amd64':
   Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/ghc_bindist.bzl", line 282, column 31, in _ghc_bindist_impl
                execute_or_fail_loudly(ctx, ["make", "install"])
        File "/root/.cache/bazel/_bazel_root/0d15a96fd12ed9dca003553c71536f42/external/rules_haskell/haskell/private/workspace_utils.bzl", line 18, column 13, in execute_or_fail_loudly
                fail("\n".join(["Command failed: " + " ".join(arguments), exec_result.stderr]))
Error in fail: Command failed: make install
src/main/tools/process-wrapper-legacy.cc:80: "execvp(make, ...)": No such file or directory
ERROR: /rules_haskell/examples/cat_hs/exec/cat_hs/BUILD.bazel:6:15: //cat_hs/exec/cat_hs:cat_hs depends on @rules_haskell_ghc_linux_amd64//:toolchain-impl in repository @rules_haskell_ghc_linux_amd64 which failed to fetch. no such package '@rules_haskell_ghc_linux_amd64//': Command failed: make install
src/main/tools/process-wrapper-legacy.cc:80: "execvp(make, ...)": No such file or directory
ERROR: Analysis of target '//cat_hs/exec/cat_hs:cat_hs' failed; build aborted: Analysis failed
INFO: Elapsed time: 31.728s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
    Fetching @stackage; Restarting. 31s
    Fetching @rules_haskell_stack_update; fetching 31s

But on OS-X the problem is still pretty hairy looking:

Loading: 
Loading: 0 packages loaded
DEBUG: /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/external/rules_haskell/haskell/private/versions.bzl:60:10: WARNING: bazel version is too recent. Supported versions range from 2.1.0 to 3.3.1, but found: 3.5.0
Analyzing: target //cat_hs/exec/cat_hs:cat_hs (0 packages loaded, 0 targets configured)
DEBUG: /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/external/rules_haskell/haskell/cabal.bzl:2023:18: Stack version not recent enough. Downloading a newer version...
WARNING: /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/external/zlib.hs/BUILD.bazel:3:11: in linkstatic attribute of cc_library rule @zlib.hs//:zlib.hs: setting 'linkstatic=1' is recommended if there are no object files. Since this rule was created by the macro 'cc_library', the error might have been caused by the macro implementation
INFO: Analyzed target //cat_hs/exec/cat_hs:cat_hs (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 2] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[1 / 106] HaskellCabalLibrary @stackage//:colour; 1s darwin-sandbox ... (8 actions running)
ERROR: /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/external/stackage/BUILD.bazel:266:22: HaskellCabalLibrary @stackage//:colour failed (Exit 1): cabal_wrapper failed: error executing command 
  (cd /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples && \
  exec env - \
    LANG=en_US.UTF-8 \
    PATH=/usr/bin:/usr/local/bin:/bin:/usr/sbin:/Users/tommd/.nix-profile/bin \
    SDKROOT=macosx \
  bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:colour colour-2.3.5 true external/stackage/colour-2.3.5/Setup.lhs external/stackage/colour-2.3.5 bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d '' '--flags=' '--ghc-option=-w' '--ghc-option=-optF=-w' '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64' --)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox cabal_wrapper failed: error executing command 
  (cd /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples && \
  exec env - \
    LANG=en_US.UTF-8 \
    PATH=/usr/bin:/usr/local/bin:/bin:/usr/sbin:/Users/tommd/.nix-profile/bin \
    SDKROOT=macosx \
  bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:colour colour-2.3.5 true external/stackage/colour-2.3.5/Setup.lhs external/stackage/colour-2.3.5 bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d '' '--flags=' '--ghc-option=-w' '--ghc-option=-optF=-w' '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64' --)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox
Error: DEVELOPER_DIR not set.
`cc_wrapper-python' failed in phase `C Compiler'. (Exit code: 250)
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper.runfiles/rules_haskell/haskell/cabal_wrapper.py", line 156, in <module>
    run([runghc] + runghc_args + [setup, "configure", \
  File "/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper.runfiles/rules_haskell/haskell/cabal_wrapper.py", line 42, in run
    subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, *args, **kwargs)
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/external/rules_haskell_ghc_darwin_amd64/bin/runghc', '/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/external/stackage/colour-2.3.5/Setup.lhs', 'configure', 'lib:colour', '--verbose=0', '--user', '--with-compiler=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/external/rules_haskell_ghc_darwin_amd64/bin/ghc', '--with-hc-pkg=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/external/rules_haskell_ghc_darwin_amd64/bin/ghc-pkg', '--with-ar=/usr/bin/ar', '--with-gcc=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python', '--with-strip=/usr/bin/strip', '--enable-deterministic', '--ghc-option=-pgma', '--ghc-option=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python', '--ghc-option=-pgmc', '--ghc-option=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python', '--ghc-option=-pgml', '--ghc-option=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python', '--ghc-option=-pgmP', '--ghc-option=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/host/bin/external/rules_haskell/haskell/cc_wrapper-python -E -undef -traditional', '--ghc-option=-optc-fno-stack-protector', '--enable-relocatable', '--builddir=../../../bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/tmpmh_tuk86', '--prefix=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install', '--libdir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5_iface', '--dynlibdir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/lib', '--libsubdir=', '--bindir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/bin', '--datadir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5_data', '--datasubdir=', '--haddockdir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5_haddock', '--htmldir=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5_haddock_html', '--package-db=clear', '--package-db=global', '--flags=', '--ghc-option=-w', '--ghc-option=-optF=-w', '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64', '--package-db=/private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples/bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d']' returned non-zero exit status 1.
Target //cat_hs/exec/cat_hs:cat_hs failed to build
ERROR: /private/tmp/rh/examples/cat_hs/exec/cat_hs/BUILD.bazel:6:15 HaskellCabalLibrary @stackage//:colour failed (Exit 1): cabal_wrapper failed: error executing command 
  (cd /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples && \
  exec env - \
    LANG=en_US.UTF-8 \
    PATH=/usr/bin:/usr/local/bin:/bin:/usr/sbin:/Users/tommd/.nix-profile/bin \
    SDKROOT=macosx \
  bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:colour colour-2.3.5 true external/stackage/colour-2.3.5/Setup.lhs external/stackage/colour-2.3.5 bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d '' '--flags=' '--ghc-option=-w' '--ghc-option=-optF=-w' '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64' --)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox cabal_wrapper failed: error executing command 
  (cd /private/var/tmp/_bazel_tommd/33b3f1ba2a9517ff551b8050c05cd331/sandbox/darwin-sandbox/54/execroot/rules_haskell_examples && \
  exec env - \
    LANG=en_US.UTF-8 \
    PATH=/usr/bin:/usr/local/bin:/bin:/usr/sbin:/Users/tommd/.nix-profile/bin \
    SDKROOT=macosx \
  bazel-out/host/bin/external/rules_haskell/haskell/cabal_wrapper lib:colour colour-2.3.5 true external/stackage/colour-2.3.5/Setup.lhs external/stackage/colour-2.3.5 bazel-out/darwin-fastbuild/bin/external/stackage/colour-2.3.5/_install/colour-2.3.5.conf.d '' '--flags=' '--ghc-option=-w' '--ghc-option=-optF=-w' '--ghc-option=-optl-Wl,-rpath,@loader_path/../../../../../_solib_darwin_x86_64/external_Srules_Uhaskell_Ughc_Udarwin_Uamd64' --)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox
INFO: Elapsed time: 2.955s, Critical Path: 2.82s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

N.B. I can run with DEVELOPER_DIR=<some directory> bazel build ... and get the same output.

TomMD commented 4 years ago

If/when the bug is fixed at least for Linux, would you accept a PR that adds building these examples to the CI via a gitlab action or similar?

Profpatsch commented 4 years ago

would you accept a PR that adds building these examples to the CI via a gitlab action or similar?

We are always open to PRs (especially improving examples/documentation/testsuites!). Our CI is built run with buildkite (https://buildkite.com/tweag-1/rules-haskell) and CircleCI (for MacOS https://app.circleci.com/pipelines/github/tweag/rules_haskell).

aherrmann commented 4 years ago

It's worth pointing out that the examples and tutorial are tested on CI in nixpkgs mode. However, they are not currently tested in bindist mode.

timoffex commented 3 years ago

Are there any workarounds for the DEVELOPER_DIR issue on OS-X? I tried setting BAZEL_USER_CPP_ONLY_TOOLCHAIN=1, made sure DEVELOPER_DIR wasn't empty (but it was empty for me before... what is it even?), and did a bazel clean before retrying. None of that seemed to change anything.

$ echo $DEVELOPER_DIR
/Applications/Xcode.app/Contents/Developer
$ echo $BAZEL_USER_CPP_ONLY_TOOLCHAIN
1
$ bazel --version
bazel 3.3.1-homebrew
$ bazel clean && bazel build //cat_hs/exec/cat_hs
<some text and eventually the "DEVELOPER_DIR not set" error>

I'm on macOS Catalina (Version 10.15.4).

TomMD commented 3 years ago

@timoffex I think we need to consider bazel Haskell unmaintained for the non-nix case, there aren't CI tests or interested parties at this point.

The developer dir is actually a clang concept, observe:

$ DEVELOPER_DIR=foo clang
xcrun: error: missing DEVELOPER_DIR path: foo

I've tried adding export DEVELOPER_DIR=... to my .profile and rc but haven't had any impact so whatever is setting the environment isn't playing nice.

aherrmann commented 3 years ago

We do run tests on CI for the macOS bindist use case. However, they currently do not include the example or tutorial. The main test suite and the start script on the other hand are tested on CI in the macOS bindist configuration and are expected to work.

As the example is currently only tested in a Nix configuration, it is possible that the some of the Nix provided toolchains collide with the standard toolchains in the non-Nix use-case. To test this, you could remove all the nixpkgs_* calls in the example's WORKSPACE and see if this helps.

We are working on unifying the CI setup, we've recently ported Linux and macOS to both use GH actions and are also working on covering more tests in all configurations. Unfortunately, we gotten to the examples, yet.

tanyabouman commented 3 years ago

I reproduced the part of the problem that occurs on Ubuntu:

docker run --rm -it ubuntu:20.04 bash
# follow bazel instructions here https://docs.bazel.build/versions/master/install-ubuntu.html
apt update && apt install -y git
apt update && apt install -y python3
git clone https://github.com/tweag/rules_haskell
cd rules_haskell/examples
bazel build --verbose_failures //cat_hs/exec/cat_hs

and fixed it with

apt update && apt install -y build-essential libffi-dev libgmp-dev libtinfo5 libtinfo-dev python python3

since those are required dependencies.

However, it would be really nice if the error was a little more friendly than the current mess, so I will investigate that.