search

tweag / rules_haskell

Haskell rules for Bazel.
https://haskell.build
Apache License 2.0
266 stars 81 forks source link

can't build haskell_binary on mac #461

Closed shmish111 closed 6 years ago

shmish111 commented 6 years ago 
ld: file too small (length=0) file 'bazel-out/darwin-fastbuild/bin/external/io_tweag_rules_haskell/haskell/libempty.a' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
`cc' failed in phase `Linker'. (Exit code: 1)

so presumably libempty is supposed to be an empty file but ld on a mac won’t do it.

guibou commented 6 years ago

Could you try with master? I thought this was fixed in a recent commit.

Is there any other warning / error message earlier in the log? This was mostly due to a missing binary on darwin.

shmish111 commented 6 years ago

Here is the log using master's HEAD:

/nix/store/yziylmm7zinsgx7xd6fwf1ma0jczhjgi-bazel-0.17.1/bin/bazel build //...
INFO: Repository rule 'io_tweag_rules_haskell' returned: {"remote": "https://github.com/tweag/rules_haskell.git", "commit": "254a99edd646fa13601a7ab6a6d2c04a205260f8", "shallow_since": "2018-10-22", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "io_tweag_rules_haskell"}
INFO: Build options have changed, discarding analysis cache.
DEBUG: /private/var/tmp/_bazel_davidsmith/43cbe53c8c4fdd60fc53c47eee82daa8/external/bazel_tools/tools/osx/xcode_configure.bzl:87:9: Invoking xcodebuild failed, developer dir: /Users/davidsmith/Downloads/Xcode.app/Contents/Developer ,return code 256, stderr: Process terminated by signal 6, stdout:
/nix/store/b3fwgyla7vfcl00h6s474dzmifd0wcwh-happy-1.19.9
/nix/store/484w7d1vxb774km0yd6x6wn4hmll8mqq-alex-3.2.3
DEBUG: /private/var/tmp/_bazel_davidsmith/43cbe53c8c4fdd60fc53c47eee82daa8/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl:69:5:
WARNING: Implicitly using '<nixpkgs>' as the location of Nixpkgs.
This is not recommended because it makes the build non-hermetic.
Set which Nixpkgs to use explicitly using 'repository' or 'path' attributes.

/nix/store/zk01hssgc3nvzypgbq8lr2l8lvs4jbjs-ghc-8.2.2-with-packages
INFO: Analysed 34 targets (22 packages loaded).
INFO: Found 34 targets...
ERROR: /Users/davidsmith/tweag/plutus-bazel/language-plutus-core/BUILD.bazel:93:1: error executing shell command: '/nix/store/8p5xi4fswmpxly0glir3r4jzy2r3rrlc-bash/bin/bash -c ${1+"$@"} $(< bazel-out/darwin-fastbuild/bin/language-plutus-core/_obj/language-plutus-core-generate-evaluation-test.manifest)  bazel-ou...' failed (Exit 1)
ld: file too small (length=0) file 'bazel-out/darwin-fastbuild/bin/language-plutus-core/libHSlanguage-plutus-coreZSlanguage-plutus-core.a' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
`cc' failed in phase `Linker'. (Exit code: 1)
INFO: Elapsed time: 16.122s, Critical Path: 0.33s
INFO: 1 process: 1 darwin-sandbox.
FAILED: Build did NOT complete successfully
shmish111 commented 6 years ago

I ran bazel clean before I ran that.

shmish111 commented 6 years ago

Also, I upgraded to OSX 10.14 2 days ago, probably a bad idea

mboes commented 6 years ago

Do you have a minimal repro?

shmish111 commented 6 years ago

Well, I tried rules_haskell_example and interestingly got two different errors. FYI I do have xcode-select installed and on my path.

INFO: Analysed 11 targets (13 packages loaded).
INFO: Found 11 targets...
INFO: From Compiling rts/main.c:
rts/main.c:8:54: warning: format specifies type 'long' but the argument has type 'HsInt' (aka 'long long') [-Wformat]
  printf("Adding one to 5 through Haskell is %ld\n", add_one_hs(5));
                                             ~~~     ^~~~~~~~~~~~~
                                             %lld
1 warning generated.
INFO: From HaskellBuildLibrary //rts:add-one-hs:
clang: warning: argument unused during compilation: '-nopie' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-nopie' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-nopie' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-nopie' [-Wunused-command-line-argument]
INFO: From HaskellLinkStaticLibrary rts/libHSrtsZSadd-one-hs-1.0.0.a:
/private/var/tmp/_bazel_davidsmith/ed11f590a2813de56c6ba72df5bf054b/sandbox/darwin-sandbox/10/execroot/io_tweag_rules_haskell_examples/bazel-out/host/bin/visible-binaries/xcrunwrapper.sh: line 33: xcode-select: command not found
ERROR: /Users/davidsmith/tweag/rules_haskell_examples/rts/BUILD.bazel:21:1: Linking of rule '//rts:add-one' failed (Exit 1)
/nix/store/a10rrqfq8yz3sdv5vlzvss0b3z8l9vs9-cctools-port-895/bin/install_name_tool: object: bazel-out/darwin-fastbuild/bin/rts/add-one malformed object (unknown load command 9)
INFO: Elapsed time: 22.068s, Critical Path: 1.66s
INFO: 12 processes: 12 darwin-sandbox.
FAILED: Build did NOT complete successfully

Interestingly when I run it a second time I don't get the xcode-select error:

➜  rules_haskell_examples git:(master) /nix/store/yziylmm7zinsgx7xd6fwf1ma0jczhjgi-bazel-0.17.1/bin/bazel build //...
DEBUG: /private/var/tmp/_bazel_davidsmith/ed11f590a2813de56c6ba72df5bf054b/external/bazel_tools/tools/osx/xcode_configure.bzl:87:9: Invoking xcodebuild failed, developer dir: /Users/davidsmith/Downloads/Xcode.app/Contents/Developer ,return code 256, stderr: Process terminated by signal 6, stdout:
INFO: Analysed 11 targets (1 packages loaded).
INFO: Found 11 targets...
INFO: From Compiling rts/main.c:
rts/main.c:8:54: warning: format specifies type 'long' but the argument has type 'HsInt' (aka 'long long') [-Wformat]
  printf("Adding one to 5 through Haskell is %ld\n", add_one_hs(5));
                                             ~~~     ^~~~~~~~~~~~~
                                             %lld
1 warning generated.
ERROR: /Users/davidsmith/tweag/rules_haskell_examples/rts/BUILD.bazel:21:1: Linking of rule '//rts:add-one' failed (Exit 1)
/nix/store/a10rrqfq8yz3sdv5vlzvss0b3z8l9vs9-cctools-port-895/bin/install_name_tool: object: bazel-out/darwin-fastbuild/bin/rts/add-one malformed object (unknown load command 9)
INFO: Elapsed time: 0.445s, Critical Path: 0.25s
INFO: 6 processes: 6 darwin-sandbox.
FAILED: Build did NOT complete successfully
shmish111 commented 6 years ago

after changing rules_haskell to use the latest master, I get different errors again:

➜  rules_haskell_examples git:(master) ✗ /nix/store/yziylmm7zinsgx7xd6fwf1ma0jczhjgi-bazel-0.17.1/bin/bazel build //...
INFO: Repository rule 'io_tweag_rules_haskell' returned: {"remote": "https://github.com/tweag/rules_haskell.git", "commit": "02b74b5e43ef522b409ec8efec5709cee7da48e2", "shallow_since": "2018-10-24", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "io_tweag_rules_haskell"}
INFO: Build options have changed, discarding analysis cache.
/nix/store/x8sf53k365gx28jms7jssfg4qbwwf3va-ghc-8.2.2-with-packages
DEBUG: /private/var/tmp/_bazel_davidsmith/ed11f590a2813de56c6ba72df5bf054b/external/bazel_tools/tools/osx/xcode_configure.bzl:87:9: Invoking xcodebuild failed, developer dir: /Users/davidsmith/Downloads/Xcode.app/Contents/Developer ,return code 256, stderr: Process terminated by signal 6, stdout:
INFO: Analysed 11 targets (17 packages loaded).
INFO: Found 11 targets...
ERROR: /Users/davidsmith/tweag/rules_haskell_examples/vector/BUILD.bazel:14:1: error executing shell command: '/nix/store/8p5xi4fswmpxly0glir3r4jzy2r3rrlc-bash/bin/bash -c ${1+"$@"}  bazel-out/host/bin/visible-binaries/ghc -pgma bazel-out/host/bin/visible-binaries/cc -pgmc bazel-out/host/bin/visible-binarie...' failed (Exit 1)

vector/Data/Vector.hs:194:5: error:
     error: function-like macro 'MIN_VERSION_base' is not defined
    |
194 | #if MIN_VERSION_base(4,9,0)
    |     ^
#if MIN_VERSION_base(4,9,0)
    ^

vector/Data/Vector.hs:207:22: error:
     error: token is not a valid binary operator in a preprocessor subexpression
    |
207 | #if !MIN_VERSION_base(4,8,0)
    |                      ^
#if !MIN_VERSION_base(4,8,0)
    ~~~~~~~~~~~~~~~~~^

vector/Data/Vector.hs:235:5: error:
     error: function-like macro 'MIN_VERSION_base' is not defined
    |
235 | #if MIN_VERSION_base(4,9,0)
    |     ^
#if MIN_VERSION_base(4,9,0)
    ^

vector/Data/Vector.hs:308:5: error:
     error: function-like macro 'MIN_VERSION_base' is not defined
    |
308 | #if MIN_VERSION_base(4,9,0)
    |     ^
#if MIN_VERSION_base(4,9,0)
    ^

vector/Data/Vector.hs:392:5: error:
     error: function-like macro 'MIN_VERSION_base' is not defined
    |
392 | #if MIN_VERSION_base(4,6,0)
    |     ^
#if MIN_VERSION_base(4,6,0)
    ^

vector/Data/Vector.hs:400:5: error:
     error: function-like macro 'MIN_VERSION_base' is not defined
    |
400 | #if MIN_VERSION_base(4,8,0)
    |     ^
#if MIN_VERSION_base(4,8,0)
    ^
6 errors generated.
`cc' failed in phase `C pre-processor'. (Exit code: 1)
INFO: Elapsed time: 4.712s, Critical Path: 0.23s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
shmish111 commented 6 years ago

ok

  1. clone current rules_haskell_example
  2. cd tutorial
  3. change io_tweag_rules_haskell in WORKSPACE of tutorial to be a git_repository of the master branch
  4. bazel build //...

I get ld: file too small (length=0) file

shmish111 commented 6 years ago 
➜  tutorial git:(master) ✗ /nix/store/yziylmm7zinsgx7xd6fwf1ma0jczhjgi-bazel-0.17.1/bin/bazel build //...
INFO: Repository rule 'io_tweag_rules_haskell' returned: {"remote": "https://github.com/tweag/rules_haskell.git", "commit": "02b74b5e43ef522b409ec8efec5709cee7da48e2", "shallow_since": "2018-10-24", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "io_tweag_rules_haskell"}
INFO: SHA256 (https://github.com/bazelbuild/bazel-skylib/archive/0.2.0.tar.gz) = 981624731d7371061ca56d532fa00d33bdda3e9d62e085698880346a01d0a6ef
DEBUG: /private/var/tmp/_bazel_davidsmith/3cc8afbc22f7f82bbf26e37151ef60ae/external/bazel_tools/tools/osx/xcode_configure.bzl:87:9: Invoking xcodebuild failed, developer dir: /Users/davidsmith/Downloads/Xcode.app/Contents/Developer ,return code 256, stderr: Process terminated by signal 6, stdout:
/nix/store/qsfhacppm13v295jmjn8fv4pxjdy0wxd-ghc-8.2.2
INFO: Analysed 4 targets (20 packages loaded).
INFO: Found 4 targets...
INFO: From HaskellLinkStaticLibrary lib/libHSlibZSbooleans.a:
/private/var/tmp/_bazel_davidsmith/3cc8afbc22f7f82bbf26e37151ef60ae/sandbox/darwin-sandbox/12/execroot/tutorial/bazel-out/host/bin/visible-binaries/xcrunwrapper.sh: line 33: xcode-select: command not found
ERROR: /Users/davidsmith/tweag/rules_haskell_examples/tutorial/main/BUILD.bazel:3:1: error executing shell command: '/nix/store/8p5xi4fswmpxly0glir3r4jzy2r3rrlc-bash/bin/bash -c ${1+"$@"} $(< bazel-out/darwin-fastbuild/bin/main/_obj/demorgan.manifest)  bazel-out/host/bin/visible-binaries/ghc -pgma bazel-out/host/...' failed (Exit 1)
ld: file too small (length=0) file 'bazel-out/darwin-fastbuild/bin/lib/libHSlibZSbooleans.a' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
`cc' failed in phase `Linker'. (Exit code: 1)
INFO: Elapsed time: 15.561s, Critical Path: 2.07s
INFO: 17 processes: 17 darwin-sandbox.
FAILED: Build did NOT complete successfully
guibou commented 6 years ago

Hello,

I'm working on your issue.

Your issue about macro versions can be fixed by setting a version flag to your rule. We should document that, see #465.

Your issue about empty libempty can be fixed by upgrading to a more recent rules_haskell version. However this will just move the issue and, as you observed, you will just get another empty libSomething. The issue is that rules_haskell does not know which apple sdk is needed, hence it calls tools without setting the apple sdk and theses tools fail when creating the archive. Something changed in that matter in recent bazel version, this is why this issue only appears now.

The situation now is that rules_haskell calls xcrunwrapper.sh without setting the correct SDKROOT and DEVELOPER_DIR environment variables. xcrunwrapper.sh tries to get the value from xcode-select which is not available in the sandbox, resulting in error. However the failure of xcrunwrapper.sh is not taken into account by rules_haskell which continues it works, resulting in empty archives which will be used later. Solving bug #466 will lead to a more detailed error.

Many options to solve your issue:

This is taking time because I'm not at all used to darwin and, until yesterday afternoon, had no access to a darwin machine except the one from the continuous integration. Then I hit two other darwin related bugs in rules_haskell when setting up a configuration in which your issue appears.

judah commented 6 years ago

Another possibility might be to try to bypass the xcode toolchain with BAZEL_USE_CPP_ONLY_TOOLCHAIN=1. I recall we set that flag at Formation, and also in the tests of the Hazel repo: https://github.com/FormationAI/hazel/blob/925293994f88799ba550fd5cf3995104d1f2972c/tools/bazel.rc#L2

I'm not certain whether it would resolve your issue, though, given that it sounds like bazel has changed its behavior recently.

shmish111 commented 6 years ago

oh, that worked @judah !

The tests compile and the binary runs. unfortunately the tests fail for some reason, I think to do with where things are run from and referenced, but that's for another day.

So is this a hack? what does it actually do?

judah commented 6 years ago

See: https://github.com/bazelbuild/bazel/issues/4231 (and links from that page)

guibou commented 6 years ago

Merged #453 and the usage of BAZEL_USE_CPP_ONLY_TOOLCHAIN=1 should fix your issue. See #469 if you have any issue for the environment variable. I'm considering this issue done, feel free to reopen if this is not the case for you.