search

tweag / clodl

Turn dynamically linked ELF binaries and libraries into self-contained closures.
BSD 3-Clause "New" or "Revised" License
164 stars 6 forks source link

Migrate to bzlmod #56

Open hofbi opened 3 weeks ago

hofbi commented 3 weeks ago

Migrate the main project to bzlmod and rename all BUILD files to BUILD.bazel

Closes https://github.com/tweag/clodl/issues/53 Closes https://github.com/tweag/clodl/issues/54

hofbi commented 3 weeks ago

CI probably fails due to some older version of bazel which is used in the nixpkgs, but here I might need your input on which version of packages you want to use.

avdv commented 3 weeks ago

CI probably fails due to some older version of bazel which is used in the nixpkgs, but here I might need your input on which version of packages you want to use.

Thank you, @hofbi. That looks good! I'll update nixpkgs and push my changes in a moment...

hofbi commented 3 weeks ago

Sounds good, thanks.

What do you guys think about using bazelisk to include bazel? This way you can easily control it from the version file and it is the recommended way for installing bazel. But definitely a follow up PR in case we wanna do that.

Another task would also be to convert the tests to bzlmod.

aherrmann commented 3 weeks ago

The tests include some Nix components and might depend on some of the patches made to Bazel in nixpkgs.

hofbi commented 3 weeks ago

The tests include some Nix components and might depend on some of the patches made to Bazel in nixpkgs.

Should we also migrate the tests to bzlmod in this PR?

avdv commented 3 weeks ago

Should we also migrate the tests to bzlmod in this PR?

The tests use nixpkgs_python_configure which is part of rules_nixpkgs_python and that is currently not available from the BCR. Same situation for rules_haskell_nixpkgs. I'd say let's leave it at that for now.

hofbi commented 3 weeks ago

So are we good to merge? I can play arround with a follow up PR for the tests including the other deps as non module dependencies

avdv commented 3 weeks ago

So are we good to merge?

CI is still failing. :-(

I can play arround with a follow up PR for the tests including the other deps as non module dependencies

That would be awesome!

avdv commented 3 weeks ago

CI run on ubuntu is now green, but macos is failing with:

Repository rule cc_autoconf defined at:
  /private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/cc_configure.bzl:109:30: in <toplevel>
ERROR: An error occurred during the fetch of repository 'bazel_tools~cc_configure_extension~local_config_cc':
   Traceback (most recent call last):
    File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/cc_configure.bzl", line 91, column 33, in cc_autoconf_impl
        configure_unix_toolchain(repository_ctx, cpu_value, overriden_tools)
    File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 387, column 46, in configure_unix_toolchain
        overriden_tools["ar"] = _find_generic(repository_ctx, "libtool", "LIBTOOL", overriden_tools)
    File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 310, column 32, in _find_generic
        auto_configure_fail(msg)
    File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 112, column 9, in auto_configure_fail
        fail("\n%sAuto-Configuration Error:%s %s\n" % (red, no_color, msg))
Error in fail: 
Auto-Configuration Error: Cannot find libtool or LIBTOOL; either correct your path or set the LIBTOOL environment variable
ERROR: <builtin>: fetching cc_autoconf rule //:bazel_tools~cc_configure_extension~local_config_cc: Traceback (most recent call last):
    File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/cc_configure.bzl", line 91, column 33, in cc_autoconf_impl
        configure_unix_toolchain(repository_ctx, cpu_value, overriden_tools)
    File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 387, column 46, in configure_unix_toolchain
        overriden_tools["ar"] = _find_generic(repository_ctx, "libtool", "LIBTOOL", overriden_tools)
    File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 310, column 32, in _find_generic
        auto_configure_fail(msg)
    File "/private/var/tmp/_bazel_runner/597fade3707a30aa94c2bd004f76cc2b/external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 112, column 9, in auto_configure_fail
        fail("\n%sAuto-Configuration Error:%s %s\n" % (red, no_color, msg))
Error in fail: 
Auto-Configuration Error: Cannot find libtool or LIBTOOL; either correct your path or set the LIBTOOL environment variable
aherrmann commented 3 weeks ago

https://github.com/bazelbuild/bazel/commit/3c47b4716e6eca2b8ab06253915ef8d1bdd28038 seems to be relevant to the LIBTOOL issue. I've tried out a Nix provided cc toolchain to avoid this issue here. With that the MacOS build proceeds further, but fails with

ERROR: /Volumes/External/Users/andreas/clodl/tests/BUILD.bazel:109:15: Action clotestbin-cc-closure.zip failed: (Terminated): bash failed: error executing Action command (from target //:clotestbin-cc-closure) /nix/store/wb7cjcvaa31s5saxi7z24qwhq709hbhz-bash/bin/bash -c ... (remaining 4 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
otool: error: 'dyld[60391]: <no uuid> /private/var/tmp/_bazel_andreas/5a72690ff69d3fb0757d7797a2268561/execroot/io_tweag_clodl_tests/bazel-out/darwin_arm64-opt-exec-ST-467dab1ad9da/bin/external/io_tweag_clodl/loader': No such file or directory
Terminated: 15
hofbi commented 2 weeks ago

bazelbuild/bazel@3c47b47 seems to be relevant to the LIBTOOL issue. I've tried out a Nix provided cc toolchain to avoid this issue here. With that the MacOS build proceeds further, but fails with

ERROR: /Volumes/External/Users/andreas/clodl/tests/BUILD.bazel:109:15: Action clotestbin-cc-closure.zip failed: (Terminated): bash failed: error executing Action command (from target //:clotestbin-cc-closure) /nix/store/wb7cjcvaa31s5saxi7z24qwhq709hbhz-bash/bin/bash -c ... (remaining 4 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
otool: error: 'dyld[60391]: <no uuid> /private/var/tmp/_bazel_andreas/5a72690ff69d3fb0757d7797a2268561/execroot/io_tweag_clodl_tests/bazel-out/darwin_arm64-opt-exec-ST-467dab1ad9da/bin/external/io_tweag_clodl/loader': No such file or directory
Terminated: 15

I found https://github.com/envoyproxy/envoy-filter-example/issues/121. From there I get the impression that XCode is required to be installed for some reason.

facundominguez commented 2 weeks ago

otool: error: 'dyld[60391]: /private/var/tmp/_bazel_andreas/5a72690ff69d3fb0757d7797a2268561/execroot/io_tweag_clodl_tests/bazel-out/darwin_arm64-opt-exec-ST-467dab1ad9da/bin/external/io_tweag_clodl/loader': No such file or directory

I'm guessing loader refers to this helper binary.