search

bazelbuild / bazel-skylib

Common useful functions and rules for Bazel
https://bazel.build/
Apache License 2.0
387 stars 180 forks source link

native_test and native_binary break shared libraries with remote execution #437

Open cameron-martin opened 1 year ago

cameron-martin commented 1 year ago

In many implementation of remote execution services, the symlink that is created to the executable in native_test is not actually a symlink, and is effectively a copy. For resolving dynamic libraries, bazel uses rpaths relative to $ORIGIN. $ORIGIN refers to the resolved location of the symlink, and so these relative paths work fine when the executable is a symlink. However, when this file is actually a copy these relative rpaths no longer work. Note that the same thing happens when using the experimental hermetic linux sandbox, since this uses hardlinks and copying instead of symlinks.

I have created a reproducer, using the hermetic linux sandbox.

cameron-martin commented 1 year ago

Related discussion: https://bazelbuild.slack.com/archives/CA31HN1T3/p1679661215222949

tetromino commented 1 year ago

Thanks, the slack discussion + linked rules_python issue (https://github.com/bazelbuild/rules_python/issues/691) are very informative. Not sure how to solve this, but I'd classify this as a bug in bazel rather than skylib.

One solution might be along the lines of falling back to a wrapper script if ctx.actions.symlink is a copy and not a symlink - but currently Bazel does not tell us whether that is the case.

Best we can do here for now is update our docs to caution against use of native_test / native_binary ...

cameron-martin commented 8 months ago

One of the bazelcon talks this year was about a cross-platform native launcher, kickoff, which is a potential solution to this.