search

rmohr / bazeldnf

Build multi-arch base containers based on RPM with bazel.
Apache License 2.0
30 stars 18 forks source link

v0.5.9-rc2 breaks tar2files (ldd) targets on crossbuilds #69

Open jschintag opened 9 months ago

jschintag commented 9 months ago

When upgrading to bazeldnf v0.5.9-rc2 cross building for different arches fails to find included header files. I specifically tested this when building kubevirt with https://github.com/kubevirt/kubevirt/pull/10490:

INFO: From ImageLayer cmd/libguestfs/version-container-layer.tar:
Duplicate file in archive: ./etc/passwd, picking first occurrence
ERROR: /root/go/src/kubevirt.io/kubevirt/vendor/libvirt.org/go/libvirt/BUILD.bazel:3:11: GoCompilePkg vendor/libvirt.org/go/libvirt/go_default_library.a failed: (Exit 1): builder failed: error executing command bazel-out/k8-opt-exec-2B5CBBC6/bin/external/go_sdk/builder_reset/builder compilepkg -sdk external/go_sdk -installsuffix linux_s390x -tags selinux,selinux -src vendor/libvirt.org/go/libvirt/callbacks.go ... (remaining 214 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
In file included from vendor/libvirt.org/go/libvirt/connect_helper.h:30,
                 from kubevirt/vendor/libvirt.org/go/libvirt/connect.go:42:
vendor/libvirt.org/go/libvirt/libvirt_generated.h:35:10: fatal error: libvirt/libvirt.h: No such file or directory
   35 | #include <libvirt/libvirt.h>
      |          ^~~~~~~~~~~~~~~~~~~
compilation terminated.
compilepkg: error running subcommand external/go_sdk/pkg/tool/linux_amd64/cgo: exit status 2

This only happens when crosscompiling, most likely because the libraries are otherwise natively installed.

I could verify that the referenced PR works fine with bazeldnf v0.5.8, so the error is introduced somewhere afterwards.

jschintag commented 9 months ago

Same error also occurs when building kubevirt for aarch64 on x86 from the main branch, using bazeldnf v0.5.9-rc2

rmohr commented 9 months ago

My immediate suspicion would be https://github.com/rmohr/bazeldnf/commit/5b26a65c922c580b635cdd303939cfe93a8da59c. @andreabolognani FYI.

rmohr commented 9 months ago

That's the only actual logic change we have in 0.5.9 (and was the reason for not staying on 0.5.8).

andreabolognani commented 9 months ago

I need to finish some stuff before the weekend so I won't have time to dig into this until next week, but from a quick look I don't immediately see an obvious connection between my change and the failure you're experiencing. The list of changes is pretty damning though... :)

If someone else has the time to do it before I get a chance, it would be interesting to compare how v0.5.9-rc1 and v0.5.9-rc2 behave, both when used on KubeVirt's main branch and the s390x branch, when cross-building for aarch64.

rmohr commented 9 months ago

If someone else has the time to do it before I get a chance, it would be interesting to compare how v0.5.9-rc1 and v0.5.9-rc2 behave, both when used on KubeVirt's main branch and the s390x branch, when cross-building for aarch64.

Not sure either if I will find the time, but hope so. A nice scoped problem to solve :)

jschintag commented 9 months ago

If someone else has the time to do it before I get a chance, it would be interesting to compare how v0.5.9-rc1 and v0.5.9-rc2 behave, both when used on KubeVirt's main branch and the s390x branch, when cross-building for aarch64.

I can try out rc-1 today, though it might take a while since i can't use the cache as it might have cached a successful build before (happened already once...)

jschintag commented 9 months ago

rc-1 also has the same error. Though i also don't want to loose https://github.com/rmohr/bazeldnf/commit/5b26a65c922c580b635cdd303939cfe93a8da59c, since i was the one running into the problem in the first place. Since i'm new in the bazeldnf/bazel world, how would the resulting filepath look like? Since the import this time is called libvirt/libvirt.h and not libvirt.h, my theory would be that the resulting filepath relative from the import path is not <bazeldnf-folder>/libvirt/libvirt.h, but maybe something more like the full path <bazeldnf-folder>/usr/include/libvirt/libvirt.h? As i said, i'm not that familiar with this, this is just my uneducated guess.

andreabolognani commented 9 months ago

i can't use the cache as it might have cached a successful build before (happened already once...)

Yeah, it's always a good idea to clean everything up when testing this kind of stuff. I've been bitten by that before.

I'm also looking into it now, but can you please give a summary of what works and what doesn't based on your own testing? That'll speed up things a bit.

jschintag commented 9 months ago

Works:

I used the kubevirt main branch and cross-compiling aarch64 for all tests.

andreabolognani commented 9 months ago

Okay. I assume native compilation worked though?

jschintag commented 9 months ago

Yes, native worked. But that is most likely because the builder image has libvirt-dev installed. So it does not need the includes from bazeldnf

andreabolognani commented 9 months ago

Right. Ideally that wouldn't be present but IIUC KubeVirt wants to also be buildable without Bazel. Pretty silly IMO but I don't make the rules :)

andreabolognani commented 9 months ago

I have figured out the issue. More details here.

jschintag commented 9 months ago

Since v0.5.9 is now released, i think we can close this. The only question is if we should document the needed changes somewhere? Though i have looked through the README.md and there is no description of how the entry in BUILD.bazel is created. Is it auto-generated?

@rmohr I can add this change to the docs if you point me to where it should be added.

rmohr commented 9 months ago

The only question is if we should document the needed changes somewhere? Though i have looked through the README.md and there is no description of how the entry in BUILD.bazel is created. Is it auto-generated?

Not sure wich entries specifically you refer to. Some are auto-update (which contain the version). The rest is just written into the README.md.

jschintag commented 9 months ago

The only question is if we should document the needed changes somewhere? Though i have looked through the README.md and there is no description of how the entry in BUILD.bazel is created. Is it auto-generated?

Not sure wich entries specifically you refer to. Some are auto-update (which contain the version). The rest is just written into the README.md.

In https://github.com/kubevirt/kubevirt/pull/11106, the changes to BUILD.bazel