Open eirikb opened 2 years ago
Hi @eirikb
Indeed a non expected case.
As Conan has no further information so far, it will use only os-arch to decide.
But it is not only a problem of the cross_building
method, but it can be bigger. For example, the package_id
could be colliding. If all settings and options are exactly the same, the package_id
is the same and the binary will be consider the same (and binary compatible) among them. But I think it is not the case, the binary built for Yocto won't run on the host system and viceversa.
I think something else is necessary into the model to dissambiguate between those builds/binaries. What would be the variable thing in the packages? Something in the Yocto built image? Versioning or classifying the Linux distro?
I'm not very well traversed in the land of Yocto, but the toolchain files at least seem to contain pieces such as;
All of these variables might end up creating different binaries.
I think it is possible to distinguish binaries in cache today by introducing extra parameters in settings.yml
.
Since I build using docker with custom profiles this should be straight forward and contained to the specific toolchains.
This won't help with cross compiling though.
When building with bitbake
these parameters will show:
BB_VERSION = "1.46.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "x86_64-poky-linux"
MACHINE = "qemux86-64"
DISTRO = "poky"
DISTRO_VERSION = "3.1.14"
TUNE_FEATURES = "m64 core2"
TARGET_FPU = ""
Any combination of these might create a different binary, at least the ones not relevant to build system. This is interesting when it comes to binaries and cache, however for my current issue cross compiling is the problem. Some way to force this in Conan would be great.
Since I have a docker image for the sole purpose of cross compiling I decided to test out some hacks based on my findings from my initial test.
Note that this is very hacky, only for testing purposes, and it is done in a toolchain-purposed docker image.
# This will force Conan to always consider the build as cross compiling
RUN sed -i '/def cross_build/a \ return True' /usr/local/lib/python3.8/dist-packages/conans/client/tools/oss.py
# This will trick automake configure to consider the build cross compile
RUN sed -i '/self.build,/a \ self.build = "x86_64-linux"' /usr/local/lib/python3.8/dist-packages/conans/client/build/autotools_environment.py
I'm not sure if that last one is a problem with automake or the library, or something else.
Even without Conan, just trying to build the library from github using the toolchain it won't build, with the same errors.
I'm not sure how Yocto builds this library as I can't find the recipe.
With the changes above I can successfully run conan install
on libbacktrace
, and I can run conan create
on boost
which depends on this libbacktrace.
Running conan create
on libbacktrace
gave compilation error in test_package
. I'm not sure why, I believe this worked in my previous test.
Hi,
what's the status? Is there a feature for the autotools build helper (e.g. by forcing the --host
argument in the profile/configuration) planned?
I'm also facing an issue when "cross-building" from x86_64 Debian10 (build server, default
profile) to x86_64 Debian12 (target) when building the libxml2 library:
$conan create recipes/libxml2/all libxml2/2.11.4@ -u -tf None -pr debian6412 -pr:b default -s:b build_type=RelWithDebInfo -o shared=True -o iconv=False -o zlib:shared=True -s build_type=RelWithDebInfo
ERROR:
configure:4252: ./conftest
./conftest: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./conftest)
configure:4256: $? = 1
configure:4263: error: in `/home/thomas/.conan/data/libxml2/2.11.4/_/_/build/f0cb5ad7a4a12e1d84218fb638deb2fa5fe11a47/build-relwithdebinfo':
configure:4265: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
see also https://github.com/conan-io/conan/issues/11437
Here is my current workaround: https://github.com/conan-io/conan-center-index/commit/e207c0b94c868ff19cdaedb334dbbb70253e159f Is there a better way? I think it would also work to update my build server to Debian12. UPDATE: I now use a Debian12 build-container, it compiles as expected.
Bumping because I ran into the exact same issue when building for Yocto x64 on an Ubuntu x64 host. Interestingly everything worked and I didn't notice that Conan thought it's not cross-compiling, until I added gRPC as a dependency.
gRPC needs a runnable grpc_cpp_plugin
binary on the build machine, and the gRPC recipe would indeed install "itself" for the build environment if cross_building()
would return True
. I also tried the hack mentioned above, patching Conan to always return True
for cross_building()
but then I ran into a "loop/cycle detected error" and gave up.
So I guess there are two issues here:
cross_building()
helper and have no hand-rolled detection mechanisms.os.distro
subsetting, just like in the examples. Maybe it's worth to add a remark to the cross-compilation documentation about this.With the emerge of ARM laptops this can become a more common issue, e.g. when building from an ARM Linux laptop to Raspberry.
Also note that there is a possible duplicate issue: https://github.com/conan-io/conan/issues/9808
Bump. Ran into an issue with the benchmark
library on a Yocto x86_64 toolchain where the generated executables cannot be run on the build system (different ld loader naming convention). What about a conf that forces cross_building()
to true? We could set that in the respective profile.
As strange as it sounds I'm trying to cross compile from Linux x86_64 to Linux x86_64, using a Yocto SDK Toolchain.
I use two profiles (
-pr:b
and-pr:h
).A lot of recipes work out-of-the-box, but not all. Currently struggling with libbacktrace (because of boost). It uses autotools.
Here is output of building with two Yocto toolchains, one from x86_64 to armv7hf and one from x86_64 to x86_64: https://github.com/eirikb/proof-of-conan/runs/5742649170?check_suite_focus=true The one failing will say:
The
--host
(and--target
) is not passed.This seems logical enough, and the check in https://github.com/conan-io/conan/blob/develop/conans/client/tools/oss.py#L460 will check if
host_os
orhost_arch
is not likebuild_os
orbuild_arch
.I'm not sure how to solve this, perhaps some additional flag somewhere? Maybe it is already possible? As I understand using two profiles by itself doesn't necessarily mean cross building.
In addition I did some tests, by forcing
cross_building
in oss.py to always returnTrue
.Then
--host
and--target
would be passed, but automake didn't consider it cross build. I'm not 100% sure why not, could be a similar arch vs arch check? The parameters were _x8664-linux-gnu . If I changed these, to for example _x8664-linux it would cross build, even when they were equal. I did this directly in autotools_environment.py.