[hiredis] : test_package doesn't honor skip_test and try to run non-host compatible bianry (same for libev)

[ykoehler]

Description

I have a profile for an x86-64 host that is not the same as my build machine, everything works except test_package which then try to run the test binary but my build machine doesn't have the libc version required to do so.

Conan "can_run" still return true, but in my profile I set skip_test to try to fix this problem and the hiredis/test_package and libev/test_package both do not look for skip_test and still try to run the binary which is clearly not compatible.

Package and Environment Details

all of them.

Conan profile

[options] tools.build:skip_test=True

Steps to reproduce

execute conan create on a x86_64 host which is not compatible with the build machine (cross-compilation using different libc version and such)

Logs

Click to expand log

``` 18:36:22 ======== Testing the package: Executing test ======== 18:36:22 hiredis/1.1.0 (test package): Running test() 18:36:22 hiredis/1.1.0 (test package): RUN: ./test_package 18:36:22 ./test_package: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./test_package) ```

[ykoehler]

I modified the recipe with this to get success on my fork for now:

recipes/hiredis/all/test_package/conanfile.py

if can_run(self) and not self.conf.get("tools.build:skip_test", default=False):

[uilianries]

@ykoehler Thank you for reporting that behavior. I'll transfer this issue to Conan client repository, because is more a generic behavior than a specific package problem.

[memsharded]

Hi @ykoehler

Thanks for your feedback. The skip_tests conf applies to:

• CMake generated Presets, defining cache_variables["BUILD_TESTING"] = "OFF", that will affect cmake configure step
• Inhibit execution of cmake.test() and cmake.ctest()
• Inhibit execution of bazel.test()
• Inhibit execution of meson.test()

The conf doesn't apply at all to test_package or any other execution of binaries, like can_run(), as it has the meaning related to unit-test, not to test_package.

If you want to inhibit the execution of the test_package you can do conan create ... -tf=""

[ykoehler]

The problem I have is that test_package is running a executable built for a different environment, in the build environment and fails.

The "can_run()" should have detected that the env is not compatible, but the can_run is not sophisticated enough, I was told by conan team to then use skip_test, but clearly here we see that test_package recipe are not honoring this.

As such, building an cross-compilation package for a host that is close but still not compatible with the build machien is therefore broken.

[ykoehler]

Here are my profiles:

21:36:22  ======== Input profiles ========
21:36:22  Profile host:
21:36:22  [settings]
21:36:22  arch=x86_64
21:36:22  build_type=RelWithDebInfo
21:36:22  compiler=gcc
21:36:22  compiler.libc=glibc
21:36:22  compiler.libc.version=2.34
21:36:22  compiler.libcxx=libstdc++
21:36:22  compiler.version=10.3
21:36:22  os=Linux
21:36:22  os.version=4.14.188
21:36:22  [options]
21:36:22  nanopb/*:shared=True
21:36:22  [conf]
21:36:22  tools.build:sysroot=/usr/my-sysroot
21:36:22  tools.build:skip_test=True
21:36:22  tools.cmake.cmake_layout:build_folder_vars=['settings.os.platform']
21:36:22  tools.gnu:host_triplet=x86_64-other-linux-gnu
21:36:22  [buildenv]
21:36:22  CONAN_CMAKE_FIND_ROOT_PATH=/usr/x86-10.3-glibc-2.34
21:36:22  CONAN_CMAKE_SYSROOT=/usr/my-sysroot
21:36:22  CC=/usr/x86-10.3-glibc-2.34/bin/gcc
21:36:22  CXX=/usr/x86-10.3-glibc-2.34/bin/g++
21:36:22  STRIP=/usr/x86-10.3-glibc-2.34/bin/strip
21:36:22  AR=/usr/x86-10.3-glibc-2.34/bin/ar
21:36:22  AS=/usr/x86-10.3-glibc-2.34/bin/as
21:36:22  LD=/usr/x86-10.3-glibc-2.34/bin/ld
21:36:22  RANLIB=/usr/x86-10.3-glibc-2.34/bin/ranlib
21:36:22  
21:36:22  Profile build:
21:36:22  [settings]
21:36:22  arch=x86_64
21:36:22  build_type=RelWithDebInfo
21:36:22  compiler=gcc
21:36:22  compiler.cppstd=gnu14
21:36:22  compiler.libc=glibc
21:36:22  compiler.libc.version=2.31
21:36:22  compiler.libcxx=libstdc++11
21:36:22  compiler.version=9
21:36:22  os=Linux
21:36:22  os.platform=native
21:36:22  os.version=5.15
21:36:22  [options]
21:36:22  nanopb/*:shared=True
21:36:22  protobuf/*:with-zlib=False
21:36:22  [conf]
21:36:22  tools.cmake.cmake_layout:build_folder_vars=['settings.os.platform']

[ykoehler]

That fact that I used a sysroot, should hint the "can_run" function that this cannot execute a binary locally. Yet got told that skip_test was the way to go, which I do not agree but so far worked ....

[memsharded]

The "can_run()" should have detected that the env is not compatible, but the can_run is not sophisticated enough, I was told by conan team to then use skip_test, but clearly here we see that test_package recipe are not honoring this.

I am afraid that there is some misunderstanding there, feel free to mention here their Github users and bring them to the conversation to clarify. The skip_test is exclusively for unit-test nothing to do with can_run

The correct approach to control can_run behavior is the tools.build.cross_building:can_run configuration. You can easily control whether the can_run will pass or not by adding that conf in command line or in your profile. It is know that it is impossible to automatically detect all possible scenarios where certain binaries can run or not, and thus the explicit approach via conf is the way to go.

[ykoehler]

ok will try that one.