[question] [system_tools] is ignored

[Roman-Byshliaha-Bose]

What is your question?

My package depends on gRPC, but there is no pre-built package (which is ok since I want to build it locally). It has dependencies on openssl and zlib, which are present in my toolchain and I am trying to avoid fetching&building them by using [system_tools] inside my profile file (especially taking into account openssl build fails to be built using qcc for QNX). But seems like the section gets ignored and "conan install" still result in building everything:

======== Computing dependency graph ======== grpc/1.54.3: Not found in local cache, looking in remotes... grpc/1.54.3: Checking remote: conancenter grpc/1.54.3: Downloaded recipe revision 3e0d68cd1578502c9c3a0e8366f0ab77 abseil/20230125.3: Not found in local cache, looking in remotes... abseil/20230125.3: Checking remote: conancenter abseil/20230125.3: Downloaded recipe revision 5431a4c609f5fb48bb8d8567e953243f c-ares/1.19.1: Not found in local cache, looking in remotes... c-ares/1.19.1: Checking remote: conancenter c-ares/1.19.1: Downloaded recipe revision 420a0b77e370f4b96bee88ef91837ccc openssl/3.1.2: Not found in local cache, looking in remotes... openssl/3.1.2: Checking remote: conancenter openssl/3.1.2: Downloaded recipe revision ef79f25fe0968d69538feca6a6b3bc7f zlib/1.2.13: Not found in local cache, looking in remotes... zlib/1.2.13: Checking remote: conancenter zlib/1.2.13: Downloaded recipe revision e377bee636333ae348d51ca90874e353 re2/20230301: Not found in local cache, looking in remotes... re2/20230301: Checking remote: conancenter re2/20230301: Downloaded recipe revision 56bcddd1eaca2b093fd34525ae40ee9b protobuf/3.21.12: Not found in local cache, looking in remotes... protobuf/3.21.12: Checking remote: conancenter protobuf/3.21.12: Downloaded recipe revision d9f5f4e3b86552552dda4c0a2e928eeb cmake/3.27.1: Not found in local cache, looking in remotes... cmake/3.27.1: Checking remote: conancenter cmake/3.27.1: Downloaded recipe revision 7d5f14447c023feef9f59640be53619a Graph root conanfile.py (Application/0.0.1): /workspace/conanfile.py Requirements abseil/20230125.3#5431a4c609f5fb48bb8d8567e953243f - Downloaded (conancenter) c-ares/1.19.1#420a0b77e370f4b96bee88ef91837ccc - Downloaded (conancenter) grpc/1.54.3#3e0d68cd1578502c9c3a0e8366f0ab77 - Downloaded (conancenter) openssl/3.1.2#ef79f25fe0968d69538feca6a6b3bc7f - Downloaded (conancenter) protobuf/3.21.12#d9f5f4e3b86552552dda4c0a2e928eeb - Downloaded (conancenter) re2/20230301#56bcddd1eaca2b093fd34525ae40ee9b - Downloaded (conancenter) zlib/1.2.13#e377bee636333ae348d51ca90874e353 - Downloaded (conancenter) Build requirements abseil/20230125.3#5431a4c609f5fb48bb8d8567e953243f - Downloaded (conancenter) c-ares/1.19.1#420a0b77e370f4b96bee88ef91837ccc - Downloaded (conancenter) cmake/3.27.1#7d5f14447c023feef9f59640be53619a - Downloaded (conancenter) grpc/1.54.3#3e0d68cd1578502c9c3a0e8366f0ab77 - Downloaded (conancenter) openssl/3.1.2#ef79f25fe0968d69538feca6a6b3bc7f - Downloaded (conancenter) protobuf/3.21.12#d9f5f4e3b86552552dda4c0a2e928eeb - Downloaded (conancenter) re2/20230301#56bcddd1eaca2b093fd34525ae40ee9b - Downloaded (conancenter) zlib/1.2.13#e377bee636333ae348d51ca90874e353 - Downloaded (conancenter) Resolved version ranges cmake/[>=3.22]: cmake/3.27.1 openssl/[>=1.1 <4]: openssl/3.1.2

======== Computing necessary packages ======== abseil/20230125.3: Checking 3 compatible configurations: abseil/20230125.3: '86fcbc6f9328921ad53d0c66af8227848fa5336f': compiler.cppstd=gnu14 abseil/20230125.3: '63845af80758a53b0562a78e00375a686aa9a9b0': compiler.cppstd=17 abseil/20230125.3: 'b4dd6b64486ea9fc984eb567907be24bce56b5eb': compiler.cppstd=gnu17 abseil/20230125.3: Checking 7 compatible configurations: abseil/20230125.3: 'c01616259c2d166eadec282346c9fb6dce0e3530': compiler.cppstd=14 abseil/20230125.3: '99c9bcacb68e3379fad0ffaab3c0268baafd0cd2': compiler.cppstd=gnu14 abseil/20230125.3: '13be611585c95453f1cbbd053cea04b3e64470ca': compiler.cppstd=17 abseil/20230125.3: Main binary package '8631cf963dbbb4d7a378a64a6fd1dc57558bc2fe' missing. Using compatible package '13be611585c95453f1cbbd053cea04b3e64470ca' re2/20230301: Checking 5 compatible configurations: re2/20230301: 'fbbc29c59c5b7c9fc95c46efe865ba853bdb5a79': compiler.cppstd=11 re2/20230301: '62857aa27f40e0f3e236bab6777895621a03de99': compiler.cppstd=gnu11 re2/20230301: '8104e209f50c9ed379b9f1bcfca4dbe3ec5fd719': compiler.cppstd=gnu14 re2/20230301: '13eeea4a03b9b3ce9456a939f4674c1077b2fe73': compiler.cppstd=17 re2/20230301: '1ad8dcf9d992ee0ff0762dc1ac65344cae65e8a0': compiler.cppstd=gnu17 re2/20230301: Checking 9 compatible configurations: re2/20230301: '6bbd4c92174be78d0cd34897be671d1a14d65b19': compiler.cppstd=11 re2/20230301: 'df0fc4003c937b1a78f3e7477c18b172862d5d37': compiler.cppstd=gnu11 re2/20230301: 'f01398aca5a4421fb0b9532b4b482baefcc1b97a': compiler.cppstd=14 re2/20230301: 'e91be93c00bf6b096620928b358da16c01dc2075': compiler.cppstd=gnu14 re2/20230301: '81d35562505eb9186290237bea6e2684b0ba3cc8': compiler.cppstd=17 re2/20230301: Main binary package '2071b77a5967e89bf82bb707e63d635b7110bc9c' missing. Using compatible package '81d35562505eb9186290237bea6e2684b0ba3cc8' protobuf/3.21.12: Checking 7 compatible configurations: protobuf/3.21.12: 'a9c7924dc50af10e7f66785281c207b5869f0240': compiler.cppstd=98 protobuf/3.21.12: '8bf44f285166f699a2fbdbe540bf9c823258f135': compiler.cppstd=gnu98 protobuf/3.21.12: 'eb08d219c007e7ea18338d8e22217a7d0960e536': compiler.cppstd=11 protobuf/3.21.12: 'bdd13251da01e145d6230642e8fba60f50b78afd': compiler.cppstd=gnu11 protobuf/3.21.12: 'a31b39ed55e959d2a0f4b0c90810fe61be88a85f': compiler.cppstd=gnu14 protobuf/3.21.12: 'cf5e5939c98c6c500d801d5d36f81104800c0cca': compiler.cppstd=17 protobuf/3.21.12: '02785f639795b87b69380a3a014ed763aa33e362': compiler.cppstd=gnu17 protobuf/3.21.12: Checking 11 compatible configurations: protobuf/3.21.12: 'c66f42c1868bda5ae02fb2f9af8f2741123b1455': compiler.cppstd=98 protobuf/3.21.12: '84720d97f0737b4e8d8e2a9eb86f8e94cec7514f': compiler.cppstd=gnu98 protobuf/3.21.12: '03ae1d1eef1e58ddb77efcb28ae319502930be4b': compiler.cppstd=11 protobuf/3.21.12: '0d98878c1b85e50b6d218c636ac057e7bfb2763b': compiler.cppstd=gnu11 protobuf/3.21.12: 'ac07a349203a320c6c914034f9298d17ac240bf9': compiler.cppstd=14 protobuf/3.21.12: '8df3afc48e41f2f00e10820b0736936225d9aeef': compiler.cppstd=gnu14 protobuf/3.21.12: '0a68446b9947784d37544a468d7db3ff36a7d85d': compiler.cppstd=17 protobuf/3.21.12: Main binary package '910a7810a932b82a3cead4ca418689ccf4b78668' missing. Using compatible package '0a68446b9947784d37544a468d7db3ff36a7d85d' grpc/1.54.3: Checking 7 compatible configurations: grpc/1.54.3: '52ad361594e596ee7f88fd87ba37ab2174ab2091': compiler.cppstd=14 grpc/1.54.3: '0c78842b83334618f01889ff889888b3f146a13a': compiler.cppstd=gnu14 grpc/1.54.3: 'd2bc8b69f52a34297ea0e3c293a1d3376047f865': compiler.cppstd=17 grpc/1.54.3: Main binary package 'bb54137d5dff5145bca4d8cad9c427c7728276d5' missing. Using compatible package 'd2bc8b69f52a34297ea0e3c293a1d3376047f865' grpc/1.54.3: Checking 3 compatible configurations: grpc/1.54.3: 'bbaf9e53251a5c445698328e8848090fe4bfecd2': compiler.cppstd=gnu14 grpc/1.54.3: '92c1c1630dc2345d02cd57738fe072522429dec0': compiler.cppstd=17 grpc/1.54.3: 'b8c70db77bc0292fdcd707b7203e3222765d18da': compiler.cppstd=gnu17 Requirements abseil/20230125.3#5431a4c609f5fb48bb8d8567e953243f:6283d35affd48d8ff9204d332970fe6c55a78f5d - Build c-ares/1.19.1#420a0b77e370f4b96bee88ef91837ccc:3fe9f3724ae9e6f32a7fbc3da5dc76fb7f1fb812 - Build grpc/1.54.3#3e0d68cd1578502c9c3a0e8366f0ab77:7955e9e862aa7386b69abeb9eef60a4ea8e243d2 - Build openssl/3.1.2#ef79f25fe0968d69538feca6a6b3bc7f:f00ea2c7d4ce533c8a2d1ed26d6910728dbe6867 - Build protobuf/3.21.12#d9f5f4e3b86552552dda4c0a2e928eeb:ad308974caa2eac0bf7ca345d9ef88ce9bf7c29a - Build re2/20230301#56bcddd1eaca2b093fd34525ae40ee9b:983c6606085cad4d4ea99760cba431d1d387d9ea - Build zlib/1.2.13#e377bee636333ae348d51ca90874e353:1206343ed397d681a20ab89a11a6d1294c9aa63e - Build Build requirements cmake/3.27.1#7d5f14447c023feef9f59640be53619a:63fead0844576fc02943e16909f08fcdddd6f44b#afe733606f5234ea564079b4778ddc22 - Download (conancenter) grpc/1.54.3#3e0d68cd1578502c9c3a0e8366f0ab77:d2bc8b69f52a34297ea0e3c293a1d3376047f865#4a798af94b09341c78429af09aa05910 - Download (conancenter) protobuf/3.21.12#d9f5f4e3b86552552dda4c0a2e928eeb:0a68446b9947784d37544a468d7db3ff36a7d85d#8c1cd7067fc6ab80e7f4674aadd93aef - Download (conancenter) Skipped binaries abseil/20230125.3, c-ares/1.19.1, openssl/3.1.2, re2/20230301, zlib/1.2.13

My profiles are the following: ======== Input profiles ======== Profile host: [settings] arch=armv8 build_type=Release compiler=qcc compiler.cppstd=14 compiler.libcxx=cxx compiler.version=8.3 os=Neutrino os.version=7.1 [system_tools] openssl/1.1.1 zlib/1.2.13 [conf] tools.cmake.cmaketoolchain:user_toolchain=['/workspace/toolchains/qnx_arm_toolchain.cmake'] [buildenv] CC=qcc CXX=q++ AR=ntoaarch64-ar RANLIB=ntoaarch64-ranlib

Profile build: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=11 os=Linux

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @Roman-Byshliaha-Bose

Thanks for your question. I am afraid that system_tools is only for tool_requires. Please check from: https://docs.conan.io/2/reference/config_files/profiles.html#system-tools

but it’s intended to list only the tool requires that are already in your own system

If openssl and zlib are also regular requires it is not possible to skip them and use them from the system.

[memsharded]

Some possible ideas as alternatives when using regular requires could be like:

• Using conditionals in recipes like if self.settings.compiler == "qcc": self.requires("...") in your requirements() method, and control what you depend on. If many of your recipes depend on this logic, you can reuse it via python_requires
• You could create/adapt the openssl and zlib libraries, so in certain platforms they are just a proxy to the system one (similar to what xxx/system packages are doing in ConanCenter.
• Fixing the build of the failing recipes for qnx (this is probably recommended)

[Roman-Byshliaha-Bose]

@memsharded thanks for your fast and accurate reply! Tis is something I missed - that it can be applied only to tool_requires section.

1) openssl/zlib are transitive dependnecies of my main dependency, which is grpc. And I doubt I can influence on their recipe. 2) nice idea, but won't work since those deps I want to not be fetched are transitive ones. 3) I agree that this is the most appropriate way, but it can take more time than I have on my hands.

QNX cross-compilation builds are always problematic. Do you have any hints on what to do if I need to either patch a dependencies source code or pass extra CMAKE flags that would do the trick for me? (for example it's possible to tell main grpc CMAKE file to either build deps or to use the packages in the toolchain)

I would appreciate any help.

[memsharded]

I think the new guides we are writing in the docs, like https://docs.conan.io/2/devops/using_conancenter.html, can give some hints. In short, it is very challenging that ConanCenter recipes can serve to all different use cases and scenarios. In some of them, it might be necessary to tweak and adapt some recipes in a way that cannot be merged in ConanCenter. Working with the approaches recommended there is an alternative, we have seen a bunch of users doing this successfully.

[memsharded]

Do you have any hints on what to do if I need to either patch a dependencies source code or pass extra CMAKE flags that would do the trick for me? (for example it's possible to tell main grpc CMAKE file to either build deps or to use the packages in the toolchain)

For this, this might help: https://docs.conan.io/2/examples/tools/cmake/cmake_toolchain/inject_cmake_variables.html

[Roman-Byshliaha-Bose]

I'll try the strategies mentioned above, thanks for the hints!

[memsharded]

Thanks for your feedback, and don't hesitate to open new tickets for any further issue, question or clarification that you might need!