conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.31k stars 986 forks source link

[question] install builds complete dependency tree from tool_requires #17169

Closed angelfromhell closed 1 month ago

angelfromhell commented 1 month ago

What is your question?

Maybe i am using conan here wrong. I am trying to install a compiled package

conan install --tool-requires=application/v0.1.1-dump -s build_type='Release'

the package is available with the settings Host / arch / build_type this package is using several static libraries which are also conan packages

def requirements(self):
    self.requires(static lib name)
    ...

i use here a different gcc as defined in the default profile. so by not providing this config in the install command, several packages are missing in that configuration, But as the application is already build they are also not required.

My expectation would be as the application is already build and available in the right configuration it would ignore the further build tree. Instead it calculates the complete graph, complaining about missing packages that are not required anymore.

have you got a hint what i am overseeing?

Have you read the CONTRIBUTING guide?

memsharded commented 1 month ago

Hi @angelfromhell

This is expected and by design, as discussed with the Conan 2 "tribe" (kind of a committee). The reason is that knowing the dependencies, also the build dependencies always is better to avoid many reproducibility and traceability issues, and also necessary to allow some build-requires to affect the package_id of their consumers.

My expectation would be as the application is already build and available in the right configuration it would ignore the further build tree. Instead it calculates the complete graph, complaining about missing packages that are not required anymore.

What is required in general if the graph is configured correctly (like package_type is correct, etc), is that only the recipes are necessary, but not the binaries. The binaries will be marked as "Skip" and won't be downloaded. Downloading only the recipe is very fast, and in the case of regular self.requires(static lib name) it is completely necessary to evaluate the package_id of your application package, and to ensure that Conan will require a new package_id for application if some dependency change, requiring a build from source to integrate the changes in the dependencies in the application.

Please let me know if this clarifies the behavior, and confirm if you are actually seeing the binaries as "Skip".

angelfromhell commented 1 month ago

hello memsharded

thank you very much for the explanation. i can understand that you need the complete tree of recipes, not that i need to do a superfluous build of all libs with a different compiler version, but the application at the end will not be rebuild

So build done with

application -> build with gcc14 ( contains settings arch, os build_type => compiler has been removed) lib1 -> build with gcc14 lib2 -> build with gcc14

now doing an install ( default profile defines gcc13 ) results in (checked with graph explain) lib1 not available for gcc13, closest find is gcc14 ....

building the libs results in the graph being ok but at the end the application is not rebuild this seems a bit unnecessary. So basically when i want to do a installation based on only the defined settings, i would need to repack it into a new conan package?

memsharded commented 1 month ago

That would seem a different topic, more related to the binary model and package_id than to the tool-requires.

If the libs are built with gcc13 and those libs are statically linked into application, then the application package_id will depend on the package_id of the libs, and it will compute a new package_id and require a re-build. This is totally expected, as an application running with the libs binary code compiled with gcc13 will be different (performance, even bugs) than the same app compiled using the libraries compiled with gcc14.

Unless you are also forcing the application somehow to depend differently on the libs, the re-build of application will happen. If it is not happening, we would need more details, to be able to reproduce, like some full recipes (using conan new it is easy to create example packages.

This is irrespective of other possible ways to define the install, that includes per-package settings, or defining a different binary compatibility model via the compatibility.py plugin, that can be done. But lets try to focus on the package_id issue first.

angelfromhell commented 1 month ago

ok thanks for the fast response, yet this needs a bit more time for me to prepare a full example

memsharded commented 1 month ago

Ok, then I think it is better to close this question as responded (the OP about the full dependency graph), and when you are ready you can create a new ticket with the details about the package_id behavior.

angelfromhell commented 1 month ago

hi memsharded,

lets do it like that. I just tried to make a dummy example of 1 lib and 1 bin conanfile without any c files. Although i provide the profile gcc14, it still stores the information of the lib with gcc13...

i will try to play around a bit more and open a new ticket

best regards Angel

memsharded commented 1 month ago

Thanks for the feedback, looking forward your new ticket.