search

conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.3k stars 986 forks source link

[feature] Add `--build` option to only build (missing) build requirements #17207

Closed planetmarshall closed 1 month ago

planetmarshall commented 1 month ago

What is your suggestion?

I would like to be able to pass an option to --build at install time to only build "build requirements".

Motivation

we have a scenario where all our requirements are built by CI, which publishes a lockfile which is used by developers locally.

The problem is that many of our developers have ARM macs, which we do not have available for CI systems (we have a couple of Intel Macs available).

Whilst we can cross-compile normal project dependencies, we cannot do this for build dependencies which have to match the Build environment.

I would like to allow developers to build any missing build dependencies locally, but not normal dependencies, as we consider missing dependencies to be a configuration error.

Workaround

We can do this currently by explicitly listing the dependencies with multiple --build:missing options, but we would have to update this whenever the lockfile changes.

Have you read the CONTRIBUTING guide?

planetmarshall commented 1 month ago

Incidentally this does beg the question of why build requirements are in the dependency graph when all of the built dependencies are available. If end users didn't have to download build requirements for packages they don't actually have to build then that would also solve our problem.

memsharded commented 1 month ago

Hi @planetmarshall

Thanks for your suggestion.

https://github.com/conan-io/conan/pull/17117 will introduce a conf that will allow to full skip the tool-requires and the test-requires when desired. However, I am not fully sure this is what you really need.

Incidentally this does beg the question of why build requirements are in the dependency graph when all of the built dependencies are available.

They are not. Only the "recipes" are in the dependency graph, but not the actual packages. This is done because of some reasons:

If end users didn't have to download build requirements for packages they don't actually have to build then that would also solve our problem.

This is already happening. Conan packages that are not necessary because they are tool-requires for dependencies, will be marked as "skip" and they will not be downloaded. If not, we would need more details to reproduce and understand what could be happening. Could you please double check this?

planetmarshall commented 1 month ago

This is already happening. Conan packages that are not necessary because they are tool-requires for dependencies, will be marked as "skip" and they will not be downloaded. If not, we would need more details to reproduce and understand what could be happening. Could you please double check this?

We were experiencing the issue raised here - https://github.com/conan-io/conan/issues/15339 - that is conan install was failing when the network was unavailable despite having all the dependencies in the cache. As suggested we added tools.graph:skip_binaries=False to our default profiles (both host and build profiles)

It's proven to be really hard to reproduce but I will raise a bug if it reoccurs. Thanks.