conan-io / conan-center-index

Recipes for the ConanCenter repository
https://conan.io/center
MIT License
918 stars 1.65k forks source link

[question] Why are tool_requires dependencies not explicitly defined for basic toolchains? #24444

Closed jacobfriedman closed 2 days ago

jacobfriedman commented 2 days ago

What is your question?

There are no stated requirements for zlib. Yet, the following is declared:

from conan.tools.cmake import CMake, CMakeToolchain, cmake_layout

There's no stated requirement in the conanfile for cmake (nor make, for that matter).
Why isn't the tool_requires explicitly stated?

Here are the appropriate requirements for zlib, which also need to be added to zlib's test_package:

    def build_requirements(self):
        self.tool_requires("cmake/3.27.4")
        self.tool_requires("make/4.3")

This is a common occurrence for most of the packages I've seen.

Have you read the CONTRIBUTING guide?

EDIT: requirements(self) -> build_requirements(self)

RubenRBS commented 2 days ago

Hi @jacobfriedman thanks for your question.

The general Conan Center index recipe policy is to assume that there are some baseline tools installed in the system where the recipes get created

One of such tools is CMake, for which we assume CMake 3.15 is there. It's only when libraries depend on newer cmake features where you can see a explicit requirement to that upper bound, like self.tool_requires("cmake/[>=3.20 <4]").

We know that this is a challenge for barebones enironments that do not provide those binaries, for which Conan has some tools to help with that, namely the [tool_requires] (Docs here) section of your profiles.

Let me know if this helps :) - Happy to discuss your use-case in which these binaries are unavailable from the start

jacobfriedman commented 2 days ago

The use-case is a barebones install. Ignoring the use-case, I would expect each recipe to, at least, explicitly state in its build requirements that cmake 3.15 is required, if it is indeed required. Problems arise in packaging when making an assumption. I'm not sure who's has the time to scour through all the recipes, but it should be done. I can start with a pull request for zlib, stating self.tool_requires("cmake/[>=3.15 <4]"). Would that be agreeable?

jcar87 commented 2 days ago

The use-case is a barebones install.

as far as we are aware, this is not the most common use case in Conan Center based on the overall feedback.

Problems arise in packaging when making an assumption. Apologies if this isn't clear in the documentation of Conan Center - but as my colleague is mentioning, in Conan Center there is an expectation of some minimal tooling already present in the system, CMake is included in that as well.

There are historical reasons why some recipes have tool requirements on CMake, and some don't - this is whenever a recipe requires a newer version of CMake than what we assume is the baseline - the CI in Conan Center targets CMake 3.15 - we want to keep this baseline close to "the oldest version that users are likely to be on" - which I would imagine is around whatever RHEL7 or Ubuntu 18.04 have, by now.

There is a secondary issue, which has been mitigated more recently, which each is that reach recipe in the past expressed a requirement on a different version of cmake - and solving some graphs would have required a myriad of cmake versions, which is undesirable, especially back when they had to be built from source.

I can start with a pull request for zlib, stating self.tool_requires("cmake/[>=3.15 <4]"). Would that be agreeable? Not at the moment - we are considering in the future to express this in all recipes that require cmake - but we also need to find a solution for users that expressly want to use their system version, or have that expectation.

In the meantime, as my colleague mentions, adding the following to your host profile:

[tool_requires]
cmake/[>=3.15 <4]

To your host profile should give you the ability to operate on a barebones system, without having to modify any recipe.

jacobfriedman commented 2 days ago

If building cmake from scratch, the addition to a global (host) profile on a local/siloed remote wouldn't necessarily do it. I wonder what else I need to add to my host profile to satisfy other packages, and if there are any other chicken-and-egg scenarios. If only those requirements were explicitly stated! From an analyst's perspective, you'd want to know what tool was used to build a program, and what tools that program requires.

Yes, this is an edge case. Yes, I could use a different profile for my cmake build. However, making assumptions about a user's configuration is not satisfactory for my particular concern. And; RHEL7 and U18 do not ship with cmake.

It may be grunt work. Aside from the time taken, what is the drawback of explicitly stating tool_requires in all recipes?

jacobfriedman commented 2 days ago

I'm thinking of build_requirements' hyperexplicit boostrap-dependency graph down to early GCC compilers. Or, skip all that and just install bootstrap llvm. run-requirements will also need specific parts of llvm.