Documentation on toolchains is lacking

[cbarrete]

Toolchains are a central concept in Buck2, since they are supported in the core rather than just being something defined in the prelude, but they are essentially not documented: it is not listed under Concepts in the sidebar, not part of the Glossary of Terms, and there is little information about what a toolchain really is or does.

Some questions that would be worth answering:

• What does Buck2 call a toolchain and what is it used for?
• What purpose does rule's is_toolchain_rule serve?
• When should one write a rule versus a toolchain?
• How can/should one use an existing toolchain?
• How do toolchains relate to configuration/constraints?

I have a vague notion of "toolchains let you decouple a rule from the actual compilers, runtimes, sysroots, etc. that it might use", but that's not enough for me to write good documentation about them I'm afraid...

[zjturner]

This isn't sufficient to write complete documentation, but a toolchain rule is almost no different than any other rule. For most purposes, you can think of them as actually being identical. What is a rule? It's a "function" (in scare quotes because a language lawyer will disagree with me) that returns a Provider (a collection of "structs", again in scare quotes).

When you have a target A that depends on a target B, all that actually means is "the rule implementing A has access to the provider (struct) returned by the rule when B was defined.

Think of it like this pseudocode:

function rule_one():
   return [SomeProvider(x = 4, y = 3)]

function rule_two(dep):
    dep = dep as SomeProvider
    print(f"dep.x = {dep.x}, dep.y = {dep.y}")

rule_one(name = "a")
rule_two(name = "b", dep = ":a")

What's a toolchain? It's the same thing. The toolchain just has its implementation return a provider (struct) that contains information about the toolchain. The path to the compiler, what args do you run it with, etc.

Then any rule that needs that toolchain (say, for example, how a cxx_library needs information about your C++ compiler), accepts the toolchain as a dep. the cxx rules do exactly this, they take an implicit / hidden parameter that takes a dep on toolchains//:cxx, which you are expected to define and which returns information about your C++ compiler.

All of the answers to your questions are therefore almost exactly the same whether you're talking about a toolchain rule or a non-toolchain rule.

The only thing I'm not sure about is why buck2 cares whether something is a toolchain rule or not (e.g. why is the is_toolchain_rule necessary?). I don't know, but I also haven't found a case where I needed to know yet.

[cbarrete]

Thanks for the detailed answer!

I should have made it clearer that I have some understanding of what toolchains are, I know the answers to some of the questions form having implemented Buck2 toolchains myself, but those questions should be answered by documentation.

What prevents me from writing documentation and have it be corrected in code review is that I don't have the answers to the fundamental/conceptual questions, and it's also not clear to me what "best practices" look like. This is especially true when writing toolchains/rules that are not covered by the prelude: regular rules can get one pretty far, so it would be make it explicit at which point turning that into a toolchain makes sense, and what the tradeoffs are.

I've tried writing this documentation myself a few times now, but I got stuck each time because "basically rules that are depended on implicitly" isn't quite precise enough for official documentation :/