Rename dependency attributes

Ericson2314 commented 7 years ago

nativeBuildInputs, buildInputs, propagatedNativeBuildInputs, and propagatedBuildInputs are horrible names:

Word order + capitalization impedes easy grepping
native is vague wrt cross compilation
build time vs run time usage is unclear
both the host platform and target platform of dependencies can be varied, so in fact there is 2 axis of "native" vs "foreignness"

In https://github.com/NixOS/nixpkgs/pull/26805 I do add 4 more sorts of dependencies giving us:

Current name	@Ericson2314's Ideal name	Host is	Target is
`depsBuildBuild`	`depsBuildBuild`	build platform	build platform
`nativeBuildInputs`	`depsBuildHost`	build platform	host platform
`depsBuildTarget`	`depsBuildTarget`	build platform	target platform
`depsHostHost`	`depsHostHost`	host platform	host platform
`buildInputs`	`depsHostTarget`	host platform	target platform
`depsTargetTarget`	`depsTargetTarget`	target platform	target platform

A disadvantages of these names, per the comments below, is for the vast majority of deps the target platform is irrelevant---my names are unintuitive and noisy in this case.

N.B. I'd love to structure things purely when the deps are needed---run time or build time---which is a pretty understandable concept even for those who have never heard of cross compiling. But this just isn't correct (yet). Firstly, it isn't enough granularity---see when dependent's host = depender's build, there's 3 possible target platforms. Worse, host=host dependencies may be used in all possible ways: only at build-time (static linking), only at run-time (dynamic loading of plugin library), or both (linking and headers).

A completely different approach would be rely more on language-specific builders: For C, let people think in terms of headers (and pkg-config files, etc) and libraries (dynamic or static), and then C-specific infrastructure would take care of everything behind the scenes regarding propagation and multiple outputs. I should probably elaborate this approach more fully in the another issue or RFC.

Split out from https://github.com/NixOS/nixpkgs/issues/4855

CC @dezgeg @edolstra

matthewbauer commented 7 years ago

Ok quick question: once you get an stdenv built, host will always equal build, right? The average package maintainer can assume they're the same, I think.

ip1981 commented 7 years ago

Please don't use Target. It's confusing and irrelevant to the most software. Build inputs should be "host", native build inputs -"build".

lprndn commented 7 years ago

Passing by, I just wanted to give my humble point of view for the new names. First of all, I'm not an expert user, more like intermediate, meaning that I don't really know what are the internals of nix* but I can build some packages on my own. I do agree that current name should change and I don't doubt of the technical reasons behind the new ones. What worries me is that, if someone, like me, doesn't have the necessary knowledge (which I think is not basic and shouldn't be expected from an average user), it just doesn't make sense. For example, buildInputs is not perfect but I can quickly get it. If I had stumbled upon depsHostTarget while reading my first nix expression, I might still be trying to wrap my mind aroud it, reading GNU docs...

I think that, at least for the most commonly used functions, it could be nice to use names that don't actually tell how that function works but what it is used for (aka what you need when you dont know internals). But it is a design choice.

It's not really mandatory and it's the kind of question that can stuck a PR for ages but I just wanted to remind we can choose to take that into account. (or not btw)

Have a nice day,

Ericson2314 commented 7 years ago

See my edited OP----I originally wanted to just talk about build time vs run time, as that makes sense without thinking about cross compilation, but that breaks down pretty quick.

ip1981 commented 7 years ago

In fact, using "build inputs" for "host build inputs" (GNU style) and "native build inputs" for "build build inputs" (tautology, yeah) is another commonly recognized scheme. Typical example are C/C++ libraries and code generators: libraries are almost always "build inputs" (or host build inputs in GNU style) and code generators are native build inputs (build build inputs).

So, to me, current state of buildInputs and nativeBuildInputs is fine (as long as we talk about interface for package developers). If there is some internal inconsistency, it better be fixed, true.

Ericson2314 commented 7 years ago

@ip1981 Well, take solace in the fact that I don't think I'll be renaming anything anytime soon. I might land the new deps with those names, but they are all rather obscure so will be easy to rename later----I could __-prefix them all to indicate instability too.

vcunat commented 7 years ago

@matthewbauer: no. It's quite common e.g. for embedded HW that you build everything on x86_64 even though your host will be different, because it would take much longer to build directly on the weak HW.

grahamc commented 6 years ago

Not doable for 18.09.

Ekleog commented 5 years ago

(triage) This will likely need an RFC, as it's a difficult balance to strike between simplicity and ease.

stale[bot] commented 4 years ago

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

Search for maintainers and people that previously touched the related code and @ mention them in a comment.
Ask on the NixOS Discourse.
Ask on the #nixos channel on irc.freenode.net.

Ericson2314 commented 3 years ago

I actually i would like to keep this one open until we do something about breaking are own naming conventiton. It's a new-user-hostile thing to just throw in the towel for historical reasons, so I'm hoping we can at least do a little better, even if it's, say, supporting buildInputs and depsHostTarget in perpetuity.

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

roberth commented 3 years ago

Still relevant. The dependency attributes are confusing to newcomers, even just the buildInputs/nativeBuildInputs ones. Just encountered another example.

The systematic names are even worse for newcomers. target is nearly always irrelevant, so I believe it is desirable to omit this aspect from the two dominant dependency attribute names. Those who do need to work with badly designed compilers will have to read docs, as they already do.

buildInputs is an awful name because it is too generic by itself. A derivation represents a build, and everything that the derivation consists of is input to the build. So surely it has a more specific meaning? Maybe it's about the inputs that are responsible for executing the build? It has build in the name, so that must be it. Well, no. nativeBuildInputs is similarly awful. native describes a situation where no translation is necessary, ie a relation between two system-typed variables. It gives no clue as to which variables are equal.

Then what

Currently my best name for buildInputs is components. This word suggests that the dependencies have the same role as our own. This leads to the correct conclusion that these dependencies' outputs must be suitable for becoming part of our own output.

For nativeBuildInputs, I'd choose buildTools. This suggests that the dependencies will be used as tools for the build process. Hence they must be executable by the derivation builder.

These are very clear names that work for the vast vast majority of builds. I hope we can adopt names like these. I'll suggest a scheme below, but I hope we can at least change the legacy names.

Bonus scheme

It would be great if we could generalize these understandable names to the more exotic dependency relations that involve target. I find it hard to judge my own understanding of the subject, but I'll give it a shot:

Current name	@roberth's Suggested name	Host is	Target is	@roberth's Simplistic summary
`depsBuildBuild`	`bootstrapTools`	build platform	build platform	The dependency is used as a compiler whose output only runs during the derivation.
`nativeBuildInputs`	`buildTools`	build platform	host platform	The dependency is used as a regular build tool. Target-independent or produces code that runs on the host.
`depsBuildTarget`	`targetTools`	build platform	target platform	The dependency is used as a compiler for the target's runtime system. Bad form.
`depsHostHost`	_{bootstrapComponents but mostly irrelevant}	host platform	host platform	Currently redundant; we use `depsBuildBuild` instead. We don't even have tests for this. Apparently, metaprogramming dependencies don't need to be injected by the compiler, so far.
`buildInputs`	`components`	host platform	target platform	The dependency is of the same type as this derivation: outputs are suitable for static or dynamic inclusion in the output. Propagation works without change of role (index).
`depsTargetTarget`	`targetComponents`	target platform	target platform	The dependency is a component that the tool we're building will inject into its compilation outputs.

While this scheme doesn't map systematically to the underlying implementation, I don't believe that's relevant, because we ought to be designing an interface. With my suggested naming scheme, two rules apply that lead to a basic but correct understanding:

*Tools: dependency runs during my build
*Components: dependency doesn't run during my build

When a stdenv user encounters an advanced use case, they'll already know whether their dependency is a tool or component, so they only have to read through one or two variations of, say, *Tools.

I also expect components to help with the intuition around propagation and setup hooks, the thought being: component builds are like my own build, so their propagation and setup hooks behave like my own.

In fact, I believe presenting tables like these is only helpful for those working on stdenv and harmful to those who try to understand cross compilation for their use case or even just how to get things to work with strictDeps in the future. You really don't have to understand the implementation and interrelation of these dependency roles to work with them. It's like teaching full on monads to someone who only needs to work with lists and only once a month. They're just too condensed and anyone who thinks they're a "visual learner" (debunked concept btw) will try to understand the whole table, which is not productive and seems to turn people away.

As hinted, I'm open to all suggestions to improve the naming scheme, or to help me fix anything I've presented wrong. After all, it's a tricky subject and even trickier to convey understandably.

roberth commented 3 years ago

To clarify, these are my goals with these names, in order of importance:

The names aren't clearly wrong.
Newcomers can write a reasonable mkDerivation call without any knowledge whatsoever about cross compilation. This implies that a newcomer can decide which attribute to use for each dependency without knowing cross compilation jargon.
The names aren't vague.
The naming system is consistent with cross compilation.
Where possible, steer towards packages being more cross-buildable.

(1) should be obvious. The rest stems from making cross compilation subordinate to regular use, to a somewhat reasonable extent. While I recognize that Nixpkgs cross compilation is a very compelling feature that does attract contributors, most contributors are drawn to the project for their normal builds. Easy normal builds lead to more continued use, leading to more adoption, leading to better cross compilation. Also, easy normal builds lead to shorter reviews, leading to more contributors, adoption and better cross. So, a shallow learning curve does benefit cross compilation as well.

I am confident that these are strong effects, so in the final decision, I would not sacrifice any goal that's earlier in the list for a later one. The intent really is a list of priorities, not a weighting of criteria.

stale[bot] commented 2 years ago

I marked this as stale due to inactivity. → More info