partitionedAttrs from multiple partitions?

[terlar]

I wonder if it would make sense to support a list for the partitionedAttrs. For example if you have multiple partitions providing packages and want to propagate them.

Currently you can do:

partitionedAttrs.packages = "dev";

But what if you had multiple partitions providing packages?

partitionedAttrs.packages = ["component1" "component2"];

With this feature you could replace other use cases for subflakes as well.

What do you think, would it make sense to support?

[roberth]

Partitions are intended as a performance optimization to avoid loading some modules (from some inputs) when accessing flake outputs that are not affected by those modules. This happens at a cost though, because this optimization requires distinct evalModules calls for each partition. Concretely this means that the choice to use, say, a dev partition is a tradeoff between

• when using partitions:
  • evaluating packages quickly by not loading dev modules (and the inputs they come from)
• when not using partitions:
  • reusing evaluation work that could be shared between packages and devShells evaluations

Merging multiple partitions into a flake output would give you the worst of both, because you'll have more total evaluation work due to having these partitions, and you still need to evaluate all the listed ones in order to produce attrNames packages. Note that attrNames packages is a prerequisite to packages.foo in Nix's data model.

To keep it efficient, you will have to list explicitly which partition is responsible for which attribute. The current type, mapping single output attribute names to single partition names enforces that, but it doesn't let you specify anything inside those single output attributes. Those would have to be attribute paths, or even a pattern of sorts, because we have to deal not just with records but also dicts. I suppose it'd look like

{
  partitionedAttrs.packages."*".foo = "component1";
  partitionedAttrs.packages."*".bar = "component2";
  partitionedAttrs.devShells = "dev";
}

I do wonder whether a more versatile abstraction could be built into the module system proper. This is a general solution to probably an even more general problem.

[terlar]

Hmm, that is true, thank you for the clarification.

I guess generally it could be solved by just splitting things into modules.

I just thought that it would be good to let some things be their own flakes as that would allow the different components to control their inputs. I imagine you would want to share inputs (and evaluation) most of the time, but in some cases you would want to override or add extra inputs.

I was just thinking whether a mono-repo style with multiple components aggregated at the top-level using partitions was feasible. But perhaps a different approach would have to be used.