Open Ten0 opened 9 months ago
Hey @Ten0,
For main.nix
files, you can set extendingMakesDirs = ["/"];
as described in the documentation. With this setting Makes will name CLI jobs according to absolute paths within the repository. That is, for file /project1/service1/run/main.nix
, the CLI output would be /project1/service1/run
.
With makes.nix
things are a little more complicated. makes.nix
is an interface between:
makes.nix
.This is why, when declaring builtins from makes.nix
files, CLI outputs do not follow a naming pattern based on directory structure like main.nix
files but rather group all jobs for the same builtin.
Example:
For the following makes.nix
file
{
formatPython,
...
}: {
formatPython = {
project1.targets = ["/project1"];
project2.targets = ["/project2"];
project3.targets = ["/project3"];
};
}
We get the following CLI outputs:
/formatPython/project1
/formatPython/project2
/formatPython/project3
With some boilerplate code you could still use all builtins from main.nix
files, granting directory-based CLI outputs.
Example:
/project1/service1/format/main.nix
{
formatNix,
makeScript,
projectPath,
...
}: let
bin = formatNix {
name = "project1-service1";
targets = [(projectPath "/project1/service1")];
};
in
makeScript {
entrypoint = "format-nix-for-project1-service1";
name = "project1-service1-format";
searchPaths.bin = [bin];
}
From that main.nix
you can do anything that nix allows so you can orchestrate everything from a makes/services.nix
file:
/project1/service1/format/main.nix
{
formatNix,
makeScript,
projectPath,
...
}: let
config = import (projectPath "/makes/services.nix") {
inherit formatNix;
inherit makeScript;
inherit projectPath;
};
in
# customFormatNix is a more generic function that runs formatNix as explained before
config.customFormatNix {
name = config.project1.service1.format.name;
targets = config.project1.service1.targets;
}
You could even declare everything in the services.nix
file and simply import it from the main.nix
:
makesArgs: let
config = import (makesArgs.projectPath "/makes/services.nix") {inherit makesArgs;};
in
config.project1.service1.format.run
Thank you for your answer! 🙂
So it looks like my understanding was correct: to generate target paths, I'm tied by either the builtins (which I can't customize) or the "one folder with a main.nix per output" approach (which would force me to create one directory per output with a main file that would specify a bunch of boilerplate and finally .target_name
(=folder_name
), which woudn't fit at all with my current directory structure or be one I'd like anyway) 🙁
Is there any technical constraint that prevents building a system that has a more flexible interface similar to the one I described above? Having the root expression that evaluates to just at-most-one-incantation-of-each-type-of-builtin doesn't allow for flexible program structure compared to something along the lines of [{ name, thing_to_do }]
.
At first glance it looks like we could evaluate only the part of the current folder's expressions that are related to outputs names when starting the CLI, and then only when actually building, evaluate the derivation
attribute of each { name, derivation }
that the root expression outputs, but there may be big drawbacks that I'm missing.
Would a design where we add an outputs
builtin that takes as parameter [{ name, derivation }]
work or is there some constraint that prevents passing such complex inputs to builtins? (like, they have to get evaluated before they get passed or something along those lines)
To be very clear: I love the idea of the project of having a CI system globally powered by nix, with a CLI that enables choosing the targets we're interested in, especially making full use of caching across all users of the organization for every single derivation. The potential of it looks insane! However, if it seems that the only way to efficiently add outputs to the CLI is to make incantations in the makes.nix
after PR-ing a new builtin to this repository, then it seems pretty clear to me that that doesn't scale unless you happen to also be the company that manages the Makes
project: we can't have every company use this and PR every single customization they need as a global builtin in the project:
Thanks a lot 🙂
I'm considering using this project.
IIUC, I currently have two ways to generate an "output" for the CLI:
builtin
, and that will generate an arbirary number of targets depending on some magic inmakes
' own codeextension
, which always means creating a folder for the target, and having amain.nix
in that folder.What I want to achieve is the following directory structure:
and I want my possible "OUTPUT"s in the CLI to look like this:
where the build/test/run/deploy outputs behavior are all defined by my
/makes/service.nix
That means I need to be able to customize:
<main or makes>.nix
s (so that with a single nix file in a service folder I can declare the multiple targets that I need)<main or makes>.nix
s (to be able to passimport ./makes/service.nix
)My understanding is that this is not possible currently, and instead any such
service.nix
would have to be declared as a builtin inmakes
itself, and then I would put only the configuration of that in somemakes.nix
, which also means I could only instantiate one of each of those permakes.nix
, whereas my ideal syntax would probably be:(where
outputAttrSet
would convert from the attrset of output to probably[{ name, derivation }]
, allowing for further customization if required)Is it actually already possible to achieve something like this? If so, how? If not, is this something that could maybe be made possible in the future or are there any clear blockers?
Thanks,