Mic92 / nix-fast-build

Combine the power of nix-eval-jobs with nix-output-monitor to speed-up your evaluation and building process.
MIT License
225 stars 10 forks source link

Ram usage building .#nixosConfigurations #73

Open covert8 opened 1 month ago

covert8 commented 1 month ago

Hi love the project, thinking of doing some work on it and I would like some feedback before starting.

By default the .#checks get build and I added my .#nixosConfigurations like so:

checks.x86_64-linux = lib.mapAttrs' (name: config: lib.nameValuePair "${name}" config.config.system.build.toplevel) self.nixosConfigurations;

However, this resulted in additional downloads when doing a plain nix flake check and prolonging the check duration; so I found this sub optimal as I like to check on a tmpfs.

To avoid the default .#checks I passed -f .#nixosConfigurations but that results in an OOM on the evaluator (>64GB;no tmpfs ; 5 hosts) and not completing any derivation in a timely fashion. I ended with this:

hosts=$(nix flake show --json | jq -r '.nixosConfigurations | keys | .[] | select(. != "C60006") |  ".#nixosConfigurations." + . + ".config.system.build.toplevel"')
while read -r host; do
    nix-fast-build -f "$host" --no-nom -j 1 --eval-workers 1 --skip-cached
done <<<$hosts

My questions:

Mic92 commented 4 weeks ago

My questions:

* Is there a way to simplify the while loop?

If you only run one eval thread at the time, you can also just pass all hosts at once, it shouldn't make a big difference.

* Do you think it is feasible to remove already uploaded derivations (when using the cachix daemon) to make more ram available during a build?

This is what --skip-cached should do, assuming you have configured your system correctly to use the cache.

* Would you accept a pr supporting the `attic` client?

Attic support is fine, it's probably pretty similar to cachix.

covert8 commented 2 days ago

If you only run one eval thread at the time, you can also just pass all hosts at once, it shouldn't make a big difference.

Could you give an example of this? I can't seem to get it to build multiple hosts if i specify multiple flake urls or if i combine them.