We need more resources.

hercules-ci / support

User feedback, questions and our public roadmap. help@hercules-ci.com

5 stars 1 forks source link

We need more resources. #61

Open nixinator opened 2 years ago

nixinator commented 2 years ago

Hercules CI implementation of our Private and Public Repo's are going well.

However, like any thing that is a success, developers need more and more CI resources and 'demand' speedier builds.

https://nixbuild.net could help us, in crunch time, or a backup to our self hosted runners.

However, can herciles-ci agent work with nixbuild.net

roberth commented 2 years ago

You could already configure nixbuild.net as a remote builder on your agents. I can split the concurrency configuration to allow evaluation and effects to be kept to a reasonable number while running many build tasks remotely. This will allow the build tasks to scale to very large demands.

nixinator commented 2 years ago

Thats amazing and super powerful.

Would you be willing to share the configuration, and i'll give it a whirl soon.

roberth commented 2 years ago

Configuring the nixbuild.net remote builder can be done as usual. Splitting the concurrency numbers in the configuration has to be implemented first in hercules-ci-agent. You could try out the setup with a sub-optimal concurrency configuration today.

The reason for having (at least) two distinct concurrency numbers in the config is

if the concurrency number is too low, the remote builders are underutilized
if the concurrency number is too high and a sufficiently large number of evaluations is runnable, the agent could be overwhelmed by memory pressure from the concurrent nix evaluators

With two numbers, both situations can be avoided simultaneously, for smooth and scalable operation.

roberth commented 2 years ago

(and of course you could run more agents to handle the load)

dhess commented 2 years ago

This sounds interesting and I have a question about how it would work with Hercules CI.

Unless something has changed with recent versions of Nix, if you configure any remote builders on a Nix machine, and you build a derivation(s) with nix-build on that Nix machine, nix-build will always use remote builders, and never perform any local builds (assuming the remote builders have the same arch as the local machine, that is).

If you add remote builders to a machine running the Hercules CI agent, will the agent machine only perform evaluation locally and run all builds remotely? Or does the Hercules CI agent have some ability to load-balance local and remote builds? (Perhaps this is what you mean by "sub-optimal concurrency" in your earlier comment.)