Support "Hot" Slaves or Standby Pool

[blaines]

I'd like to have a pool of nodes ready to start jobs rather than provisioning one-at-a-time. I'm wondering if an API could be created to register nodes and pass back JNLP tokens. If that was possible, I could then create an ECS "Service" with a pool of nodes ready to start jobs.

To some degree this seems possible: https://support.cloudbees.com/hc/en-us/articles/115003896171-Creating-node-with-the-REST-API

I looked into doing this myself, but I don't understand the internals of Jenkins well enough to create an API. Although I did manage to write some groovy to create nodes from within Jenkins.

Other resources: https://stackoverflow.com/questions/42683324/create-jenkins-jlnp-slave-programmatically

[pgarbe]

Hi @blaines, let my try to understand your use case. When you talk about nodes, is it Jenkins agents or the underlying EC2 instances of the ECS cluster?

The purpose of the Jenkins ECS plugin is to create ECS Tasks on demand. An ECS Task represents a run-once agent in Jenkins und dies after the execution. The only thing where you've to take care is that the underlying cluster scales automatically.

[ElijahLynn]

YES, just came here to ask if this was possible!

Our use case is that sometimes I have a job that literally takes 10 seconds to complete, BUT it takes nearly a minute just to provision the slave container/task to accept the job. I want some containers sitting there, ready to accept a task. I want my 10 second job to take close to 10 seconds total. When trying to get rapid feedback for PRs and tests this 1 minute penalty is painful.

[pgarbe]

So, if I understand both of you correctly, it's about improving the startup time that is needed to launch the agent as ECS task, right?

This sounds absolutely fair. On the other side these cloud plugins don't support idle agents on purpose as it costs money and in general it's fast and easy to create the agents on demand.

Have you checked where you loose your time? I've seen some improvements by pulling the Docker images of the container on the underlying EC2 host to have it already cached on the machine.

[ElijahLynn]

That's a good idea, I'll check that out, thank you.

[tstibbs]

@pgarbe note that with Fargate you can't pre-pull the containers because you don't have any control/access to the underlying instances.

[blaines]

Thanks pgarbe and ElijahLynn! Yes it's certainly the Jenkins agents that we'd like to optimize. I'll look into what performance could be gained by caching the underlying image.

Separately from that, I'm imaging the ability to pre-configure a "pool". The pool could have a desired number of slaves, and potentially autoscale as needed. With an ECS Service there's potentially no autoscaling or pool logic to be implemented in the plugin. The hardest bit for me to wrap my head around (and what's keeping me from doing it myself) is how to issue the JNLP authentication token to hot-slaves.

This sounds absolutely fair. On the other side these cloud plugins don't support idle agents on purpose as it costs money and in general it's fast and easy to create the agents on demand.

We certainly want to save on cost, but during the day we have many builds and we'd like to ensure agents are ready to accept tasks immediately. An engineer spending less time watching a build is worth the cost.

For my own reference, it seems multiple labels can be specified for an agent and could help facilitate assignment of jobs to a standby-pool or on-demand launch.

agent { label 'reserved||ondemand' }

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[svaranasi-corporate]

Stale or not, this one needs to be supported by the plugin. Can this be re-opened, please?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.