Feature Request: Schedule-based minRunners Configuration

[kahirokunn]

What would you like added?

I suggest enhancing the user experience by providing the ability to dynamically control minRunners based on the time of day and the day of the week, similar to cron syntax. This would allow users to adjust the minRunners parameter to suit their particular needs. For instance, from 9:00 to 18:00 on weekdays, one might want to set minRunners to 10. After business hours, where the workload is likely reduced, we could then set it to 0.

Why is this needed?

Currently, it seems that the minRunners setting is static and needs manual intervention to change. With dynamic settings, the system will be able to better adapt to differing workload patterns and would improve overall performance and efficiency.

Additional context

The proposed enhancement would be useful for organizations that have clear busy periods. This change could directly impact cost savings by allowing a reduction in resources during quiet periods, whilst still providing ample resources during busy periods. Furthermore, it could promote more effective use of resources and avoid resource waste.

[joshua-reddish]

This would be very helpful to my org as well! It is one of the greater obstacles to adopting RSS, as many teams are unwilling to give up the quicker pickup times from the ARC system.

[joneteus]

HorizontalRunnerAutoscaler already has scheduling options for minReplicas https://github.com/actions/actions-runner-controller/blob/master/docs/automatically-scaling-runners.md#scheduled-overrides

[rteeling-evernorth]

HorizontalRunnerAutoscaler already has scheduling options for minReplicas https://github.com/actions/actions-runner-controller/blob/master/docs/automatically-scaling-runners.md#scheduled-overrides

This is great, but not available on the newer com.github.actions runner. I get the feeling many would love to see this as a feature however. (myself included)

[gfrid]

HorizontalRunnerAutoscaler already has scheduling options for minReplicas https://github.com/actions/actions-runner-controller/blob/master/docs/automatically-scaling-runners.md#scheduled-overrides

This is great, but not available on the newer com.github.actions runner. I get the feeling many would love to see this as a feature however. (myself included)

was looking for that feature, would be great to have it back

[asafhm]

I've created PR #3564 to address this need

[kenmuse]

With the latest versions of ARC, I'd recommend considering the Kubernetes-native approach: use a ChronJob. It gives you more control (you can configure more than just minRunners) without requiring any changes to the ARC codebase to support it. It also helps to preserve the Linux principle of each component doing just one task. The approach also enables more complex logic when you need it, and it provides you with a history of each schedule run. I posted a sample deployment YAML and an explanation of what each component is doing.

[asafhm]

@kenmuse What's unclear to me, is how do you preserve the original value of the minRunners in that example post-override? Let's say I have a CronJob that scales down my runners at midnight. I would like the same number of runners to come up in the morning, and for that I'll need another CronJob, but that will also require me to repeat the original minRunners value in it, since the change happened to the spec of the scaleset cannot be "rolled back". I.e. After the first override, the original minRunners field value is lost. Am I correct?

[WTPascoe]

With the latest versions of ARC, I'd recommend considering the Kubernetes-native approach: use a ChronJob. It gives you more control (you can configure more than just minRunners) without requiring any changes to the ARC codebase to support it. It also helps to preserve the Linux principle of each component doing just one task. The approach also enables more complex logic when you need it, and it provides you with a history of each schedule run. I posted a sample deployment YAML and an explanation of what each component is doing.

This is a little ick for gitops workflows because it means 2 additiona commits per scale period - scale-up and scale-down. That's a lot of noise in the commit history for something that used to be part of the product.

For environments where changes are audited, this creates a requirement to separately track and report these too.

[air-hand]

@rentziass as codeowner This issue is a feature request for gha-runner-scale-set (though labeled community). Please review #3564 from @asafhm

[alexnovak]

Instead of implementing this directly within the ARC, or leaving it to a cronjob, maybe there'd be value in editing minRunners through the scale subresource?

The specific word "scale" feels like a bit of a misnomer here, but controlling through that mechanism might work better with some gitops workflows, and would allow for management with some common autoscaling tooling like keda, which has a very nice cron autoscaler. Keda fans could also scale on some more esoteric events that your autoscaler of choice might know about (say, a prometheus metric that would signal to your team that might need a large number of warm runners fairly soon).

Implementation wise, this should also be fairly trivial. Call my bluff here if I'm misappropriating some of these fields, but on the autoscalingrunnerset CRD, I think we'd just need to add the following to its subresources block

scale:
  specReplicasPath: .spec.minRunners
  statusReplicasPath: .status.currentRunners