hashicorp / nomad-autoscaler

Nomad Autoscaler brings autoscaling to your Nomad workloads.
Mozilla Public License 2.0
426 stars 84 forks source link

Allow metrics to be pushed instead of polled #405

Open lgfa29 opened 3 years ago

lgfa29 commented 3 years ago

In some scenarios polling for metrics is either wasteful (metrics that don't change often) or too slow (when scaling needs to happen fast). In these cases it would be helpful to be able to push metric values to the Autoscaler instead of wait for it to poll.

mrkurt commented 3 years ago

This is our scenario! We have two possibly related problems:

  1. Thousands of autoscaling jobs, but only a handful change at any given time
  2. A need to launch new allocations "on demand"

In our case, it would be useful to push scaling metrics rather than polling. Both to minimize load on our metrics cluster, and to make (2) work. (2) would let us scale to zero and launch an allocation when a connection comes in.

We can probably do this with Nomad job updates ourselves, but after playing with the autoscaler + polling I feel like it might save us a lot of work.