helm: promote autoscaling to stable

dimitarvdimitrov commented 4 months ago

https://github.com/grafana/mimir/pull/7282 added autoscaling to Helm as an experimental feature. This issue is about remaining work in order to promote autoscaling to stable.

Bugs

[x] https://github.com/grafana/mimir/issues/7473

Docs/Migration procedure

[ ] https://github.com/grafana/mimir/issues/7367

Remote URL UX

[x] Support a remote different from the metamonitoring setup https://github.com/grafana/mimir/pull/7392
[ ] Take the additional headers and auth from metamonitoring setup. Currently basic auth and extra headers are ignored.
[ ] Default to X-Scope-OrgID: metamonitoring if the config is already sending metrics to the same Mimir installation (the same way that metamonitoring computes it)
[ ] Add validation that is same mimir cluster is used, then metamonitoring is also enabled

Helm-jsonnet diffing

[ ] Add helm-jsonnet diffing so that the autoscaling configs don't get out of sync when we change one and forget to change the other. This is a matter of enabling autoscaling on select components in these two files and then making sure there are no differences between the rendered manifests. Minor differences can still be ignored via kustomiztions like this one

Make dashbaords compatible with helm-deployed KEDA objects

[ ] Similar to the /operations/mixin changes in https://github.com/grafana/mimir/pull/4687 - the matchers for keda objects should allow for helm-deployed objects

beatkind commented 4 months ago

@dimitarvdimitrov if you want, you can assign me to both of these issues. #7368 & #7367

dimitarvdimitrov commented 4 months ago

thanks @beatkind 🙏 I'm not sure of the level of detail in these issues, so ask away if anything isn't 100% clear

beatkind commented 4 months ago

Some thoughts on

Support a remote different from the metamonitoring setup

https://github.com/grafana/mimir/pull/7282#discussion_r1487522360

Basically drills down to have a global section for kedaAutoscaling:

kedaAutoscaling:
     prometheusAddress: http://... 
     customHeaders:
       {}
     pollingInterval: 10

QuentinBisson commented 3 months ago

@dimitarvdimitrov is there an issue to add hpa to the components that do not have it yet like the ingester?

dimitarvdimitrov commented 2 months ago

for ingesters I could only find an internal one unfortunately - grafana/mimir-squad#1410. I see that @jhalterman was last working on that. Jonathan is there a public issue for this work?

jhalterman commented 2 months ago

The issue you cited is the only one. There's nothing public yet.

grafana / mimir