Add scrape interval as a config option

[simskij]

Enhancement Proposal

As discussed prior, we should add scrape interval as a config option for the charm. Since the subordinate can't possibly rely on putting a scrape-config charm in between itself and the principal, this seems like a palatable option in this case.

[simskij]

Adding this would likely also address the issue in https://github.com/canonical/grafana-agent-operator/issues/75.

[nobuto-m]

And https://github.com/canonical/prometheus-k8s-operator/issues/544. It was opened for prometheus-k8s initially but everybody is moving to grafana-agent (cos_agent interface) anyway.

I'm so happy to see it's moving forward!

[simskij]

And canonical/prometheus-k8s-operator#544. It was opened for prometheus-k8s initially but everybody is moving to grafana-agent (cos_agent interface) anyway.

I'm so happy to see it's moving forward!

Right, and for machine charms I think this makes sense as there is no option of putting any scrape-config charm between a subordinate and its principal. 😅

For grafana-agent-k8s however, the need to use scrape-config will remain.

I'm happy that you're happy! 🎉

[nobuto-m]

Another issue I can foresee is that $__rate_interval in Grafana dashboard depends on Scrape interval in Grafana datasource configuration. https://grafana.com/docs/grafana/latest/datasources/prometheus/configure-prometheus-data-source/#interval-behavior

The config is global to the data source if I'm not mistaken. Unless having multiple datasources defined with the same Prometheus instance but with a different scrape_interval config per data source, $rate_interval won't be set to the expected value even if the new config option allows a shorter scrape_interval. I mean $rate_interval could be calculated as 4m even if grafana-agent scrapes an exporter say every 15 seconds.

[simskij]

Resolved, see juju config option global_scrape_interval.