[Prometheus] Support non-comma delimiter in prometheus.io/rule annotation

[shashankram]

As per https://roadie.io/backstage/plugins/prometheus/#graphs, the prometheus.io/rule annotation does not allow specifying query strings such as foo_bar{foo='foo',baz='baz'} because the comma character is used to delimit multiple rules within a single annotation. It suggests using recording rules for more complex queries, however it may not be feasible to use recording rules for queries that are dynamic in nature, e.g. a query determined at runtime after the Prometheus instance has been deployed.

I would like to be able to specify multiple queries such that the query may contain a comma to specify multiple labels, e.g. prometheus.io/rule: "foo_bar{foo='foo',bar='bar'}; foo_baz{foo='foo',baz='baz'}". This should be feasible if there's an additional annotation to provide a custom delimiter for the queries, ; in this case.

[fjudith]

Hello @shashankram

You should keep complexity on the Prometheus side an manage to create kind of UID label in the Prometheus Rule. The annotatons should only use that rule and use the UID as a single workload selector.

ex. in Kubernetes the UID can be a combination of --<service | pod>.

https://github.com/testruction/fastapi-sqlalchemy-cockroachdb/blob/4047f17f8a23d567cc3786cf192dac2c23fdd7bd/chart/templates/backend/prometheus/prometheusrule.yaml#L17

[jomeier]

Hi, Yes, please replace the delimiter to something, that does not interfere with the PromQL Syntax. Greetings, Josef

[Xantier]

Thanks Josef,

Could you spin up a Pull Request for this fix and we can review and merge it for a new release?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.