grafana / mimir-proxies

Proxies to help you ingest your metrics into Grafana Mimir.
GNU Affero General Public License v3.0
76 stars 6 forks source link

Add tagged metrics support to whisper converter #93

Open ywwg opened 1 year ago

ywwg commented 1 year ago

(imported issue)

Right now the whisper converter only supports untagged metrics. We should add support for tagged metrics.

From the docs, the tags are stored in a separate database from the whisper files. So it looks like:

- [ ] We will need an example tagdb, preferably in sqlite, but it looks like the tagdb can be any sql db and therefore the converter would need to be able to connect to an SQL server.  (Or can we insist that they provide an sqlite file? Or can we ask them to dump a CSV? That CSV might be GIGANTIC so probably not)
- [ ] in Pass1, when we hit a metric we need to query the tagdb -- I am not able yet to find the schema / API... maybe this:https://github.com/graphite-project/graphite-web/blob/master/webapp/graphite/tags/models.py
- [ ] once we have the tags for a metric it's easy to call LabelsFromTaggedName
ywwg commented 1 year ago

I think based on the schema we'd do a big JOIN, something like SELECT Tag.tag FROM Tag JOIN SeriesTag ON SeriesTag.tag == Tag.id AND SeriesTag.Series == Series.id WHERE Series.path=="METRICNAME"

and then

SELECT Tag.tag, TagValue.value FROM TagValue JOIN SeriesTag ON SeriesTag.tag == Tag.id AND SeriesTag.TagValue == TagValue.id WHERE Tag.id IN "THE RESULTS FROM ABOVE"

ywwg commented 1 year ago

ah, it'll be simpler than that, we don't need the tag values

ywwg commented 1 year ago

hm, the db may just be for querying. the actual storage appears to be an alternative scheme: https://github.com/graphite-project/graphite-web/blob/master/webapp/graphite/tags/utils.py#L125-L134