Closed stradig closed 3 weeks ago
Need to investigate how to export data using SQL statements. This does not seem to be supported by vector.dev.
IIRC the existing metrics are exposed by sql-exporter — I think vector could just pull from there, if we want to expose it via vector.
yep, I found https://vector.dev/docs/reference/configuration/sources/prometheus_scrape/ that directly scrapes exporter data.
So we have 4 possible ways to go forwad:
@skyzh @sharnoff sounds correct? Which ones do you prefer?
My thoughts — I want to avoid adding tech debt by linking together components that weren't previously linked.
neondatabase/neon
, adding a new link. Also has the downside of repeating metric values because the autoscaler-agent fetch frequency would be greater than vector's refresh frequency.re:
sql-exporter fetches a number of things and it might overload the database if we fetch it every 5-15s
The current state of #895 is to have a configurable port and frequency — we can fetch as slow as we need to. For the ext-metrics datasources, we already do query every 15s (or maybe even more frequently?). Once a secondary sql-exporter is added with just the cheap metrics, we can e.g. add support for gradual rollout of fetching from a different port, faster, eventually switching everything over once old VMs restart.
@skyzh Can you share your opinion on options 2 vs 3?
If we want to have a second sql-exporter, I'm fine with either option 2 or 3. Otherwise, there needs to be a place to fetch these metrics, and it is easier to happen in vm-monitor.
...to be specific, I assume that autoscaling agent will at some point scrape these data at a high frequency, and I don't want these SQLs to be executed when we scrape sql-exporter:
Therefore, I'm proposing not go into the normal metrics sql-exporter for autoscaling metrics.
Discussed briefly with @skyzh — tl;dr:
Status:
Earlier this week, LFC-aware scaling was completely rolled out to all regions. Closing this :)
We want to be able to experiment with the algorithm to see which of those values can improve performance for autoscaled computes.