Open koivunej opened 1 month ago
Discussed in meeting: Generation number increasing by at most 1 to reuse cached values is the best filter.
No idea right now: how to store generations in the cache? Cache might not be leaked outside, so perhaps that is easy to do.
@skyzh can you take a look at this one when you're back in the office?
I think storing generation would make sense, and I would like to have a new format for consumption_metrics.json (I assume nobody else except pageserver is using this json file).
struct NewRawMetric {
key: MetricsKey,
event_type: EventType,
generation: Generation,
timestamp: u64,
value: u64,
}
I'm refactoring the storing/restoring code right now...
... it seems that for upload_metrics_bucket
and upload_metrics_http
, we will need to keep the original message format
otherwise, I think I'll have a PR to migrate to the new format, and another patch to implement the ">= generation => do not report" logic
Internal slack thread: https://neondb.slack.com/archives/C061CPK7UQL/p1726556769067039?thread_ts=1726493755.104459&cid=C061CPK7UQL
Symptom:
Sent/exported synthetic sizes:
More broadly, what happened:
How to fix it: