The billing system tries to solve one specific problem:
Receive events in ordered by time
And generate a snapshot per time unit (hour in postgres, day in
zuora) grouped by job, event_type, and internal_instance_id.
It does this using a 20TB+ bigquery table.
With this tiny change that increases entropy merely a little, we could
replace that with sum by (job, event_type, internal_instance_id) (rate(billing_ingester_events_total[1h])) as long as we take care to
preserve hour alignment.
This only adds the label to dev, because I'm still trying to see that
it works.
The billing system tries to solve one specific problem:
It does this using a 20TB+ bigquery table.
With this tiny change that increases entropy merely a little, we could replace that with
sum by (job, event_type, internal_instance_id) (rate(billing_ingester_events_total[1h]))as long as we take care to preserve hour alignment.This only adds the label to dev, because I'm still trying to see that it works.
This could also replace the awful tables in https://cloud.weave.works/admin/billing/organizations with something like a grafana dashboard.