Feed update status. For each (feed_pk, update_result) a counter with the number of updates since the process started.
Number of entities. For each (entity_type, feed_pk) a gauge with the number of entities in the system.
Ideally these metrics would be updated after every feed update, but the multi-process architecture of Transiter makes this hard/impossible. Instead, in the scheduler, we can have a periodic task (1 minute) that runs the relevant DB queries and populates the metrics.
For feed update status we'll need to keep track of the feed_update_pks for feed updates that occurred before the scheduler started (and should be ignored) and that have already been incorporated into the metric (should not be double-counted). Probably we'll want to get all feed updates with pk bigger than X, increment the counters, and then set X to be the largest pk observed. At start up, X would be set to the largest pk already present.
Desired metrics:
Ideally these metrics would be updated after every feed update, but the multi-process architecture of Transiter makes this hard/impossible. Instead, in the scheduler, we can have a periodic task (1 minute) that runs the relevant DB queries and populates the metrics.
For feed update status we'll need to keep track of the feed_update_pks for feed updates that occurred before the scheduler started (and should be ignored) and that have already been incorporated into the metric (should not be double-counted). Probably we'll want to get all feed updates with pk bigger than X, increment the counters, and then set X to be the largest pk observed. At start up, X would be set to the largest pk already present.