search

hasadna / open-bus

:bus: Analysing Israel's public transport data
93 stars 27 forks source link

should have processes that calculate aggregations for past data #372

Closed OriHoch closed 2 years ago

OriHoch commented 2 years ago

currently, most aggregations processes calculate latest data / data from last day. we need processes that calculate aggregations for past data and make sure all past data has all aggregations. this process should also allow to re-calculate aggregations if logic is changed / bug is fixed.

OriHoch commented 2 years ago

process is documented here: https://github.com/hasadna/open-bus-pipelines/blob/main/docs/RELOADING_DATA.md