m-lab / etl-gardener

Gardener provides services for maintaining and reprocessing mlab data.
Apache License 2.0
13 stars 5 forks source link

Migrate to a prerequisite driven system #316

Open gfr10598 opened 3 years ago

gfr10598 commented 3 years ago

Rather than being date and job oriented, with a state machine per job, move to a system where a job is a bunch of actions and prerequisites, and some jobs may have prerequisites that span multiple datatypes and other action outputs.