JustFixNYC / nycdb-k8s-loader

Loading and updating of NYC-DB data via containerized batch processing.
6 stars 2 forks source link

Make the scheduled tasks use cron expressions #3

Closed toolness closed 5 years ago

toolness commented 5 years ago

Right now the aws_schedule_tasks.py script uses the simple scheduled events syntax, which is very limited, especially since the intervals are based on when the scheduled tasks are first created (so using an interval of rate(1 day) will trigger every day at the time that the task was first created). We should probably use cron syntax instead, so that e.g. tasks can be run at night regardless of when the schedule was created.