medium-term, this would be the right tool for the job.
it's also open source, so there's very little lock-in.
For now, to keep the scope of this manageable, we can try to accomplish the same with cron jobs on github actions.
However, medium-term, apache airflow is a better tool:
it does not pollute the actions log with non-code changes (it's technically possible to run gh actions for any reason, but non-commit events still feel awkward).
airflow's DAG natively "understand" the dependencies between the ETL steps; the actions runtime doesn't, we have to retrofit it by returning statuses from whichever services we query
airflow's DAGs support more complicated graphs; these may be possible in gh actions, but are hard to reason about / understand.
medium-term, this would be the right tool for the job. it's also open source, so there's very little lock-in.
For now, to keep the scope of this manageable, we can try to accomplish the same with cron jobs on github actions.
However, medium-term, apache airflow is a better tool: