VE FR - st3am-etl-1.2 - How to set schedule parameter in DAGS

[DelphinoVarela]

Please provide instructions to schedule DAGs

[be-ant]

AIRFLOW divides the time in scheduling intervals and schedules the next run AFTER the end of the scheduling interval. A scheduling interval can be defined in a simple way in the schedule parameter of a DAG, using cron definitions (@daily, @hourly, " 1 1 "...). See crontab Guru for getting the right expression. This works perfectly with regular schedules (every night at 5 am) but not with irregular ones (example, Mondays and Wednesdays at 1 PM and Thursdays at 3 PM)

Keep in mind that the information presented in the top right part of the screen presents the interval to be processed (even if it says Next Run), not when will it be triggered.

Practical examples

Example 1: If we put in the schedule parameter this value ("/5 4-5 "then the DAGS runs will be executed every 5 minutes between 4 and 5 am.

Example 2 This shows that we are running a 30 min schedule. The picture was taken at 1626, but it looks like the next run is in the past... the fact is that the next run will use the data collected from 1600 to 1630 (and will be triggered at 1630)

At 16:32 we see that there was an execution at 16:30

More info

For a more detailed info see the documentation at https://docs.astronomer.io/learn/scheduling-in-airflow

There are more complex ways (see https://airflow.apache.org/docs/apache-airflow/stable/howto/timetable.html) in case that does not cover the needs of the customer.

[DelphinoVarela]

Set up okay