Dagrun "skipped" for timezone aware DAGs with schedules affected by daylight saving time

[wolfier]

What do you see as an issue?

The issue is that users are unaware of scheduling behaviour between transition of when daylight saving time is observed and when it is not.

For daylight time saving observing timezones, at the cusp of when switching between when daylight saving time is NOT observed to when it is, there is an hour that does not exist. For example, 2 AM Central Time ("America/Chicago") does not exist on March 10th 2024.

• 2024-03-10 7:59 UTC is 1:59 AM CT (CST)
• 2024-03-10 8:00 UTC is 3:00 AM CT (CDT)
• 2024-03-10 8:01 UTC is 3:01 AM CT (CDT)

For daylight time saving observing timezones, at the cusp of when switching between when daylight saving time is observed to when it is NOT, there is an hour that is repeated. For example, 1 AM Central Time ("America/Chicago") happens twice on November 3rd 2024.

• 2024-11-03 6:59 UTC is 1:59 AM CT (CDT)
• 2024-11-03 7:00 UTC is 1:00 AM CT (CDT)
• 2024-11-03 7:01 UTC is 1:01 AM CT (CST)
• 2024-11-03 7:59 UTC is 1:59 AM CT (CST)
• 2024-11-03 8:00 UTC is 2:00 AM CT (CST)
• 2024-11-03 8:01 UTC is 2:01 AM CT (CST)

Solving the problem

A description of the effects of these impacted schedule intervals in terms of dagrun creation as that is what drives task scheduling. Without a dagrun, there are no tasks.

Anything else

No response

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[lopezvit]

I'll try to get back to this in the future if I have time, trying to provide a better working example where this is happening, but basically these are the steps to reproduce:

1. Create a DAG with:
   1. catchup=True
   2. hourly schedule (e.g. schedule_interval="1 5-19 * * *")
   3. start_date with a DST aware timezone in the past (e.g.: "start_date": pendulum.datetime(2020, 1, 8, 11, 8, tz="Europe/Helsinki")
2. Schedule said DAG

It will try to catchup until the Sunday 25th of October 2020, but wont continue with the catchup after that, totally preventing the operation. As a workaround until this is fixed I changed to the fix timezone: "start_date": pendulum.datetime(2020, 1, 8, 11, 8, tz=2) Notes: maybe it is not necessary that it is daily and probably it doesn't need to start in January, but that is what I've find that it reproduces it.

[shahar1]

@wolfier Would you still like to work on this issue?