Templates: macros datetime.utcnow and datetime.utcfromtimestamp are deprecated in Python 3.12

[iamtodor-fb]

What do you see as an issue?

The Templates: macros documentation page leads to the deprecated datetime.utcnow and datetime.utcfromtimestamp.

Upon now we are using current_timestamp_utc="{{ macros.datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S') }}" which is deprecated. Now I'm looking for a way to specify the UTC timezone in the macros, that is not deprecated.

Per documentation:

Variable	Description
macros.datetime	The standard lib’s datetime.datetime
macros.timedelta	The standard lib’s datetime.timedelta
macros.dateutil	A reference to the dateutil package
macros.time	The standard lib’s time
macros.uuid	The standard lib’s uuid
macros.random	The standard lib’s random.random

However for example,

>>> import datetime
>>> datetime.datetime.now()
datetime.datetime(2024, 8, 22, 18, 3, 31, 124275)
>>> datetime.datetime.now(datetime.timezone.utc)
datetime.datetime(2024, 8, 22, 16, 3, 49, 878177, tzinfo=datetime.timezone.utc)

timezone.utc comes from datetime and it cannot be imported on a macros level.

If I miss anything please feel free to ping me and I would be happy to provide more details

Solving the problem

Please update the documentation on how to specify thedatetime.utcnow in macros.

Anything else

I also found the related issue, where the internal Airflow code was transitioned to the up-to-date toolset https://github.com/apache/airflow/issues/32344

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[jscheffl]

Your link of documentation was pointing to a deprecated docs page. The current page is: https://airflow.apache.org/docs/apache-airflow/stable/templates-ref.html#macros

Can you check the current documentation again please?

[iamtodor-fb]

@jscheffl thank you for the reply.

1. I double checked and I did not find the deprecation mention on the page https://airflow.apache.org/docs/apache-airflow/2.7.3/templates-ref.html#macros
2. Both pages, the one I shared and that you shared https://airflow.apache.org/docs/apache-airflow/stable/templates-ref.html#macros look very similar. And there is also no specification of how to provide the datetime.utcnow in macros

[potiuk]

Yes. It would be great to find out and document it. @iamtodor-fb - would you like to make some research and find out? If not someone elsee will have to pick it up and update the docs follow the helpful @tirkarthi link from #32344 PR by @tirkarthi : https://github.com/tirkarthi/airflow/tree/fix-datetime - and describing it in the docs as you suggested would be nice.

Marked it as good first issue.

[iamtodor-fb]

@potiuk thank you! I did a workaround - I explicitly set the timezone in the airflow.cfg file as all our pipelines run in the same timezone, but it would be very nice to have it adjustable at macros level

[potiuk]

@potiuk thank you! I did a workaround - I explicitly set the timezone in the airflow.cfg file as all our pipelines run in the same timezone, but it would be very nice to have it adjustable at macros level

Yes. Would be nice. That's why it waits for someone to research and document it.

[iamtodor-fb]

@potiuk seems like my mentioned work-around didn't work for me cause bigquery adjusts the timezone, so the timezone has to be explicitly provided. We managed to provide it as following: macros.datetime.now(macros.dateutil.tz.UTC).isoformat()