Officialy support python 3.12

[FloChehab]

What's the use case?

Python 3.12 is officially out. We are looking forward for dagster to officially support this release.

Ideas of implementation

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[hbruch]

Note: due to https://github.com/sdispater/pendulum/issues/696, this probably might require upgrading to pendulum v3

[cbini]

I think the Dagster team might want to consider moving away from pendulum. I love the library, but it's not actively developed. Looking through the repo issues and discussion, they originally planned to release v3 early 2022 and the main dev doesn't appear to have time to dedicate to the project as their preoccupied with Poetry.

This discussion is kind of concerning: https://github.com/sdispater/pendulum/discussions/771

[cbini]

Christmas came early: pendulum 3.0

[streitl]

Hey Dagster team, any news on when pendulum will be updated to v3 and Python 3.12 supported? Thanks!

[alangenfeld]

latest update here: https://github.com/dagster-io/dagster/pull/19456#issuecomment-1915671977

[alexismanin]

Does a todo list of actions for Python 3.12 compatibility exist somewhere ?

From what I see, at least universal_path blocks Python 3.12 migration. Trying to use a dagster_aws.s3.S3Resource in Python 3.12 environment causes the following error:

ImportError: cannot import name '_PosixFlavour' from 'pathlib' (/usr/lib64/python3.12/pathlib.py)
...
File ".../venv/lib64/python3.12/site-packages/dagster_aws/s3/__init__.py", line 6, in <module>
  from .io_manager import (
File ".../venv/lib64/python3.12/site-packages/dagster_aws/s3/io_manager.py", line 20, in <module>
  from upath import UPath
File ".../venv/lib64/python3.12/site-packages/upath/__init__.py", line 2, in <module>
  from upath.core import UPath
File ".../venv/lib64/python3.12/site-packages/upath/core.py", line 8, in <module>
  from pathlib import _PosixFlavour  # type: ignore
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

UPDATE : The Python 3.12 compatibilty PR on universal_path has been merged last week. If it is released soon, we might hope to reach for Python 3.12 compatibility soon ?

UPDATE 2: Testing universal_path master (i.e. declaring explicit dependency to universal_pathlib@git+https://github.com/fsspec/universal_pathlib.git#egg=f7af1748ba7d2020a126cafdb47bad0261aa198e) solves above error.

[ja-me-sk]

While not a core dependency, I initially ran into this as a limitation with dagster-duckdb alongside pendulum when creating the dagster_university project.

As suggested by docs, using the supported version of 3.11.x works fine on setup.

[cbini]

universal_pathlib released an update yesterday with 3.12 support: https://github.com/fsspec/universal_pathlib/releases/tag/v0.2.0

Haven't tried it yet, but will today and keep you posted.

[gibsondan]

dagster should work on python 3.12 now!