Closed bdice closed 1 week ago
Some quick data points:
I'm +1 on doing this in this timeframe.
DLFW builds are using Python 3.10 (DLFW support matrix), as that's the stable python3 that ships with Ubuntu 22.04 (Python version across Linux distros), so this change shouldn't be disruptive for those builds.
And nemo-curator
is on python >= 3.10
(code link), so they shouldn't be affected either.
We are exploring dropping Python 3.9 in RAPIDS 24.10.
Just curious to know what the ETA is for dropping Python 3.9?
I believe we will need to remain pinned to dask 2024.7.1 until then. In the meantime, we cannot catch breaking changes in dask/distributed.
The plan is to do this "within 24.10", but we can't be more precise than that.
I believe we will need to remain pinned to dask 2024.7.1 until then. In the meantime, we cannot catch breaking changes in dask/distributed.
For Python 3.9, yes, but I think rapids-dask-dependency
could take on a dependency on newer dask
/ distributed
for Python 3.10+
Like this:
# TODO: drop the python_version specifiers and just use latest versions built from GitHub once
# RAPIDS drops Python 3.9 support
dependencies = [
"dask @ git+https://github.com/dask/dask.git@main ; python_version >= '3.10'",
"distributed @ git+https://github.com/dask/distributed.git@main ; python_version >= '3.10'",
"dask-expr @ git+https://github.com/dask/dask-expr.git@main ; python_version >= '3.10'",
"dask==2024.7.1 ; python_version < '3.10'",
"distributed==2024.7.1 ; python_version < '3.10'",
"dask-expr ; python_version < '3.10'",
]
That'd allow projects to start testing against the latest development version of Dask throughout the 24.10 release cycle.
That'd allow projects to start testing against the latest development version of Dask throughout the 24.10 release cycle.
Okay, thanks for the response @jameslamb! This may be our best option -- I'll look into it.
For rapids-dask-dependency
, likely Conda will work ok as-is since the solver will pick packages that are Python 3.9 (or 3.10+) compatible. Though please let us know if not
Dropping 3.9 builds should now be safe, and would also unblock the work of updating our dask pinning.
This PR does that: https://github.com/rapidsai/shared-workflows/pull/235
This change in shared-workflows
is also safe to merge before we do the cleanup in each repo to remove dependencies, configs, etc. referencing Python 3.9. We're just building/testing fewer packages.
Closing the loop on Dask, we dropped Python 3.9 from CI, so no special casing was needed (or used) there
Just a normal unpinning of Dask as part of RAPIDS post-release cleanup
Thanks James! 🙏
Have shepherded a good chunk of these through
Of the remainder:
Please let me know if there are others you need help following up on
Thanks John!
A few are marked as draft. Maybe we can take them out of draft?
Yep, my approach has been to open these PRs in draft first, to not bother people with notifications for review until I see CI (or at least, most CI) passing.
Happy to see all of those that were still in draft as of yesterday are looking ok, so I've opened them all up for review.
Discussed in today's build infra sync, this is almost done. Once complete, the team will start working on Add support for Python 3.12 https://github.com/rapidsai/build-planning/issues/40
Did some more GitHub searches, found a few more things that were missed.
All that's left here is those PRs and dask-build-environment
: https://github.com/rapidsai/dask-build-environment/pull/97.
This is close enough to done that we can move forward with Python 3.12 (#40)
Did another set of GitHub searches, and I think this work is now complete! Thanks for the help everyone 😁
Thanks James! 🙏
This issue tracks dropping Python 3.9, following from the discussion on https://github.com/rapidsai/build-planning/issues/40:
We are beyond the SPEC 0 date for dropping Python 3.9 now, and expect that RAPIDS dependencies will gradually drop support. We are exploring dropping Python 3.9 in RAPIDS 24.10.
Tasks:
dependencies.yaml
to removepy: 3.9
sections like thisdependencies.yaml
to raise the fallback matrix's Python lower boundfind_package(Python 3.9 ...)
pyproject.toml
:requires-python
and specifiers3.9
to make sure nothing was missed!Things to be careful about:
pynvjitlink
is used by more than just RAPIDS (like https://github.com/NVIDIA/nvmath-python), and uses its own CI matrix to support a broader range of versions than RAPIDS (link)... we may sometimes drop a Python version across RAPIDS but still support it inpynvjitlink
rapids-reviser
,rapids-build-backend
,rapids-metadata
, etc. shouldn't have their floors bumped unless there's a compelling reason to do so... that'd make it harder to fix bugs in those tools that affect older releases