Closed jrbourbeau closed 2 years ago
Generally this sounds like a good idea and should simplify some things in Dask + Distributed.
That said, there's at least a couple issues I can think of atm.
First is PyPy hasn't quite made it to 3.8 yet. It sounds like it is close, but may need a few more weeks based on Matti's recent summary here ( https://github.com/conda-forge/conda-forge-pinning-feedstock/issues/2089#issuecomment-1010721498 ). Related to that, in conda-forge we are discussing building packages for PyPy 3.8, but this likely needs to wait until things are completed in PyPy as well as for the existing migrations to clear (like Python 3.11). So may take a bit more time before packages are built for PyPy 3.8.
Second RAPIDS currently still supports Python 3.7. Not exactly sure when we plan to drop that. Maybe Ben can comment.
Third there are still users/providers on Python 3.7 that haven't yet made the move. Believe Kaggle is one. Not sure who else.
Thanks for the feedback @jakirkham!
Second RAPIDS currently still supports Python 3.7. Not exactly sure when we plan to drop that. Maybe Ben can comment.
@quasiben do you have any insight into when RAPIDS is planning to drop Python 3.7? FWIW it looks like there's some preparation for this which is already underway https://github.com/rapidsai/cudf/pull/10092
First is PyPy hasn't quite made it to 3.8 yet. It sounds like it is close, but may need a few more weeks based on Matti's recent summary here ( conda-forge/conda-forge-pinning-feedstock#2089 (comment) )
That's a fair point and it'd be good for us to be concious of pypy users. That said, with major dependencies like NumPy (latest release is Python 3.8+) and pandas (upcoming 1.4 release is Python 3.8+) dropping support for Python 3.7, it will make it harder for Dask to continue to support Python 3.7. @mattip, thank you for pushing on pypy3.8. Do you have an estimate for when a release might be?
It should be done within a month. I want to get a release candidate out next week, covid and flu willing. If dask does drop Python3.7, that only means new versions of dask would not be available, right? It is not as if you would yank the older packages.
I think the plan is to drop 3.7 in our upcoming release if we are successful at getting the 3.9 builds working across all RAPIDS libraries.
FYI deprecation notice is here: https://docs.rapids.ai/notices/rsn0013/
If dask does drop Python3.7, that only means new versions of dask would not be available, right? It is not as if you would yank the older packages.
That's right, Matti
@jrbourbeau the rapidsai-nightly channel now has 3.9 packages and will be continuing down the path of dropping 3.7 support
I believe RAPIDS has now dropped support for 3.7. All new packages are for published for Python 3.8 and 3.9 in https://anaconda.org/rapidsai/rapids/files and https://anaconda.org/rapidsai-nightly/rapids/files
Thanks for the update @quasiben -- I'll suggest we go ahead and drop Python 3.7 then
Time to merge the PRs?
Let's get an approval on https://github.com/dask/distributed/pull/5683 first. But then yeah, https://github.com/dask/distributed/pull/5683 and https://github.com/dask/dask/pull/8572 should go in together
Oh are they done/ready for review?
This is now complete for Dask + Distributed. Possibly other packages will need similar changes, but that can be followed up on separately. Going to go ahead and close as resolved.
This has been done in dask-kubernetes
too.
I have noticed that Read the Docs still has Python 3.7
as the default for building documentation and that is causing build errors in places since Dask 2022.2.1
was released. I've raised dask/dask-kubernetes/pull/412 to bump to 3.8 (the highest Read the Docs allows right now) resolve things there. I suspect this will need doing on all projects that use Read the Docs.
There is a way to override this in a .readthedocs.yaml
config file. Maybe this is also overridable in the ReadTheDocs settings?
FWIW we discussed adding a .readthedocs.yaml
config file in issue ( https://github.com/dask/dask/issues/2568 ) and wound up doing this when dropping Python 3.7 ( https://github.com/dask/dask/pull/8572 ).
There is a way to override this in a .readthedocs.yaml config file. Maybe this is also overridable in the ReadTheDocs settings?
I couldn't see anything in the UI to set this.
I've added a .readthedocs.yaml
file to dask-kubernetes
and that resolved the issue. But this will need to be done for all projects that use Dask/Distributed.
I'd like to propose that Dask drops support for Python 3.7. NEP 29 (which there was general interest in for Dask https://github.com/dask/community/issues/66) states that support for Python 3.7 can be dropped starting on Dec 26, 2021. Additionally, other projects in the ecosystem (e.g. NumPy, Xarray) have already begun dropping support for Python 3.7. I've brought this up offline with a group of Dask maintainers and they didn't have any objections, but wanted to raise an issue here to solicit feedback from others.
cc @jsignell @quasiben @jakirkham @fjetter @crusaderky @ian-r-rose