search

dask / community

For general discussion and community planning. Discussion issues welcome.
20 stars 3 forks source link

Drop Python 3.7 #213

Closed jrbourbeau closed 2 years ago

jrbourbeau commented 2 years ago

I'd like to propose that Dask drops support for Python 3.7. NEP 29 (which there was general interest in for Dask https://github.com/dask/community/issues/66) states that support for Python 3.7 can be dropped starting on Dec 26, 2021. Additionally, other projects in the ecosystem (e.g. NumPy, Xarray) have already begun dropping support for Python 3.7. I've brought this up offline with a group of Dask maintainers and they didn't have any objections, but wanted to raise an issue here to solicit feedback from others.

cc @jsignell @quasiben @jakirkham @fjetter @crusaderky @ian-r-rose

jakirkham commented 2 years ago

Generally this sounds like a good idea and should simplify some things in Dask + Distributed.

That said, there's at least a couple issues I can think of atm.

First is PyPy hasn't quite made it to 3.8 yet. It sounds like it is close, but may need a few more weeks based on Matti's recent summary here ( https://github.com/conda-forge/conda-forge-pinning-feedstock/issues/2089#issuecomment-1010721498 ). Related to that, in conda-forge we are discussing building packages for PyPy 3.8, but this likely needs to wait until things are completed in PyPy as well as for the existing migrations to clear (like Python 3.11). So may take a bit more time before packages are built for PyPy 3.8.

Second RAPIDS currently still supports Python 3.7. Not exactly sure when we plan to drop that. Maybe Ben can comment.

Third there are still users/providers on Python 3.7 that haven't yet made the move. Believe Kaggle is one. Not sure who else.

jrbourbeau commented 2 years ago

Thanks for the feedback @jakirkham!

Second RAPIDS currently still supports Python 3.7. Not exactly sure when we plan to drop that. Maybe Ben can comment.

@quasiben do you have any insight into when RAPIDS is planning to drop Python 3.7? FWIW it looks like there's some preparation for this which is already underway https://github.com/rapidsai/cudf/pull/10092

First is PyPy hasn't quite made it to 3.8 yet. It sounds like it is close, but may need a few more weeks based on Matti's recent summary here ( conda-forge/conda-forge-pinning-feedstock#2089 (comment) )

That's a fair point and it'd be good for us to be concious of pypy users. That said, with major dependencies like NumPy (latest release is Python 3.8+) and pandas (upcoming 1.4 release is Python 3.8+) dropping support for Python 3.7, it will make it harder for Dask to continue to support Python 3.7. @mattip, thank you for pushing on pypy3.8. Do you have an estimate for when a release might be?

mattip commented 2 years ago

It should be done within a month. I want to get a release candidate out next week, covid and flu willing. If dask does drop Python3.7, that only means new versions of dask would not be available, right? It is not as if you would yank the older packages.

quasiben commented 2 years ago

I think the plan is to drop 3.7 in our upcoming release if we are successful at getting the 3.9 builds working across all RAPIDS libraries.

quasiben commented 2 years ago

FYI deprecation notice is here: https://docs.rapids.ai/notices/rsn0013/

jakirkham commented 2 years ago

If dask does drop Python3.7, that only means new versions of dask would not be available, right? It is not as if you would yank the older packages.

That's right, Matti

quasiben commented 2 years ago

@jrbourbeau the rapidsai-nightly channel now has 3.9 packages and will be continuing down the path of dropping 3.7 support

quasiben commented 2 years ago

I believe RAPIDS has now dropped support for 3.7. All new packages are for published for Python 3.8 and 3.9 in https://anaconda.org/rapidsai/rapids/files and https://anaconda.org/rapidsai-nightly/rapids/files

jrbourbeau commented 2 years ago

Thanks for the update @quasiben -- I'll suggest we go ahead and drop Python 3.7 then

jsignell commented 2 years ago

Time to merge the PRs?

jrbourbeau commented 2 years ago

Let's get an approval on https://github.com/dask/distributed/pull/5683 first. But then yeah, https://github.com/dask/distributed/pull/5683 and https://github.com/dask/dask/pull/8572 should go in together

jakirkham commented 2 years ago

Oh are they done/ready for review?

jakirkham commented 2 years ago

This is now complete for Dask + Distributed. Possibly other packages will need similar changes, but that can be followed up on separately. Going to go ahead and close as resolved.

jacobtomlinson commented 2 years ago

This has been done in dask-kubernetes too.

jacobtomlinson commented 2 years ago

I have noticed that Read the Docs still has Python 3.7 as the default for building documentation and that is causing build errors in places since Dask 2022.2.1 was released. I've raised dask/dask-kubernetes/pull/412 to bump to 3.8 (the highest Read the Docs allows right now) resolve things there. I suspect this will need doing on all projects that use Read the Docs.

jakirkham commented 2 years ago

There is a way to override this in a .readthedocs.yaml config file. Maybe this is also overridable in the ReadTheDocs settings?

FWIW we discussed adding a .readthedocs.yaml config file in issue ( https://github.com/dask/dask/issues/2568 ) and wound up doing this when dropping Python 3.7 ( https://github.com/dask/dask/pull/8572 ).

jacobtomlinson commented 2 years ago

There is a way to override this in a .readthedocs.yaml config file. Maybe this is also overridable in the ReadTheDocs settings?

I couldn't see anything in the UI to set this.

I've added a .readthedocs.yaml file to dask-kubernetes and that resolved the issue. But this will need to be done for all projects that use Dask/Distributed.