Drop support for python 3.8

[fjetter]

NEP29 suggests to drop python3.8 on April 14th 2023 which is getting closer.

The first release that would fall into this time window would be 2023.4.1 that is scheduled to be released on April 28th if there are no interruptions to release schedule.

Is there anything stopping us to follow this recommendation?

[martindurant]

Historically we haven't followed strict guidelines like the NEP, but whenever maintenance of the old version caused sufficient friction. Is there any cost to releasing for py3.8?

[jacobtomlinson]

I like the idea of following NEP29 in the same way we use black. We just do it and don't think too much about it because it reduces cognitive burden in the long run.

Also, I'm excited about Dict unions!

[crusaderky]

numpy has already dropped Python 3.8 in main. Next release will require >=3.9. xarray already dropped it back in January. pandas will drop it in 2.1.

+1 for mindlessly adhering to NEP29. It removes unnecessary burden of decision.

[crusaderky]

Historically we haven't followed strict guidelines like the NEP, but whenever maintenance of the old version caused sufficient friction. Is there any cost to releasing for py3.8?

Amy CI job that runs on Python 3.8 will be stuck with an obsolete version of numpy, pandas, xarray, and more will follow.

[martindurant]

I'm happy to go with following NEP, merely wanted to point out what we had been doing so far. There are, of course, more python releases now than there used to be.

[jrbourbeau]

The only consideration I can think of is user groups that are slower to update their Python version (e.g. they could rely on some other team that handles that). That said, those users wouldn't see any breakages, they just wouldn't be able to upgrade to the latest-and-greatest version of Dask.

Interesting, looking at PyPI stats (https://pypistats.org/packages/dask) Python 3.8 is the most common version of Python currently used with dask

Albeit, I don't know what fraction of downloads correspond to actual users vs. bots. Also, Python 3.7 is the third most popular Python version, which we dropped support for over a year ago 🤷

All that said, I do see value in following NEP29 (especially if the surrounding ecosystem is). So I'm +1 for trying it out. We can always revisit this decision in the future if it turns out to be problematic.

[martindurant]

Ray still ships for py36, at least on linux :) https://pypi.org/project/ray/#files

[jrbourbeau]

This issue has been open for three weeks and the only feedback has been mild-to-positive for dropping Python 3.8 in accordance with NEP29 (and following NEP29 in general). It'll take some time for PRs actually dropping Python 3.8 to happen. Unless the situation in this issue changes by then, let's call this lazy consensus in favor of following NEP29.

[pentschev]

As a data point only, RAPIDS has started the process of dropping Python 3.8 support today, so we should be good (hopefully). I'm sure that the RAPIDS CI contributes quite a bit to the Dask downloads, so once our process completes we will probably notice a drop in the download count too.

@shwina @quasiben FYI

[phofl]

pandas dropped 3.8 yesterday, but this won't be visible for users till 2.1 comes out