Release 2023.10.0

[jrbourbeau]

Release version	2023.10.0
Planned release date	2023-10-13
Status	On-track
Release manager	@jrbourbeau

Best effort

Try to close before the release but will not block the release

• [ ] https://github.com/dask/dask/pull/10549
• [x] https://github.com/dask/distributed/issues/8265
• [ ] https://github.com/dask/dask/pull/10544
• [x] https://github.com/dask/distributed/pull/8223
• [x] https://github.com/dask/distributed/pull/8262
• [x] https://github.com/dask/distributed/pull/8264

Blocker

Issues that would cause us to block and postpone the release if not fixed

• [ ] ...
• [ ] ...

Comments

cc @fjetter @hendrikmakait @rjzamora @quasiben @phofl

[hendrikmakait]

Best effort:

• https://github.com/dask/distributed/pull/8262
• https://github.com/dask/distributed/pull/8264

[fjetter]

Re changelog

• https://github.com/dask/dask/pull/10535 should likely be the top highlight for this release. This will have a very strong impact on the array computing community
• There are a couple of changes on distributed side that can be lumped together into a "P2P is now much more robust and less likely to fail" section. I don't think describing each individually will be useful for users.
  • https://github.com/dask/distributed/pull/8262
  • https://github.com/dask/distributed/pull/8264
  • https://github.com/dask/distributed/pull/8242
  • https://github.com/dask/distributed/pull/8124
  • https://github.com/dask/distributed/pull/8244
• There are also a couple of somewhat related changes that improves high scheduler CPU for large graphs
  • https://github.com/dask/distributed/pull/8238
  • https://github.com/dask/distributed/pull/8240
  • https://github.com/dask/dask/pull/10547

Generally speaking, how far along are we with combining dask + distributed changelogs into one? Most of these changes are actually part of the distributed repo but I believe the dask docs are read by many more people and I think users are rather interested in the holistic overview and less in where exactly the change was made.

[jrbourbeau]

Thanks @fjetter

Generally speaking, how far along are we with combining dask + distributed changelogs into one?

However far along we want to be : ) I think merging changelogs makes sense given dask and distributed are co-released, tightly pinned to each other, and most folks look at docs.dask.org instead of distributed.dask.org. It's also in line with our general effort to "move things from the distributed docs over to the dask docs". I'll plan on merging changelogs for this release unless there are major objections.

[crusaderky]

I don't think that https://github.com/dask/dask/pull/10544 will be fully ready for the release. The current situation is that there are two non-trivial 3.12-specific failures:

• what looks like a malformed hlg when manipulating masked arrays
• dask.dataframe diverging from pandas after concatenating categorical dataframes

I did not investigate them further.

I can see some value in merging the PR in a flawed state today and have these two issues smoothed out in time for the next release. Or we could delay the whole 3.12 support to the next release and prevent a bumpy start to users.

CC @graingert

[fjetter]

I don't think we're in a rush with py3.12

[jrbourbeau]

FYI will start pushing out the release in a few minutes

[jrbourbeau]

Closing as 2023.10.0 is out on PyPI, conda-forge, and docker. Thanks all