search

dask / community

For general discussion and community planning. Discussion issues welcome.
19 stars 3 forks source link

Release 2024.4.1 #371

Closed jrbourbeau closed 3 months ago

jrbourbeau commented 3 months ago
Release version 2024.4.1
Planned release date 2024-04-04
Status On-track
Release manager @jrbourbeau

Best effort

Try to close before the release but will not block the release

Blocker

Issues that would cause us to block and postpone the release if not fixed

Comments

The 2024.4.0 release, import dask.dataframe raises an error on Python 3.11.9 (other Python versions are okay) https://github.com/dask/dask/pull/11035. Things have been fixed on main and I'd like to push out a new release later today with the fix.

cc @fjetter @phofl @rjzamora @quasiben @hendrikmakait @crusaderky

rjzamora commented 3 months ago

Just a note that https://github.com/dask/dask/pull/11035 may not have correctly resolved the problem. We are now seeing <function PandasDelegate._add_delegate_accessors.<locals>._create_delegator_method.<locals>.f at 0x7f0ecacbf560> - Will try to follow up soon.

jrbourbeau commented 3 months ago

Thanks for the heads up @rjzamora. I'll hold off on releasing until we're sure things have been resolved.

rjzamora commented 3 months ago

Okay - It looks like the problem I reported above was likely due to a corrupted environment. Someone internal reported that they were hitting this new TypeError. However, they are no longer able to reproduce the problem after a fresh build. We could always remove the new TypeError to be safe, but I'm not sure it's necessary.

crusaderky commented 3 months ago

Best effort:

phofl commented 3 months ago

We got the same report from someone else https://github.com/dask/dask/issues/11038

cc @rjzamora so I think the failure was legit?

rjzamora commented 3 months ago

@phofl - Yes, the DatetimeAccessor error was very real. We were also seeing a PandasDelegate error after 11035 was merged, but it seems to have been a local environment problem.

jrbourbeau commented 3 months ago

Okay, so sounds like https://github.com/dask/dask/pull/11035 is needed and there's not a known related follow up needed either. I'll wait a bit for further feedback and then will start pushing this release out

rjzamora commented 3 months ago

@jrbourbeau - I submitted https://github.com/dask/dask/pull/11039 to make sure we don't run into the hypothetical problem I mentioned. Feel free to ignore, but might be worth merging first.

rjzamora commented 3 months ago

On a related note: I wonder how many users/organizations are temporarily stuck on 2024.1.1 until they have had time to migrate to the new dask-expr API?

For the impending rapids 24.04 release, we are pouring many engineering hours into finding a way to patch/vendor our own version of dask-2024.1.1 with the python 3.10 fix. We originally considered this to be our own problem to deal with. However, I now strongly suspect that many other organizations would benefit from a patched 2024.1.1 release. Any thoughts on this?

jrbourbeau commented 3 months ago

FYI 2024.4.1 is out on PyPI (conda-forge + docker soon)

jrbourbeau commented 3 months ago

FYI it appears there were some CDN issues with conda-forge today and dask-core=2024.4.1 still isn't available to download (cc @jakirkham in case I'm misunderstanding). I'll try bumping the remaining feedstocks tomorrow

jrbourbeau commented 3 months ago

Closing as 2024.4.1 is out on conda-forge and docker registries now

On a related note: I wonder how many users/organizations are temporarily stuck on 2024.1.1 until they have had time to migrate to the new dask-expr API?

@phofl probably has a better sense than I do for this. I've mostly seen folks turning query planning off who run into issues for one reason or another (though issues have been quickly resolved in dask-expr too)

Maybe @quasiben @charlesbluca can do something similar to https://github.com/dask/community/issues/316 to help. Regardless, this seems like a separate issue, so I'm going to close this issue out (hope that's okay)