search

dask / community

For general discussion and community planning. Discussion issues welcome.
19 stars 3 forks source link

Release 2022.10.1 #283

Closed jrbourbeau closed 1 year ago

jrbourbeau commented 1 year ago
Release version 2022.10.1
Planned release date 2022-10-28
Status On-track
Release manager @jrbourbeau

Best effort

Try to close before the release but will not block the release

Blocker

Issues that would cause us to block and postpone the release if not fixed

Comments

cc @ian-r-rose @fjetter @quasiben @jakirkham

douglasdavis commented 1 year ago

Digging a bit with the CLI issue..

After installing dask and distributed with conda-forge

If I run python -m dask...

$ python -m dask

shows what we want:

Usage: python -m dask [OPTIONS] COMMAND [ARGS]...

  Dask command line interface.

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  docs       Open Dask documentation (https://docs.dask.org/) in a web browser.
  info       Information about your dask installation.
  scheduler  Launch a distributed scheduler.
  ssh        Launch a distributed cluster over SSH.
  worker     Launch a distributed worker attached to an existing SCHEDULER.

but

$ dask

shows

Usage: dask [OPTIONS] COMMAND [ARGS]...

  Dask command line interface.

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  docs  Open Dask documentation (https://docs.dask.org/) in a web browser.
  info  Information about your dask installation.

$ python -m dask is running dask.__main__:main while $ dask is running dask.cli:cli. (note the pip(setuptools) installation uses dask.__main__:main). I think the conda-forge entry point should be dask.__main__:main

douglasdavis commented 1 year ago

Opened a PR conda-forge/dask-feedstock#200 fixed in dask-core-feedstock!

jakirkham commented 1 year ago

Thanks all for helping dig into the CLI issues! 🙏

Given the pain around that, would like to propose hardening the Click requirement in Dask going forward ( https://github.com/dask/dask/pull/9595 ). Ideally would like to include that in this release

jrbourbeau commented 1 year ago

I've opened this issue around requiring python<3.11 https://github.com/dask/dask/issues/9605. If folks have thoughts on the topic I'd appreciate feedback on that issue.

jrbourbeau commented 1 year ago

We haven't quite gotten https://github.com/dask/distributed/pull/5648 over the finish line yet, so I'm going to push up PRs to put an upper bound of bokeh < 3 for dask and distributed given that we know there are compatibility issues with the bokeh=3 release scheduled to come out later today (cc @fjetter @crusaderky)

mrocklin commented 1 year ago

@gjoseph92 @fjetter are we turning on task queuing on the scheduler by default in this release?

gjoseph92 commented 1 year ago

No, it won't be ready in time. See https://github.com/dask/distributed/issues/7213.

jrbourbeau commented 1 year ago

bokeh < 3 PRs:

gjoseph92 commented 1 year ago

@jrbourbeau not critical, but could be good to get https://github.com/dask/distributed/pull/7224 in if CI passes and you have time to review.

jrbourbeau commented 1 year ago

Ah, sorry @gjoseph92 I need to start pushing out the release. Unfortunately I think that one will have to get in in the next release

gjoseph92 commented 1 year ago

No worries, queuing is still behind a feature flag so it's not too big of a deal.

jrbourbeau commented 1 year ago

2022.10.1 is out on PyPI and conda-forge PRs are open:

jrbourbeau commented 1 year ago

Alright, PyPI and conda-forge packages are released.

We ran into a problem building the Python 3.8 dask-notebook image over in dask-docker (xref https://github.com/dask/dask-docker/pull/255) due to an issue in the underlying jupyter/base-notebook notebook (xref https://github.com/jupyter/docker-stacks/issues/1810). I've attempted a couple of possible fixes, but haven't gotten those build to work yet. I've proposed a temporary workaround in https://github.com/dask/dask-docker/pull/256 where we drop Python 3.8 builds. This is a possible workaround, but admittedly feels a bit heavy handed. I'm on PTO next week -- @jacobtomlinson @jakirkham @charlesbluca could I ask you to handle docker images for this release by either finding an actual fix, or by merging the PR that temporarily drops Python 3.8 builds?

jakirkham commented 1 year ago

FWIW commented on the upstream (Jupyter) issue.

mrocklin commented 1 year ago

Heads up: https://github.com/dask/dask/issues/9609

Does anyone here have time to investigate what happened here with Bokeh? It appears that we're affecting Airflow negatively.

On Sat, Oct 29, 2022 at 3:53 AM jakirkham @.***> wrote:

FWIW commented on the upstream (Jupyter) issue.

— Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/283#issuecomment-1295775628, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTFYPEGQJAQRJA6DSWLWFTQYXANCNFSM6AAAAAAROE5LGY . You are receiving this because you commented.Message ID: @.***>

mrocklin commented 1 year ago

Pushing out a hotfix https://github.com/dask/community/issues/284

If there is anything I should know folks should jump in very soon.

ian-r-rose commented 1 year ago

There isn't anything I'm aware of that would prevent a hotfix, though dask/distributed#7224 is a non-trivial change. Perhaps @gjoseph92 can comment on whether it's likely to be an issue (I suspect he'd be fine with it)

jrbourbeau commented 1 year ago

I think @gjoseph92 is comfortable including that change has he asked for it to be included in the 2022.10.1 release, but it just didn't make it in time (xref https://github.com/dask/community/issues/283#issuecomment-1295438871)

EDIT: queuing is also still behind a feature flag, so I think it's safe to move forward with that change included

mrocklin commented 1 year ago

I've already chatted with @gjoseph92 and he confirmed.

I've yanked this release from PyPI. Can someone (maybe @jakirkham) help with yanking from conda-forge channels?

charlesbluca commented 1 year ago

could I ask you to handle docker images for this release by either finding an actual fix, or by merging the PR that temporarily drops Python 3.8 builds?

Any opposition to unblocking the 3.8 Docker builds locally with https://github.com/dask/dask-docker/pull/257 until the upstream issue is resolved? This would involve patching the impacted Jupyter dockerfile, which could get hairy in the long run, but it doesn't seem like CI is running very often on dask-docker for this to be an issue.

jakirkham commented 1 year ago

Would follow these steps for marking packages broken in conda-forge. Unfortunately I don't have bandwidth for this atm

charlesbluca commented 1 year ago

Thanks @jakirkham! Opened up https://github.com/conda-forge/admin-requests/pull/504 to request the packages be marked as broken

jakirkham commented 1 year ago

As to Jupyter docker-stacks, outlined a path forward in comment ( https://github.com/jupyter/docker-stacks/issues/1810#issuecomment-1297546767 ). If anyone has bandwidth to try, should be a couple line change

jakirkham commented 1 year ago

Updating this thread, Charles' workaround was merged yesterday. Mamba released 1.0.0 early today. This subsequently fixed Jupyter docker-stacks ( https://github.com/jupyter/docker-stacks/issues/1810#issuecomment-1300850292 ). With that fixed, Charles reverted the workaround ( https://github.com/dask/dask-docker/pull/260 ).

Edit: Also merged Charles PR yesterday to mark conda-forge packages broken.

jakirkham commented 1 year ago

Sounds like there is nothing more to do here. So going to close this out, but please feel free to reopen if I've missed something.

Thanks all! 🙏