search

dask / distributed

A distributed task scheduler for Dask
https://distributed.dask.org
BSD 3-Clause "New" or "Revised" License
1.58k stars 720 forks source link

Add `asynchronous: bool` flag to `get_client` #7658

Open fjetter opened 1 year ago

fjetter commented 1 year ago

Part of our API are various utility methods that are supposed to return the current, the default or a new client.

Top-level API is worker.py::get_client https://github.com/dask/distributed/blob/0a88b761be5887e9a72887c44e80f513cadf7020/distributed/worker.py#L2713-L2778

When this function is being used from an asynchronous context the returned client object is asynchronous=False by default and will basically be useless. For this purpose, it would be helpful to add this as an argument to get_client

My proposal would be, in the case of a default or current client to raise a RuntimeError in case the default/current doesn't correspond to the appropriate asnychronous setting.

This relates to https://github.com/PrefectHQ/prefect/issues/12971

scharlottej13 commented 1 year ago

Thanks again for opening this issue @fjetter! Any idea on when someone might have bandwidth to work on this?

dchudz commented 1 year ago

When this function is being used from an asynchronous context

In my experience, it's not just that, but also this function doesn't work at all from an async task.

So unless I'm doing something silly, making it work is also part of the work here.

shughes-uk commented 1 year ago

I got hit by this today, would be nice to merge the fix