Support for KMeans with an async Client

JavierYepez commented 2 years ago

Hi,

I'd like to ask if It's possible to add a version of KMeans that works with an async client.

Here is an example that should work:

import distributed
import dask_ml
import dask_ml.datasets
import dask_ml.cluster

client = distributed.Client(asynchronous=True)
await client

X, y = dask_ml.datasets.make_blobs(n_samples=10000000,
                                   chunks=1000000,
                                   random_state=0,
                                   centers=3)
km = dask_ml.cluster.KMeans(n_clusters=3, init_max_iter=2, oversampling_factor=10)
km.fit(X)

Thank you

TomAugspurger commented 2 years ago

Should be doable, as long as nothing in the .fit method is blocking on the main thread.

On Dec 24, 2021, at 10:03 AM, Javier Yepez @.***> wrote:

Hi,

I'd like to ask if It's possible to add a version of KMeans that works with an async client.

Here is an example that should work:

import distributed import dask_ml import dask_ml.datasets import dask_ml.cluster

client = distributed.Client(asynchronous=True) await client

X, y = dask_ml.datasets.make_blobs(n_samples=10000000, chunks=1000000, random_state=0, centers=3) km = dask_ml.cluster.KMeans(n_clusters=3, init_max_iter=2, oversampling_factor=10) km.fit(X) Thank you

— Reply to this email directly, view it on GitHub https://github.com/dask/dask-ml/issues/897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAOIVMPDCZ7G6SLEBDGI3USSKUJANCNFSM5KWZ35MA. You are receiving this because you are subscribed to this thread.

inesortega commented 2 years ago

Hi! any plans on implementing this feature?

TomAugspurger commented 2 years ago

I don’t plan to work on it. A PR would be welcome, but we’ll need to find some help reviewing it: I’m not really sure what the best way to mix sync and async code with an async client.

On Mar 11, 2022 at 2:07:51 AM, Inés Ortega-Fernández < @.***> wrote:

Hi! any plans on implementing this feature?

— Reply to this email directly, view it on GitHub https://github.com/dask/dask-ml/issues/897#issuecomment-1064871314, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAOIWY6TAHMTF2I5ICYDLU7L5NPANCNFSM5KWZ35MA . You are receiving this because you commented.Message ID: @.***>

dask / dask-ml

Support for KMeans with an async Client #897