search

dask / community

For general discussion and community planning. Discussion issues welcome.
20 stars 3 forks source link

Cythonized Builds #228

Closed quasiben closed 2 years ago

quasiben commented 2 years ago

@jakirkham and @charlesbluca have successfully implemented nightly cythonized builds for Distributed. Folks can test in a new or existing environment with following snippet:

mamba/conda install -c dask/label/dev distributed-impl==cython

The goal for cythonizing the scheduler is to improve performance where the number of tasks is so large that the scheduler itself becomes a bottle neck

In addition to announcing here would anyone object to me sending out a tweet ?

martindurant commented 2 years ago

No objection from me. The hope is, people try to benchmark some workloads and let us know to what degree cython is making a positive difference.