Closed quasiben closed 2 years ago
@jakirkham and @charlesbluca have successfully implemented nightly cythonized builds for Distributed. Folks can test in a new or existing environment with following snippet:
mamba/conda install -c dask/label/dev distributed-impl==cython
The goal for cythonizing the scheduler is to improve performance where the number of tasks is so large that the scheduler itself becomes a bottle neck
In addition to announcing here would anyone object to me sending out a tweet ?
No objection from me. The hope is, people try to benchmark some workloads and let us know to what degree cython is making a positive difference.
@jakirkham and @charlesbluca have successfully implemented nightly cythonized builds for Distributed. Folks can test in a new or existing environment with following snippet:
The goal for cythonizing the scheduler is to improve performance where the number of tasks is so large that the scheduler itself becomes a bottle neck
In addition to announcing here would anyone object to me sending out a tweet ?