dask / distributed

A distributed task scheduler for Dask
https://distributed.dask.org
BSD 3-Clause "New" or "Revised" License
1.55k stars 712 forks source link

Disable `TaskProgress` dashboard component if there are too many task groups #8677

Closed hendrikmakait closed 2 weeks ago

hendrikmakait commented 2 weeks ago

Problem

Updating the TaskProgress involves looping over all task groups, which may become prohibitively expensive if we have too many task groups: scheduler-profile

Solution

We should disable this component based on some configurable threshold to avoid overwhelming the scheduler CPU.

hendrikmakait commented 2 weeks ago

Looking at the code more closely...why do we even iterate over the task groups instead of just keeping the aggregate counts on the prefix? We already keep accumulated counts in TaskPrefix.state_counts.

hendrikmakait commented 2 weeks ago

Closed as superseded by #8680.