Closed hendrikmakait closed 2 weeks ago
Looking at the code more closely...why do we even iterate over the task groups instead of just keeping the aggregate counts on the prefix? We already keep accumulated counts in TaskPrefix.state_counts
.
Closed as superseded by #8680.
Problem
Updating the
TaskProgress
involves looping over all task groups, which may become prohibitively expensive if we have too many task groups:Solution
We should disable this component based on some configurable threshold to avoid overwhelming the scheduler CPU.