When rechunking data with P2P in a pattern where each output depends only on a few inputs, P2P causes the task group count to explode because it splits the rechunk into several independent partial rechunks, each having its own task groups. This can cause significant strain on the scheduler.
When rechunking data with P2P in a pattern where each output depends only on a few inputs, P2P causes the task group count to explode because it splits the rechunk into several independent partial rechunks, each having its own task groups. This can cause significant strain on the scheduler.
Reproducer:
fails with
AssertionError: assert 190 < 7
. 6 would be the count if all independent partials shared their rechunks.