iris-hep / idap-200gbps-atlas

benchmarking throughput with PHYSLITE
6 stars 1 forks source link

Debugging long wait time after task completion #147

Open alexander-held opened 2 months ago

alexander-held commented 2 months ago

At full scale there can be up to a few minutes of wait time after task completion, plus some wait time until the dashboard initially starts showing tasks. We should try out batch_size as described in https://github.com/dask/distributed/issues/2181.