In order to narrow in on possible sources for memory issues, here is a simple approach for reading all input that strips out coffea, uproot.dask, and dask-awkward. This is just Dask distributing Python functions that read the data through uproot.open. It seems to run rather well and is stable in the tests I have done so far.
This PR adds Dask distributing uproot.open, Dask distributing xrdcp, plus new functionality to track worker counts in the background to determine scheduling efficiencies / overhead.
In order to narrow in on possible sources for memory issues, here is a simple approach for reading all input that strips out coffea, uproot.dask, and dask-awkward. This is just Dask distributing Python functions that read the data through
uproot.open. It seems to run rather well and is stable in the tests I have done so far.This PR adds Dask distributing
uproot.open, Dask distributingxrdcp, plus new functionality to track worker counts in the background to determine scheduling efficiencies / overhead.