search

dask-contrib / dask-awkward

Native Dask collection for awkward arrays, and the library to use it.
https://dask-awkward.readthedocs.io
BSD 3-Clause "New" or "Revised" License
61 stars 19 forks source link

WIP: Use threaded dask for optimize pass #514

Closed martindurant closed 3 months ago

martindurant commented 5 months ago

As described in slack, this change appears to deliver ~30% faster optimization on my system, without breaking anything. I am posting here, so that others can test more easily.

martindurant commented 5 months ago

I have tried this now with various queries out of https://github.com/CoffeaTeam/coffea-benchmarks/blob/master/coffea-adl-benchmarks.ipynb (some of which should be sufficiently complex) and I am no longer sure there is a benefit. Would be good to try with even bigger coffea workflows.