search

xarray-contrib / flox

Fast & furious GroupBy operations for dask.array
https://flox.readthedocs.io
Apache License 2.0
124 stars 18 forks source link

Optimize cohorts for blockwise distribution of groups #299

Closed dcherian closed 10 months ago

dcherian commented 10 months ago

Expected for some resampling workloads

In the future, we should automatically choose blockwise for the user.

| Change   | Before [bf936f52] <main>   | After [1f51a343] <cohorts-blockwise-opt>   |   Ratio | Benchmark (Parameter)                                      |
|----------|----------------------------|--------------------------------------------|---------|------------------------------------------------------------|
| -        | 1.86±0.06ms                | 651±10μs                                   |    0.35 | cohorts.PerfectBlockwiseResampling.time_find_group_cohorts |