holoviz / datashader

Quickly and accurately render even the largest data.
http://datashader.org
BSD 3-Clause "New" or "Revised" License
3.3k stars 366 forks source link

Support summary containing by reduction with other reductions #1257

Closed ianthomas23 closed 1 year ago

ianthomas23 commented 1 year ago

Fixes #1256.

Adds support for the following combinations of summary and by reductions:

  1. summary(by("cat1"), other_reduction) and with the order swapped.
  2. summary(by("cat1"), by("cat1"))
  3. summary(by("cat1"), by("cat2"))

The biggest change here is in the handling of categorical columns in compiler.py make_append(). Previously the categorical flag was constant for all reductions within a summary, now it is checked for each constituent reduction. Where categorical columns are reused, they are only extracted once.

Also, the addition of kwargs to xr.DataArray in by.finalize has been modified to follow the same approach as in where reductions, which is to add the new categorical coords/dims to a copy of the kwargs, not modify them in-place. Hence each constituent reduction of a summary can have different categorical coordinates from others.

Tested on CPU and GPU, with and without Dask.

codecov[bot] commented 1 year ago

Codecov Report

Merging #1257 (e45fee4) into main (4a1c3fc) will decrease coverage by 0.04%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1257      +/-   ##
==========================================
- Coverage   83.44%   83.41%   -0.04%     
==========================================
  Files          35       35              
  Lines        8882     8886       +4     
==========================================
  Hits         7412     7412              
- Misses       1470     1474       +4     
Impacted Files Coverage Δ
datashader/compiler.py 88.55% <100.00%> (+0.17%) :arrow_up:
datashader/reductions.py 77.80% <100.00%> (-0.27%) :arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more