holoviz / datashader

Quickly and accurately render even the largest data.
http://datashader.org
BSD 3-Clause "New" or "Revised" License
3.24k stars 363 forks source link

Cannot use by reduction within a summary reduction #1252

Closed ianthomas23 closed 11 months ago

ianthomas23 commented 11 months ago

Aggregations that use a by reduction within a summary, such as ds.summary(name=ds.by("cat")) do not work. Tested with datashader 0.15.1.

Test code:

import datashader as ds
import numpy as np
import pandas as pd

x = np.arange(2)
df = pd.DataFrame(dict(
    y_from = [0.0, 1.0, 0.0, 1.0, 0.0],
    y_to   = [0.0, 1.0, 1.0, 0.0, 0.5],
    cat    = ['a', 'b', 'a', 'b', 'a'],
))
df["cat"] = df["cat"].astype("category")

canvas = ds.Canvas(10, 10)
agg = canvas.line(
    source=df, x=x, y=["y_from", "y_to"], axis=1,
    agg=ds.summary(other=ds.by("cat")),
)

Error produced is:

<snip>
Internal error at <numba.core.typeinfer.CallConstraint object at 0x124115890>.
too many positional arguments
During: resolving callee type: type(CPUDispatcher(<function count._append_no_field at 0x1217eed40>))
<snip>

If numba is disabled the error message is more informative:

<snip>
  File "/Users/iant/.miniconda3/envs/ds0.15.1_3.11/lib/python3.11/site-packages/datashader/glyphs/line.py", line 897, in _bresenham
    append(i, x0, y0, *aggs_and_cols)
  File "<string>", line 2, in append
TypeError: count._append_no_field() takes 3 positional arguments but 4 were given

The same error occurs using datashader 0.13.0 and 0.14.0, so I assume it has never worked.

ianthomas23 commented 11 months ago

I have a fix for this, but I will wait until #1251 fixes the CI.