holoviz / datashader

Quickly and accurately render even the largest data.
http://datashader.org
BSD 3-Clause "New" or "Revised" License
3.31k stars 366 forks source link

Aggregating lines with column aggregator breaks on cuDF #845

Open philippjfr opened 4 years ago

philippjfr commented 4 years ago

When trying to perform line aggregation on a cuDF and specify an aggregator with a column you get an error:

import cudf
import datashader as ds

nyc_df = cudf.read_parquet('~/Downloads/nyc_taxi_wide.parq')

cvs = ds.Canvas()

cvs.line(nyc_df, ['pickup_x', 'pickup_y'], ['dropoff_x', 'dropoff_y'], axis=1, agg=ds.mean('trip_distance'))
~/development/datashader/datashader/compiler.py in <genexpr>(.0)
     96 
     97 def make_info(cols):
---> 98     return lambda df: tuple(c.apply(df) for c in cols)
     99 
    100 

~/development/datashader/datashader/reductions.py in apply(self, df)
     36             else:
     37                 nullval = 0
---> 38             return cupy.array(df[self.column].to_gpu_array(fillna=nullval))
     39         else:
     40             return df[self.column].values

~/miniconda3/envs/rapidsai/lib/python3.7/site-packages/cupy/creation/from_data.py in array(obj, dtype, copy, order, subok, ndmin)
     39 
     40     """
---> 41     return core.array(obj, dtype, copy, order, subok, ndmin)
     42 
     43 

cupy/core/core.pyx in cupy.core.core.array()

cupy/core/core.pyx in cupy.core.core.array()

cupy/core/core.pyx in cupy.core.core._convert_object_with_cuda_array_interface()

TypeError: zip argument #2 must support iteration

cupy version: '6.3.0' cudf version: '0.10.0'

jonmmease commented 4 years ago

Thanks @philippjfr, do you have numba 0.46 installed? If so, can you try downgrading to 0.45? As I mentioned in https://github.com/holoviz/datashader/pull/841, there seems to be an incompatibility between numba 0.46 and the current stable release of cupy, which should be fixed with the next cupy release. In the meantime, GPU support will require numba < 0.46.