philippjfr
commented
1 month ago So the proximate cause here is that while this used to work I actually suspect it didn't work as desired. The problem now is that the individual categories now become the dimensions of the ImageStack while previously they were merely the values of the kdim of the NdOverlay that would hold the individual layers. The real problem/question here is how the empty strings should be treated, specifically HoloViews upgrades all unique non-NaN values along the count_cat dimension to categories and the resulting DataArray will then hold all these categories INCLUDING the empty string category. This doesn't actually cause any rendering artifacts because all the empty string category coordinates are NaNs but apparently we have always created this bogus category in this case.
I suspect we should either automatically convert the empty strings to NaNs so they are not treated as a valid category OR we should issue an exception indicating that '' is not a valid category value.
The code below:
With some pretty old versions of HoloViews, Datashader and Pandas, it works.
But it breaks with newer versions with this error
ValueError: Dimension name cannot be empty.git bisectpoints to https://github.com/holoviz/holoviews/pull/5751.CAT_SEP = ''withCAT_SEP = np.nanmakes the code work now. So I wonder, should all the lines containNaN, i.e. is the original example just bad data? Or is it just a regression?Before:
Traceback now
```python > / Name Modified File Size spaths = [ np.array([[1, 2], [3, 4], [5, 6]]), np.array([[7, 8], [9, 10]]), np.array([[11, 12], [13, 14], [15, 16], [17, 18]]), np.array([[19, 20], [21, 22]]), np.array([[23, 24], [25, 26]]), ] nan_row = np.array([[np.nan, np.nan]]) paths = np.concatenate([x for spath in spaths for x in [spath, nan_row]]) df = pd.DataFrame(paths, columns=['x', 'y']) # CAT_SEP = np.nan CAT_SEP = '' origin = [] for s, spath in zip('abcdefgh', spaths): for _ in range(len(spath)): origin.append(s) origin.append(CAT_SEP) df['origin'] = origin df['origin'] = df['origin'].astype('category') path = hv.Path(df, ['x', 'y']) path aggregator = ds.count_cat('origin') datashaded = hd.datashade( path, aggregator=aggregator, ) datashaded.opts(bgcolor='black') WARNING:param.dynamic_operation: Callable raised "ValueError('Dimension name cannot be empty')". Invoked as dynamic_operation(height=400, scale=1.0, width=400, x_range=None, y_range=None) WARNING:param.dynamic_operation: Callable raised "ValueError('Dimension name cannot be empty')". Invoked as dynamic_operation(height=400, scale=1.0, width=400, x_range=None, y_range=None) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) File [~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/IPython/core/formatters.py:974](http://localhost:8890/lab/tree/~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/IPython/core/formatters.py#line=973), in MimeBundleFormatter.__call__(self, obj, include, exclude) 971 method = get_real_method(obj, self.print_method) 973 if method is not None: --> 974 return method(include=include, exclude=exclude) 975 return None 976 else: File [~/dev/holoviews/holoviews/core/dimension.py:1275](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/dimension.py#line=1274), in Dimensioned._repr_mimebundle_(self, include, exclude) 1268 def _repr_mimebundle_(self, include=None, exclude=None): 1269 """ 1270 Resolves the class hierarchy for the class rendering the 1271 object using any display hooks registered on Store.display 1272 hooks. The output of all registered display_hooks is then 1273 combined and returned. 1274 """ -> 1275 return Store.render(self) File [~/dev/holoviews/holoviews/core/options.py:1423](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/options.py#line=1422), in Store.render(cls, obj) 1421 data, metadata = {}, {} 1422 for hook in hooks: -> 1423 ret = hook(obj) 1424 if ret is None: 1425 continue File [~/dev/holoviews/holoviews/ipython/display_hooks.py:287](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/ipython/display_hooks.py#line=286), in pprint_display(obj) 285 if not ip.display_formatter.formatters['text[/plain](http://localhost:8890/plain)'].pprint: 286 return None --> 287 return display(obj, raw_output=True) File [~/dev/holoviews/holoviews/ipython/display_hooks.py:261](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/ipython/display_hooks.py#line=260), in display(obj, raw_output, **kwargs) 259 elif isinstance(obj, (HoloMap, DynamicMap)): 260 with option_state(obj): --> 261 output = map_display(obj) 262 elif isinstance(obj, Plot): 263 output = render(obj) File [~/dev/holoviews/holoviews/ipython/display_hooks.py:149](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/ipython/display_hooks.py#line=148), in display_hook..wrapped(element)
147 try:
148 max_frames = OutputSettings.options['max_frames']
--> 149 mimebundle = fn(element, max_frames=max_frames)
150 if mimebundle is None:
151 return {}, {}
File [~/dev/holoviews/holoviews/ipython/display_hooks.py:209](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/ipython/display_hooks.py#line=208), in map_display(vmap, max_frames)
206 max_frame_warning(max_frames)
207 return None
--> 209 return render(vmap)
File [~/dev/holoviews/holoviews/ipython/display_hooks.py:76](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/ipython/display_hooks.py#line=75), in render(obj, **kwargs)
73 if renderer.fig == 'pdf':
74 renderer = renderer.instance(fig='png')
---> 76 return renderer.components(obj, **kwargs)
File [~/dev/holoviews/holoviews/plotting/renderer.py:396](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/plotting/renderer.py#line=395), in Renderer.components(self, obj, fmt, comm, **kwargs)
394 embed = (not (dynamic or streams or self.widget_mode == 'live') or config.embed)
395 if embed or config.comms == 'default':
--> 396 return self._render_panel(plot, embed, comm)
397 return self._render_ipywidget(plot)
File [~/dev/holoviews/holoviews/plotting/renderer.py:403](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/plotting/renderer.py#line=402), in Renderer._render_panel(self, plot, embed, comm)
401 doc = Document()
402 with config.set(embed=embed):
--> 403 model = plot.layout._render_model(doc, comm)
404 if embed:
405 return render_model(model, comm)
File [~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/viewable.py:735](http://localhost:8890/lab/tree/~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/viewable.py#line=734), in Viewable._render_model(self, doc, comm)
733 if comm is None:
734 comm = state._comm_manager.get_server_comm()
--> 735 model = self.get_root(doc, comm)
737 if self._design and self._design.theme.bokeh_theme:
738 doc.theme = self._design.theme.bokeh_theme
File [~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/layout/base.py:319](http://localhost:8890/lab/tree/~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/layout/base.py#line=318), in Panel.get_root(self, doc, comm, preprocess)
315 def get_root(
316 self, doc: Optional[Document] = None, comm: Optional[Comm] = None,
317 preprocess: bool = True
318 ) -> Model:
--> 319 root = super().get_root(doc, comm, preprocess)
320 # ALERT: Find a better way to handle this
321 if hasattr(root, 'styles') and 'overflow-x' in root.styles:
File [~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/viewable.py:666](http://localhost:8890/lab/tree/~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/viewable.py#line=665), in Renderable.get_root(self, doc, comm, preprocess)
664 wrapper = self._design._wrapper(self)
665 if wrapper is self:
--> 666 root = self._get_model(doc, comm=comm)
667 if preprocess:
668 self._preprocess(root)
File [~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/layout/base.py:185](http://localhost:8890/lab/tree/~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/layout/base.py#line=184), in Panel._get_model(self, doc, root, parent, comm)
183 root = root or model
184 self._models[root.ref['id']] = (model, parent)
--> 185 objects, _ = self._get_objects(model, [], doc, root, comm)
186 props = self._get_properties(doc)
187 props[self._property_mapping['objects']] = objects
File [~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/layout/base.py:167](http://localhost:8890/lab/tree/~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/layout/base.py#line=166), in Panel._get_objects(self, model, old_objects, doc, root, comm)
165 else:
166 try:
--> 167 child = pane._get_model(doc, root, model, comm)
168 except RerenderError as e:
169 if e.layout is not None and e.layout is not self:
File [~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/pane/holoviews.py:429](http://localhost:8890/lab/tree/~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/pane/holoviews.py#line=428), in HoloViews._get_model(self, doc, root, parent, comm)
427 plot = self.object
428 else:
--> 429 plot = self._render(doc, comm, root)
431 plot.pane = self
432 backend = plot.renderer.backend
File [~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/pane/holoviews.py:525](http://localhost:8890/lab/tree/~/dev/holoviews/.pixi/envs/test-39/lib/python3.9/site-packages/panel/pane/holoviews.py#line=524), in HoloViews._render(self, doc, comm, root)
522 if comm:
523 kwargs['comm'] = comm
--> 525 return renderer.get_plot(self.object, **kwargs)
File [~/dev/holoviews/holoviews/plotting/bokeh/renderer.py:68](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/plotting/bokeh/renderer.py#line=67), in BokehRenderer.get_plot(self_or_cls, obj, doc, renderer, **kwargs)
61 @bothmethod
62 def get_plot(self_or_cls, obj, doc=None, renderer=None, **kwargs):
63 """
64 Given a HoloViews Viewable return a corresponding plot instance.
65 Allows supplying a document attach the plot to, useful when
66 combining the bokeh model with another plot.
67 """
---> 68 plot = super().get_plot(obj, doc, renderer, **kwargs)
69 if plot.document is None:
70 plot.document = Document() if self_or_cls.notebook_context else curdoc()
File [~/dev/holoviews/holoviews/plotting/renderer.py:216](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/plotting/renderer.py#line=215), in Renderer.get_plot(self_or_cls, obj, doc, renderer, comm, **kwargs)
213 raise SkipRendering(msg.format(dims=dims))
215 # Initialize DynamicMaps with first data item
--> 216 initialize_dynamic(obj)
218 if not renderer:
219 renderer = self_or_cls
File [~/dev/holoviews/holoviews/plotting/util.py:270](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/plotting/util.py#line=269), in initialize_dynamic(obj)
268 continue
269 if not len(dmap):
--> 270 dmap[dmap._initial_key()]
File [~/dev/holoviews/holoviews/core/spaces.py:1216](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/spaces.py#line=1215), in DynamicMap.__getitem__(self, key)
1214 # Not a cross product and nothing cached so compute element.
1215 if cache is not None: return cache
-> 1216 val = self._execute_callback(*tuple_key)
1217 if data_slice:
1218 val = self._dataslice(val, data_slice)
File [~/dev/holoviews/holoviews/core/spaces.py:983](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/spaces.py#line=982), in DynamicMap._execute_callback(self, *args)
980 kwargs['_memoization_hash_'] = hash_items
982 with dynamicmap_memoization(self.callback, self.streams):
--> 983 retval = self.callback(*args, **kwargs)
984 return self._style(retval)
File [~/dev/holoviews/holoviews/core/spaces.py:581](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/spaces.py#line=580), in Callable.__call__(self, *args, **kwargs)
578 args, kwargs = (), dict(pos_kwargs, **kwargs)
580 try:
--> 581 ret = self.callable(*args, **kwargs)
582 except KeyError:
583 # KeyError is caught separately because it is used to signal
584 # invalid keys on DynamicMap and should not warn
585 raise
File [~/dev/holoviews/holoviews/util/__init__.py:1038](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/util/__init__.py#line=1037), in Dynamic._dynamic_operation..dynamic_operation(*key, **kwargs)
1037 def dynamic_operation(*key, **kwargs):
-> 1038 key, obj = resolve(key, kwargs)
1039 return apply(obj, *key, **kwargs)
File [~/dev/holoviews/holoviews/util/__init__.py:1027](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/util/__init__.py#line=1026), in Dynamic._dynamic_operation..resolve(key, kwargs)
1025 elif isinstance(map_obj, DynamicMap) and map_obj._posarg_keys and not key:
1026 key = tuple(kwargs[k] for k in map_obj._posarg_keys)
-> 1027 return key, map_obj[key]
File [~/dev/holoviews/holoviews/core/spaces.py:1216](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/spaces.py#line=1215), in DynamicMap.__getitem__(self, key)
1214 # Not a cross product and nothing cached so compute element.
1215 if cache is not None: return cache
-> 1216 val = self._execute_callback(*tuple_key)
1217 if data_slice:
1218 val = self._dataslice(val, data_slice)
File [~/dev/holoviews/holoviews/core/spaces.py:983](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/spaces.py#line=982), in DynamicMap._execute_callback(self, *args)
980 kwargs['_memoization_hash_'] = hash_items
982 with dynamicmap_memoization(self.callback, self.streams):
--> 983 retval = self.callback(*args, **kwargs)
984 return self._style(retval)
File [~/dev/holoviews/holoviews/core/spaces.py:581](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/spaces.py#line=580), in Callable.__call__(self, *args, **kwargs)
578 args, kwargs = (), dict(pos_kwargs, **kwargs)
580 try:
--> 581 ret = self.callable(*args, **kwargs)
582 except KeyError:
583 # KeyError is caught separately because it is used to signal
584 # invalid keys on DynamicMap and should not warn
585 raise
File [~/dev/holoviews/holoviews/util/__init__.py:1039](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/util/__init__.py#line=1038), in Dynamic._dynamic_operation..dynamic_operation(*key, **kwargs)
1037 def dynamic_operation(*key, **kwargs):
1038 key, obj = resolve(key, kwargs)
-> 1039 return apply(obj, *key, **kwargs)
File [~/dev/holoviews/holoviews/util/__init__.py:1031](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/util/__init__.py#line=1030), in Dynamic._dynamic_operation..apply(element, *key, **kwargs)
1029 def apply(element, *key, **kwargs):
1030 kwargs = dict(util.resolve_dependent_kwargs(self.p.kwargs), **kwargs)
-> 1031 processed = self._process(element, key, kwargs)
1032 if (self.p.link_dataset and isinstance(element, Dataset) and
1033 isinstance(processed, Dataset) and processed._dataset is None):
1034 processed._dataset = element.dataset
File [~/dev/holoviews/holoviews/util/__init__.py:1013](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/util/__init__.py#line=1012), in Dynamic._process(self, element, key, kwargs)
1011 elif isinstance(self.p.operation, Operation):
1012 kwargs = {k: v for k, v in kwargs.items() if k in self.p.operation.param}
-> 1013 return self.p.operation.process_element(element, key, **kwargs)
1014 else:
1015 return self.p.operation(element, **kwargs)
File [~/dev/holoviews/holoviews/core/operation.py:194](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/operation.py#line=193), in Operation.process_element(self, element, key, **params)
191 else:
192 self.p = param.ParamOverrides(self, params,
193 allow_extra_keywords=self._allow_extra_keywords)
--> 194 return self._apply(element, key)
File [~/dev/holoviews/holoviews/core/operation.py:141](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/operation.py#line=140), in Operation._apply(self, element, key)
139 if not in_method:
140 element._in_method = True
--> 141 ret = self._process(element, key)
142 if hasattr(element, '_in_method') and not in_method:
143 element._in_method = in_method
File [~/dev/holoviews/holoviews/operation/datashader.py:1542](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/operation/datashader.py#line=1541), in datashade._process(self, element, key)
1541 def _process(self, element, key=None):
-> 1542 agg = rasterize._process(self, element, key)
1543 shaded = shade._process(self, agg, key)
1544 return shaded
File [~/dev/holoviews/holoviews/operation/datashader.py:1522](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/operation/datashader.py#line=1521), in rasterize._process(self, element, key)
1519 op = transform.instance(**{k:v for k,v in extended_kws.items()
1520 if k in transform.param})
1521 op._precomputed = self._precomputed
-> 1522 element = element.map(op, predicate)
1523 self._precomputed = op._precomputed
1525 unused_params = list(all_supplied_kws - all_allowed_kws)
File [~/dev/holoviews/holoviews/core/data/__init__.py:196](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/data/__init__.py#line=195), in PipelineMeta.pipelined..pipelined_fn(*args, **kwargs)
193 inst._in_method = True
195 try:
--> 196 result = method_fn(*args, **kwargs)
197 if PipelineMeta.disable:
198 return result
File [~/dev/holoviews/holoviews/core/data/__init__.py:1213](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/data/__init__.py#line=1212), in Dataset.map(self, *args, **kwargs)
1211 @wraps(LabelledData.map)
1212 def map(self, *args, **kwargs):
-> 1213 return super().map(*args, **kwargs)
File [~/dev/holoviews/holoviews/core/dimension.py:695](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/dimension.py#line=694), in LabelledData.map(self, map_fn, specs, clone)
693 return deep_mapped
694 else:
--> 695 return map_fn(self) if applies else self
File [~/dev/holoviews/holoviews/core/operation.py:214](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/operation.py#line=213), in Operation.__call__(self, element, **kwargs)
210 return element.clone([(k, self._apply(el, key=k))
211 for k, el in element.items()])
212 elif ((self._per_element and isinstance(element, Element)) or
213 (not self._per_element and isinstance(element, ViewableElement))):
--> 214 return self._apply(element)
215 elif 'streams' not in kwargs:
216 kwargs['streams'] = self.p.streams
File [~/dev/holoviews/holoviews/core/operation.py:141](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/operation.py#line=140), in Operation._apply(self, element, key)
139 if not in_method:
140 element._in_method = True
--> 141 ret = self._process(element, key)
142 if hasattr(element, '_in_method') and not in_method:
143 element._in_method = in_method
File [~/dev/holoviews/holoviews/operation/datashader.py:412](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/operation/datashader.py#line=411), in aggregate._process(self, element, key)
410 else:
411 params['vdims'] = list(map(str, agg.coords[agg_fn.column].data))
--> 412 return ImageStack(agg, **params)
File [~/dev/holoviews/holoviews/element/raster.py:574](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/element/raster.py#line=573), in ImageStack.__init__(self, data, kdims, vdims, **params)
572 elif isinstance(data, dict):
573 vdims = [Dimension(key) for key in data.keys() if key not in _kdims]
--> 574 super().__init__(data, kdims=kdims, vdims=vdims, **params)
File [~/dev/holoviews/holoviews/element/raster.py:279](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/element/raster.py#line=278), in Image.__init__(self, data, kdims, vdims, bounds, extents, xdensity, ydensity, rtol, **params)
276 else:
277 params['rtol'] = config.image_rtol
--> 279 Dataset.__init__(self, data, kdims=kdims, vdims=vdims, extents=extents, **params)
280 if not self.interface.gridded:
281 raise DataError(
282 f"{type(self).__name__} type expects gridded data, "
283 f"{self.interface.__name__} is columnar. "
284 "To display columnar data as gridded use the HeatMap "
285 "element or aggregate the data (e.g. using np.histogram2d)."
286 )
File [~/dev/holoviews/holoviews/core/data/__init__.py:325](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/data/__init__.py#line=324), in Dataset.__init__(self, data, kdims, vdims, **kwargs)
322 if input_transforms is None:
323 input_transforms = data._transforms
--> 325 kwargs.update(process_dimensions(kdims, vdims))
326 kdims, vdims = kwargs.get('kdims'), kwargs.get('vdims')
328 validate_vdims = kwargs.pop('_validate_vdims', True)
File [~/dev/holoviews/holoviews/core/dimension.py:117](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/dimension.py#line=116), in process_dimensions(kdims, vdims)
110 elif not isinstance(dims, list):
111 raise ValueError(
112 f"{group} argument expects a Dimension or list of dimensions, "
113 "specified as tuples, strings, dictionaries or Dimension "
114 f"instances, not a {type(dims).__name__} type. "
115 "Ensure you passed the data as the first argument."
116 )
--> 117 dimensions[group] = [asdim(d) for d in dims]
118 return dimensions
File [~/dev/holoviews/holoviews/core/dimension.py:117](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/dimension.py#line=116), in (.0)
110 elif not isinstance(dims, list):
111 raise ValueError(
112 f"{group} argument expects a Dimension or list of dimensions, "
113 "specified as tuples, strings, dictionaries or Dimension "
114 f"instances, not a {type(dims).__name__} type. "
115 "Ensure you passed the data as the first argument."
116 )
--> 117 dimensions[group] = [asdim(d) for d in dims]
118 return dimensions
File [~/dev/holoviews/holoviews/core/dimension.py:60](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/dimension.py#line=59), in asdim(dimension)
50 def asdim(dimension):
51 """Convert the input to a Dimension.
52
53 Args:
(...)
58 copy is performed if the input is already a Dimension.
59 """
---> 60 return dimension if isinstance(dimension, Dimension) else Dimension(dimension)
File [~/dev/holoviews/holoviews/core/dimension.py:276](http://localhost:8890/lab/tree/~/dev/holoviews/holoviews/core/dimension.py#line=275), in Dimension.__init__(self, spec, **params)
273 all_params.update(params)
275 if not all_params['name']:
--> 276 raise ValueError('Dimension name cannot be empty')
277 if not all_params['label']:
278 raise ValueError('Dimension label cannot be empty')
ValueError: Dimension name cannot be empty
```