search

holoviz / holoviews

With Holoviews, your data visualizes itself.
https://holoviews.org
BSD 3-Clause "New" or "Revised" License
2.7k stars 402 forks source link

Better curve decimation #5359

Closed jbednar closed 2 years ago

jbednar commented 2 years ago

HoloViews provides two operations for working with very large timeseries or other curves: decimate() and rasterize()/datashade(). The two algorithms work very differently: Decimate randomly drops rows, while rasterize renders the timeseries into a fixed-size array based on the screen resolution. They can be used together, but typically it's an either/or approach, if only so that users understand what is going on: either pre-rendering, or subsampling. Both work well interactively to reveal more detail dynamically when zooming in.

decimate can achieve arbitrarily high speedups, but inherently suffers from aliasing (e.g. Moire patterns), and is thus quite a poor approximation to the original data unless the original data was very smooth. rasterize preserves the full shape of the curve and can be much faster than the default Bokeh or Matplotlib rendering, due to its tight Numba-ized loops, but inherently it will have rendering time that scales with the size of the dataset.

As an additional alternative, we should consider offering a smarter form of decimation/downsampling in HoloViews that preserves more of the original structure, such as the Largest-Triangle Three-Buckets algorithm. LTTB discards samples in a region that are bounded by the max or min of other nearby samples, preserving the envelope of the signal while discarding intermediate values. Python implementations are available at:

https://git.sr.ht/~javiljoen/lttb-numpy https://github.com/devoxi/lttb-py

The plotly-resampler package implements LTTB as well as a faster variant using a novel heuristic for discarding points, and may also be relevant. At first glance, this seems like something that could easily be implemented as an operation...

jbednar commented 2 years ago

Looks like this is a duplicate of my own issue from 4 years ago: https://github.com/holoviz/holoviews/issues/2934 I'll close this in favor of the original.

github-actions[bot] commented 1 week ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.