Add more decimate options

[jbednar]

Right now, the HoloViews decimate operator randomly samples from the available data, dynamically on each zoom or pan. This is often a useful behavior, particularly for disconnected plots like Scatter, but for connected plots like Curve (e.g. timeseries) and Path, the results are chaotic and not a particularly good representation of the underlying data. For such cases, other options would be more useful:

• [ ] Regular sampling: Regularly spaced downsampling (slicing with a stride) should be trivial to implement and would give a good approximation to the shape of smooth curves.
• [ ] Outlier preserving: Some algorithms like Largest Triangle are better at preserving outliers, but may be slower and definitely would take more work to implement.
• [ ] Mean aggregation: It would be slower than sampling, but one could aggregate over the data points in each stride chunk to take the mean, so that all data points at least affect the result.

If we implemented multiple types of sampling, the policy could be selected by mode='random', mode='regular', etc. We might want the default to be mode=None, where a mode appropriate to that Element type (e.g. random for Scatter and Points) would be selected.

[ayoungs]

I'm definitely a fan of the slicing with stride downsampling option!

[jlstevens]

Not sure about the other two suggestions but regular downsampling should be a quick, easy, understandable and useful way to decimate instead of always applying decimation randomly.

[jbednar]

Yes, that's the low-hanging fruit; I listed the other ones just so it's clear that there are many ways to do it and thus that the parameter shouldn't be Boolean, to allow for future expansion...

[RiccardoGiro]

Did you manage to implement any of those features?

[jbednar]

No, it's wide open for any interested user to dive into, and should be an easy win!

[jbednar]

Looks like I duplicated my own issue 4 years later: https://github.com/holoviz/holoviews/issues/5359

5359 includes links to LTTB sample code; please see that for details.

[jvdd]

Hey @jbednar

I wanted to let you know about a new library that I have developed called tsdownsample. It is an optimized implementation of various time series downsampling algorithms written in Rust, and it is available as a stand-alone Python package. The goal of this library was to improve the efficiency of these algorithms, while also allowing them to be used independently of plotly-resampler.

The library is still in the early stages of development, but it is beginning to mature. I am reaching out to see if there is interest in using tsdownsample to provide time series decimation options. If so, I would be happy to submit a pull request. Additionally, I welcome any feedback you may have on the interface for the downsample methods. More details can be found below :arrow_down:

The signature of the downsample methdods:

downsample([x], y, n_out, **kwargs)

Note the following:

• x is optional
• x and y are both positional arguments (this is in line with how the matplotlib.pyplot.plot signature is)
• n_out is a mandatory keyword argument (defining the number of output values)

[jbednar]

Thanks. We'll be looking at downsampling options in the coming months, and we'll be sure to check this out!

[TheoMathurin]

Am I mistaken in thinking that what's been added in https://github.com/holoviz/holoviews/pull/5552 by @philippjfr and @Hoxbro could be used in decimate to address this issue?

[hoxbro]

The first two options mentioned in the first comment were implemented in #5552

[TheoMathurin]

Alright so the idea is to use downsample1d directly instead of decimate + some options. It feels a bit like there are two parallel ways to perform downsampling but that's fine with me.

Thanks @Hoxbro!