search

sveinn-steinarsson / flot-downsample

Downsample plugin for Flot charts.
286 stars 39 forks source link

Null/Nan handling #27

Closed artrune closed 2 years ago

artrune commented 2 years ago

Many plotting libraries support having a null with a timestamp to break a trend. (Leaving a gap, preventing interpolation) Imagine you have time series data and its divided in three buckets:

values         dates
1              2021-11...
2              2021-11...
5              2021-11...

2              2021-11...
null           2021-11...
5              2021-11...

6              2021-11...
2              2021-11...
7              2021-11...

I can think of two ways on how to handle this but I'm not sure what would be ideal:

  1. Since the algorithm will always choose a time-value pair from the dataset (Because C is either calculated the next pass as B or it's the last datapoint), as long as there as there are no duplicate data points (two values on the same date/timestamp) it's safe to remove the nulls first, run LTTB and then add them back in.
  2. The other option would be to split the dataset into two and apply lttb to each of them, then at the end add the resulting datasets together and the nulls, this would keep the first and last data points before/after trend breaks (the nulls)

Any thoughts?

sveinn-steinarsson commented 2 years ago

This is a know limitation. I think someone might have adapted the algorithm to support gaps. I suggest you check out other versions https://github.com/sveinn-steinarsson/flot-downsample#the-algorithm-lttb-adapted-for-other-programming-languages-or-frameworks