Chart 3 & Data Decimation

Nico-DF commented 3 years ago

Feature Proposal

Starting from v3, Chart now proposes integrated data decimation, things that had to be done manually before.

This allows to downsample any data using Time/Linear xAxis to increase performance. But currently, if using the popular zoom plugin, the resolution is significantly lost upon zoom. One way to increase the resolution is to constantly perform the downscale not just on whole data, but on data filtered according to the current min and max values of the axis. Moreover, as it is now, the algorithm always re-decimate on the same data with the beforeElementsUpdate hook

I'm quite in a spot here, as it is really between an external plugin (zoom) and a functionality that has been added that I manually handled before, so I understand if that request is not deemed receivable. It would be quite a shame to still have an external downsampling algorithm if that functionality is now natively proposed, and I don't quite see how I can currently hack the function from the outside

Feature Use Case

The use case would be quite simple: Imagine a graph of 10k points on a 0-100 axis evenly spread, downsampled to 1k points using lttb algorithm. Performances are great and the general graph form is kept. Now, if you zoom between 0-50, you will have 500 downsampled points but you theoretically still have enough data to have a 1k downsampled curve, hence having a better resolution and being closer to local values.

Possible Implementation

Something like https://github.com/Nico-DF/Chart.js/commit/fc3a6fb7031ef29e5c056d24031b61b06e48a1b9 (quick & dirty for the moment, only briefly tested with numerical values)

Nico-DF commented 3 years ago

Side note: If data are outside range, they will not be displayed, hence the line graph will not show the trend at extremities

kurkle commented 3 years ago

Does look like a good enhancement. I'd probably export and use that:

https://github.com/chartjs/Chart.js/blob/ac0a86b95d7c53d6743d5f48fa895edca1ccd483/src/controllers/controller.line.js#L133

and pass the result to the decimation functions. That way you'd avoid extra (possibly large) array creation and utilize binary search.

Also probably good idea to get the point count before that line:

https://github.com/chartjs/Chart.js/blob/ac0a86b95d7c53d6743d5f48fa895edca1ccd483/src/plugins/plugin.decimation.js#L199

Nico-DF commented 3 years ago

LGTM, didn't saw the existing function, I only wrote what I basically did in v2 on my side, and agree about the check, it was indeed quick and dirty

Btw, I don't know how widely the chart._data is used, but on my side I called it _originalData, but that's totally another topic

etimberg commented 3 years ago

I am on-board with updating this and getting it into 3.1. I don't have time to look at a PR today, but will try and find time later in the week unless you'd like to try this @Nico-DF

Nico-DF commented 3 years ago

I could always give it a shot and let you judge it

Nico-DF commented 3 years ago

@kurkle @etimberg

I began to work on it (you can see the progress on https://github.com/Nico-DF/Chart.js/commits/master and btw I fixed what seems to be an error on the lttb algorithm)

But there is some problem. When I tried to do some tests (see https://github.com/Nico-DF/Chart.js/commit/fb214a46036659d6719ce110fd40e0a7f6544255), I remarked that:

The getStartAndCountOfVisiblePoints where I called it on the plugin was not properly initialized yet (meta._sorted was still empty)
Moreover, even when it was correctly initialized during a call without the plugin, with an axis going from 1 -> 3 and the Identity function on interval [0, 9] as data, the function responded {0, 4} whereas I expected more something like {1, 3}

Any help on this particular function?

There is no PR yet as my own test fails due to this function behavior

kurkle commented 3 years ago

It includes one point outside the drawing area, so the line looks proper when the points are not on the edge. Does that make sense?

Nico-DF commented 3 years ago

Agree for the extra point, that's not really an issue and make sense for proper display, but my main problem is the fact that when the plugin hook is called, the chart lifecycle seems not complete, hence the returned values does not correspond at all

Don't really know if I'm clear about it

kurkle commented 3 years ago

right, it can't be really used then, the decimation needs to be done before creating points. Sorry for the wrong suggestion :/

Nico-DF commented 3 years ago

Ok thanks, was not sure if it was some misuse on my part. Not a problem, I can at least keep the logic start/count to avoir creating another array of thousands of values. I'll update it soon and propose a PR if all works

kurkle commented 3 years ago

You could probably utilize the _lookupByKey or _lookup helpers to find the indices

Nico-DF commented 3 years ago

tbh, I will probably use the exact same code but in my case, I already know that the parsing option is false, hence I will not perform the check on the _sorted variable.

chartjs / Chart.js