search

engaging-computing / phpSENSE

An educational data analytics platform.
http://isenseproject.org
14 stars 10 forks source link

Investigate Vis Speedup Options #538

Closed mmcguinn closed 11 years ago

mmcguinn commented 11 years ago

There are two basic routes to consider here,

Try the 'Turbo' option in highcharts. This basically means datapoint metadata cannot be included in the chart (x,y only) but has some currently unknown speedup.

Purposely remove datapoints when zoomed out (smoothly) to reduce the number of rendered points to a reasonable level.

or maybe some insidious combination of the two!

mmcguinn commented 11 years ago

So far I've discovered that the 'Turbo' option of formatting results in no noticeable performance increase.

Removing datapoints smoothly is contingent on implementing saved zoom (the same mechanics are required in both places) so it's probably worth using that as a partway goal.

In terms of removing points, a couple of ideas have been floated (discussed in the lab):

  1. Removing n-1 of every n points. This is incompatible with any wort of smart filtering (and potentially destructive), but is very fast.
  2. A high-pass(ish) filter on the second derivative. Essentially this would be removing points that are most co-linear, and so are redundant. This is still pretty fast (linear or nlog n depending on how the floor is determined) and should be compatible with other filters.
  3. Calculate a noise floor and smooth the data. This would be used in conjunction with #2 which would then find more co-linear-like points to remove. The potential issue (or feature) here is this would be actually modifying the data rather than selectively ignoring it.
mmcguinn commented 11 years ago

@ivanrudnicki @fgmart

Data reduction is now on dev!

Simply put, each particular series (line) is restricted to being 600 points in length. There is a checkbox called 'full detail' which turns reduction off.