focus series closest to cursor

[leeoniya]

or, more accurately, de-focus all but the closest.

the cursor.focus branch is a mostly-complete implementation of this dygraphs' highlighted-series demo [1].

i'm not entirely convinced that this is something valuable enough to keep (at least in this format). i'd like to see a real use-case when seeing an isolated trend is indispensable in the presence of dozens or hundreds of series. most of the interesting info in these will just be visual outlier detection, which does not actually require a canvas redraw, but simply quick identification of the hovered series/data point and its value.

i want to stay away from "cool" features that add code and impact perf but don't add much value in terms of data inference.

[1] http://dygraphs.com/gallery/#g/highlighted-series

[dgl]

This is an important feature IMO in presenting monitoring data, a common example is showing many timeseries where they refer to identical instances of a service in a distributed system, but one has some outliner behaviour.

The actual killer feature is presenting several graphs and highlighting a series on another graph that refers to the same instance (because it's on a different graph which the user hasn't interacted with more clarity is needed, I agree on a single graph there's no reason to redraw, but good for the feature to be consistent from a UI perspective). A callback for "series focused" and a way to focus a series via an API would be awesome.

Maybe if this feature didn't want to be in the core a callback could be enough to actually implement it? Knowing which series to highlight would mean styling could be applied to the series by the user code -- right now I think it's a bit expensive for a client to change the properties of the series rather than data though. (Thinking about it this callback could maybe also be enough along with an ability to turn off the built-in labels for the user to recreate labels in a different form if they wanted to.)

[leeoniya]

another useful variant of multi-machine monitoring (in terms of visual anomaly detection) is to offset the y values of each machine in a single graph. e.g. https://github.com/highcharts/highcharts/issues/5948

redrawing the whole thing in a scenario with that many series & datapoints on cursor proximity is simply not practical.

it might be worth considering drawing each series on its own canvas, then it could become manageable perf-wise.

The actual killer feature is presenting several graphs and highlighting a series on another graph that refers to the same instance

yeah that sounds useful. i think in either case there probably needs to be an ability to split the series across canvases so what's redrawn can be optimized. e.g. all non-focused series' canvases can be set to visibility:hidden or opacity: 0.5 very cheaply with 0 canvas redraw.

[leeoniya]

another thing to test out is building the lines as Path2D objects and retaining them so if we have to redraw a single canvas for focus/defocus purposes, we don't have to rebuild them, which is the actual expensive part.

in both cases i think that mem efficiency will take a hit, but probably less in the Path2D variant.

this route may also be architecturally easier since the band-drawing impl is actually a shape constructed out of 2 adjacent series and not having to book-keep the series:canvas mapping would reduce the code complexity a bit. although i haven't given much thought to how bands should behave in a focus/defocus scenario (they're pretty new).

https://developer.mozilla.org/en-US/docs/Web/API/Path2D

[leeoniya]

i ran a couple stress tests that draws 100 series x 130,000 points to either a single canvas or to multiple canvases. the results are interesting.

in Chrome, the absolutely-poistioned multi-canvas variant actually performs much better - with the GPU spending 80% less time. what's even more surprising is that the peak memory is not too much different and the retained mem after idle is actually lower. i'm assuming it has something to do with the what path the compositor takes in each case. maybe in the multi-canvas case it takes better advantage of the system's native compositor.

in firefox, the results are less surprising. the retained mem for multi-canvas is about 2x (still tiny at 1M vs 0.5M). the rendering time is i think about 200ms slower (it's somewhat harder to measure in FF for me) with the GPU being more stressed by multiple canvases.

still, these results are encouraging for an unrealistically high stress test and i'll probably try to flesh out a multi-canvas impl sometime soon.

the stress code is attached if you want to try it yourself. the results may be highly gpu/cpu/os dependent. i tested on Win10.

multican.zip

[leeoniya]

canvas-per-series turned out to be a dead end.

after doing the actual implementation i saw some ~5% improvements in the 166K bench. however, when throwing a 600 series x 8000 points dataset at it, it crashed the GPU in both Chrome and Firefox instead of rendering it in ~1200ms as it does in single canvas variant. when i turned the series count down to 150, it rendered but the cursor interaction was painfully poor.

EDIT: i think this happens because the GPU runs out of memory. the 600 x 8000 test draws on a 1920x5000 canvas. 600 of these more than likely exceeds what can fit into the GPU VRAM. when i switch to 1920x500 canvas, it does render the full dataset, but the cursor perf remains unacceptably bad.

so it looks like i'll need to see how much memory overhead (and perf wins) there are when creating and retaining Path2D objects.

[leeoniya]

the Path2D conversion worked out well, with huge toggle perf improvement. still some work left to do in #40, but cursor.focus can now be implemented much more elegantly without worrying about a full paths rebuild.

[leeoniya]

most of this is done: https://leeoniya.github.io/uPlot/demos/focus-cursor.html

still todo is to focus matching series in other charts if cursor.sync is set: https://github.com/leeoniya/uPlot/commit/10363f4ae3e7e5a33107acae4238ec929ac32d6f#diff-fbc43a77b4e8fe64398aa79145272a6bR920

[leeoniya]

it turns out this is a pretty deep rabbit hole because there are so many possibilities. even what's already in place is pretty opinionated and i'm concerned with it becoming even more so.

here are some issues/concerns:

let's say you have 100 machines to monitor. plot 1 shows 100 x CPU, plot 2 shows 100 x RAM, plot 3 shows 100 x TCP. many of your metrics on a single plot will overlap most of the time unless you provide artificial vertical separation by offsetting each series in your data. assuming you've done that, focusing the closest series on hover across multiple charts is of limited use unless you can lock the focus, then scroll to another chart which may be off-screen. how does focus locking/unlocking work? on click? what about clicking on legend? this toggles. but maybe it should isolate? maybe you don't want to synchronize the toggle, but sync the focus?

and that's just the tip of the iceberg.

i feel like i should just provide the API primitives and leave everything else to be external because offering even 3 options and then handling exactly how they interact with each other and when will be an endless nightmare.

i'm open to suggestions on what the imperative API primitives need to look like to support a flexible amount of use-cases but not burdening the user with unnecessary complexity. it's not such an easy task...

[leeoniya]

this is done now. i made the sync demo more compelling for the stated use case:

https://leeoniya.github.io/uPlot/demos/sync-cursor.html