better chart performance

[bluemonk3y]

might provide high performance alternative to apex charts.

https://www.zingchart.com/docs/integrations/vue

Good comparison here https://leeoniya.github.io/uPlot/demos/multi-bars.html

[leeoniya]

probably not [1] ;)

[1] https://github.com/leeoniya/uPlot#performance

[logscape]

Thank you!

On Thu, 7 May 2020, 18:28 Leon Sorokin, notifications@github.com wrote:

probably not [1] ;)

[1] https://github.com/leeoniya/uPlot#performance

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/liquidlabsio/fluidity/issues/65#issuecomment-625392952, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWVFJIDOHGNC64KSU3RD2TRQLVVDANCNFSM4M3PGKHA .

[logscape]

@leeoniya - can you recommend an efficient scatter-plot chart?

[logscape]

Scatter plot: https://github.com/David-Desmaisons/vue-plotly

[leeoniya]

that specific example has x-aligned data. if this is what you want (or can massage your data to be x-aligned), then uPlot does it out of the box. you may just need to null-pad some series as i do in [1] or as seen in [2]. it's unusual to have a true scatter plot with connected points:

https://github.com/leeoniya/uPlot/issues/107#issuecomment-602158244

[1] https://leeoniya.github.io/uPlot/demos/time-periods.html [2] https://github.com/leeoniya/uPlot/issues/94

[logscape]

Thanks for the examples! I want to build a scatter plot that acts like a timeseries heatmap showing groups of individual transactions. The y axis represents the duration, the colour is a heatmap of how many transactions occur in that x/y bucket range...This approach shows the distribution of transactions by latency and density using color.

[leeoniya]

something like below?

you can certainly do this in uPlot with a splash of extra code. you'd need to implement a simple custom points renderer. like the draw-hooks, multi-bars and candlestick-ohlc demos do. e.g.

https://github.com/leeoniya/uPlot/blob/8f5eea003ac101b08a9a33cc11d6c6b63ee4cc9c/demos/draw-hooks.html#L119

[leeoniya]

might actually be a good demo to add to uPlot, if you'd like to take a stab at it :)

[logscape]

Perfect!

On Thu, 7 May 2020, 20:42 Leon Sorokin, notifications@github.com wrote:

something like below?

you can certainly do this in uPlot with a splash of extra code. you'd need to implement a simple custom points renderer. like the draw-hooks, multi-bars and candlestick-ohcl demos do. e.g.

https://github.com/leeoniya/uPlot/blob/8f5eea003ac101b08a9a33cc11d6c6b63ee4cc9c/demos/draw-hooks.html#L119

[image: image] https://user-images.githubusercontent.com/43234/81337032-f5555000-906f-11ea-983b-cf0030dd8a2b.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/liquidlabsio/fluidity/issues/65#issuecomment-625457974, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWVFJJ5GG6FTFL4H3E3UV3RQMFJLANCNFSM4M3PGKHA .

[leeoniya]

i guess the main difference is that you can't hover the individual points as you can in a true scatter plot. but you can still render out the values in a custom legend or tooltip for display.

[logscape]

Will it support click ability so I can fetch underlying data?

On Thu, 7 May 2020, 20:47 Leon Sorokin, notifications@github.com wrote:

i guess the main difference is that you can't hover the individual points as you can in a true scatter plot. but you can still render out the values in a custom legend or tooltip for display.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/liquidlabsio/fluidity/issues/65#issuecomment-625460671, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWVFJMPENEQYIMFNYHXGBDRQMF6RANCNFSM4M3PGKHA .

[leeoniya]

well, the points will be static. however, there's an API to get the data index from cursor position, so if you add a click handler you can do whatever you need there, e.g.: https://leeoniya.github.io/uPlot/demos/candlestick-ohlc.html

[bluemonk3y]

@leeoniya - I'm prototyping from your demos - just started but it feels promising while I get my head around uPlot. 2 questions....

1. My RHS Axis is being chopped off - losing the trailing 000s. Im plotting values up to 2m. Alternativeliy - I could use a Y value numbering that drops the 0s in favour of 1m 1.5m 2m etc?

2. How do I handle click events? I need to register a click handler and then figure out what the underlying data point is. Is there a demo for how I can do this?

[leeoniya]

hey @bluemonk3y

1. there's no axis auto-sizing, so you can either change the formatting or set a larger axis.size. i personally prefer the shorter labels.
2. if you want to click on points, take a look at https://github.com/leeoniya/uPlot/issues/239, if you want to handle arbitrary clicks on the plot, just bind a click handler to u.root.querySelector(".over") and then read off the properties of u.cursor. idx is the index into the data array(s) of the closest hovered x offset. otherwise you can feed left and top to u.posToVal() to get the values along a given scale, e.g. https://leeoniya.github.io/uPlot/demos/tooltips.html

[bluemonk3y]

Thanks @leeoniya - sorted. I have a few more visualizations to prototype now.

[bluemonk3y]

Hi @leeoniya - I'm now trying to build the heatmap data model (to produce similar to your image above). Would you pls advise on how you generated the heatmap image above?

I can see a few options to create uniform arrays for rendering and hoping to get your thoughts on which is best ;)

1). a collection of numeric arrays where the first part of the number represents the Y value and the decimal is used to render the heatmap colour.

 timestamp:   [ t, t+1, t+1, t+3 ... ]
   latency-1:   [ <y-value.heat> 100.100, 200.12, 300.35, 400.56...]   
   latency-2:   [ 100.22, 250.22, 300.22, 100.33...]
   latency-3:   [ 100.33, 250.44, 300.55, 100.66...]

in code it end up as:
const data = [
    [1546300800,1546387200,1546473600,etc], 
    [100.22, 250.22, 300.22, 100.33 etc,],  
    [100.33, 250.44, 300.55, 100.66 etc],
    ];

I would then need to create a uniform series, and zero fill the empty spaces. This approach is a bit ugly and bloats memory while being kinda complicated.

It would be nice (easier!) to support multi dimension arrays. This would allow a variable number of 'y' values - however Im guessing it also breaks the underlying engine.

 timestamp:   [ t, t+1, t+1, t+3 ... ]
   latency-1:   [ [<y-value.heat>, 100.100, 200.12], [500.3, 300.35], [400.56, etc]...]   

2) Another approach might be to cheat by using a map/index. The plugin then performs a lookup on the Y value decimal to get the index for a lookup. It grabs the lookup value and iterates the the Y-List array - rendering cells as it goes.

 timestamp:   [ t, t+1, t+1, t+3 ... ]
   latency-1:   [ <y-value.y-index>, 100.1, 342.2, 123.3, 133.4..]   

Y-List [  1: [123.10, 100.12, 456.12], 2:[100.10], 3:[256.5,1000.5]]

This approach lets me use multiple values for the rendering. BUT - then to support mouse click reverse lookup I can get the 'X' index - and then reverse the 'Y' offset to a value and search for it I guess?

Any thoughts? Im starting to like the index-lookup approach ;)

[leeoniya]

Would you pls advise on how you generated the heatmap image above?

i did a google image search :D

the nested approach should work fine as long as you handle the rendering and don't hand it off to uPlot to stumble over. you'll probably want to have two non-rendered high/low series w/ series.paths = () => null; to establish y scale range at each x, and then another array to pull the y values from.

having the decimal portion be the color is a bit weird, but certainly space-efficient for packing the data. i'd probably keep the colors in a different array until you have perf issues, but that's up to you.

something like this:

let data = [
  [t, t+1, t+1, t+3],
  [low1, low2, low3],  // y lows (for scale ranging)
  [high1, high2, high3],  // y highs (for scale ranging)
  [[100, 200], [300, 400], [400, 500]],  // y values (offsets)
  [[1, 2], [56, 25], [99, 13]],  // y weights (colors)
];

let opts = {
  series: [
    {},
    {
      paths: () => null,
      points: {show: false},
    },
    {
      paths: () => null,
      points: {show: false},
    },
  ],
  hooks: {
    draw: [
       u => {
        const ctx = { u };
        // use u.data[0], u.data[3], u.data[4] to draw colored rects
      }
    ]
  }
};

[bluemonk3y]

Nice, yes that makes sense - thank you!

On Thu, 4 Jun 2020, 22:05 Leon Sorokin, notifications@github.com wrote:

Would you pls advise on how you generated the heatmap image above?

i did a google image search :D

the nested approach should work fine as long as you handle the rendering and don't hand it off to uPlot to stumble over. you'll probably want to have two non-rendered high/low series w/ series.paths = () => null; to establish y scale range at each x, and then another array to pull the y values from.

having the decimal portion be the color is a bit weird, but certainly space-efficient for packing the data. i'd probably keep the colors in a different array until you have perf issues, but that's up to you.

something like this:

let data = [ [t, t+1, t+1, t+3], [low1, low2, low3], // y lows (for scale ranging) [high1, high2, high3], // y highs (for scale ranging) [[100, 200], [300, 400], [400, 500]], // y values (offsets) [[1, 2], [56, 25], [99, 13]], // y weights (colors)]; let opts = { series: [ {}, { paths: () => null, points: {show: false}, }, { paths: () => null, points: {show: false}, }, ], hooks: { draw: [ u => { const ctx = { u }; // use u.data[0], u.data[3], u.data[4] to draw colored rects } ] }};

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/liquidlabsio/fluidity/issues/65#issuecomment-639115799, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQLJJQUM2OYP5TC4F4JQCDRVAEBBANCNFSM4M3PGKHA .

[leeoniya]

you can probably avoid the scale-ranging series too, and find the min/max yourself:

let data = [
  [t, t+1, t+1, t+3],
  [[100, 200], [300, 400], [400, 500]],  // y values (offsets)
  [[1, 2], [56, 25], [99, 13]],  // y weights (colors)
];

let opts = {
  series: [
    {},
  ],
  scales: {
    y: {
      auto: false,
      range: u => {
          let [i0, i1] = u.series[0].idxs;
          // ...walk u.data[1] from i0 to i1 & accumulate min/max for current x range
          return [min, max];
      }
    }
  },
  hooks: {
    draw: [
       u => {
        const ctx = { u };
        // use u.data[0], u.data[1], u.data[2] to draw colored rects
      }
    ]
  }
};

[bluemonk3y]

Hi @leeoniya - I made a first pass and have a basic heatmap ladder working (nothing fancy - yet!) - but got stuck on the scale.min/max not being calculated even though I provided a range. I had to manually set min/max however, it also breaks zoom functionality (as expected)...... any thoughts - is it because of the 2d array?

 scales: {
        y: {
          auto: false,
          min: 0,
          max: 100,
          range: [0, 60 * 1000],
      }

[bluemonk3y]

[leeoniya]

scale.min/max are initial values (yeah i know, it's not terribly intuitive until you realize that they're the active/realtime min/max when reading back from u.scales.y.min/max). if you're providing scale.range, you don't need to set scale.min/max - they will get set with the output of scale.range.

as my example shows, your scale.range needs to be a function which calculates the min/max of the zoomed x range (which is the data between i0 and i1 indices, inclusively). this will allow zooming to work normally because the y scales will re-range to the visible data.

something like this: https://jsfiddle.net/4vkqraed/

uPlot's ranging does more though. it gives 0 special affinity and adds y-padding based on the detected min/max values, etc. uPlot does expose its ranging function that accounts for this which you can use to replicate & tweak the original behavior:

https://github.com/leeoniya/uPlot/blob/5b985552618ab4539df6ea5b50f93d1710a66016/dist/uPlot.d.ts#L105-L106

honestly though, you're probably better off just generating the faux/unrendered min/max series and allowing uPlot to do its thing.

[bluemonk3y]

Ok that makes sense - I completely missed the point of it being a function. ;-) Thanks - Ill give it a spin.

[leeoniya]

Ok that makes sense - I completely missed the point of it being a function

if you needed a static range, then you can have scale.range: [min, max] be an array, which internally just becomes () => [min, max] anyways.

actually, you may still want to implement scale.range to always include zero, as i do in the bars plugin:

https://github.com/leeoniya/uPlot/blob/3c51eb084205006f9f9b59410b1cfa5617e66177/demos/multi-bars.html#L92-L95

[bluemonk3y]

I feel like Im missing something. Adding the min/max data doesn't resolve the scale.infinity problem. Here is my commit - see line:177

[leeoniya]

you're missing the series defs. uplot walks from the series first, then from the data.

  series: [
    {},
    {
      paths: () => null,
      points: {show: false},
    },
    {
      paths: () => null,
      points: {show: false},
    },
  ],

[bluemonk3y]

Thanks - my bad - trying to prune previous code and presumed the series would have a default behaviour. This makes sense though and I feel like Im now getting the API (and liking it!).

[bluemonk3y]

Hey @leeoniya - what are your thoughts on building a horizontal, stacked, bar chart? A bit like a gant chart? or am I stretching it? I want to show a series of distributed traces layered on top of each other. A single bar will use a colour breakdown to show its individual components. Clicking on a single bar will open another chart which is then exploded where each component is in its own lane. Each component is offset by the start-time and length is duration. Each component is clickable to retrieve more data for another drilldown.

[leeoniya]

A bit like a gant chart? or am I stretching it?

aside from slight visual similarity, gantt and hz bar charts are very different.

gantt represents a timeline, where the length of the bar is along an [axial] temporal axis, which is never the case for bar charts - which instead may have time along the cross-axis. i have some preliminary thoughts on gantt & timeline charts in https://github.com/leeoniya/uPlot/issues/188.

what are your thoughts on building a horizontal, stacked, bar chart?

my opinion of stacked charts in general is that they're terrible at everything except saving space:

https://github.com/leeoniya/uPlot#non-features https://everydayanalytics.ca/2014/08/stacked-area-graphs-are-not-your-friend.html https://leeoniya.github.io/uPlot/demos/stacked-series.html another recent rant of mine: https://old.reddit.com/r/javascript/comments/gnffx5/if_cops_can_watch_us_we_should_watch_them_i/

they have the same misleading & compounded problem of charts that should, but don't start at 0.

i'm open to being convinced otherwise.

A single bar will use a colour breakdown to show its individual components. Clicking on a single bar will open another chart which is then exploded where each component is in its own lane. Each component is offset by the start-time and length is duration. Each component is clickable to retrieve more data for another drilldown.

this does actually sound like a timeline plot, and the "stack" is not a sum but kind of like a group of sub-tasks also along the same time axis? i that case i can see some value - it's one of those few cases where the stack is not a sum-trend.

i think a plugin for this would be most useful in the DOM layer rather than on canvas. this way you can get hover-ability for free by using plain css/js instead of having to hack that it by coordinate probing. perf should not be an issue since the amount of data is probably < 500 total elements. zooming, however, could turn out to be tricky if you dont want to completely destroy and recreate the dom each time.

[bluemonk3y]

Thanks, I agree - generally stacked bars suck - however in this case, due to the time-basis and the component breakdown they make sense.

[leeoniya]

btw, any progress on the time-series heatmap? would be cool to get that added as a demo if you got it working and willing to share :)

[bluemonk3y]

Sure - happy to share although its still a little rough. It supports click events on individual ladder entries, hovering on a ladder entry displays the 'y' value as well as the 'count' - where the count is used to show the number of entries in that cell.

I broke out the data series as you suggested - this helped simplify the code. I added more data entries, but the alignment isnt ideal. It will look better when I capture real data.

[bluemonk3y]

The demo code is the bar-heatmap.js and html in: https://github.com/liquidlabsio/fluidity/tree/dataflow-page/web/src/main/webapp/trial/uplot

[bluemonk3y]

I will create a PR in uPlot tomorrow then see what you think (Im in London)

[leeoniya]

@bluemonk3y i see most of your code is Apache-licensed. would you be okay releasing any PRs to uPlot under MIT? i really don't want to deal with license compat concerns.

[bluemonk3y]

Yeah, no worries.

On Tue, 9 Jun 2020, 18:06 Leon Sorokin, notifications@github.com wrote:

@bluemonk3y https://github.com/bluemonk3y i see most of your code is Apache-licensed. would you be okay releasing any PRs to uPlot under MIT? i really don't want to deal with license compat concerns.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/liquidlabsio/fluidity/issues/65#issuecomment-641451563, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQLJJWRMA2V6JAQMUKECR3RVZTYNANCNFSM4M3PGKHA .