[Lens][Visualize] Optimize rendering pipeling for high number of breakdown categories

[drewdaemon]

The fillColor routine in the partition vis renderer is called for every partition. In high-volume partition charts (e.g. top 500 terms), this becomes quite slow, relative to other rendering routines.

This slowdown can also show up in partition chart suggestions.

There's probably room to optimize fillColor to keep things snappy.

The same issue could be also reproduced in cartesian charts. To reproduce it in Lens:

• create a new runtimeKeyword field with the following value: emit(Math.random() + '');
• now create a simple Date Histogram x Count of Records x Top values of 700 of runtimeKeyword with a stacked bar chart

The minimal amount of buckets I found to kick the slowness is 200 (buckets) x 200 (breakdown).

[elasticmachine]

Pinging @elastic/kibana-visualizations @elastic/kibana-visualizations-external (Team:Visualizations)

[dej611]

To reproduce this issue in Lens:

• create a new runtimeKeyword field with the following value: emit(Math.random() + '');
• now create a simple Date Histogram x Count of Records x Top values of 700 of runtimeKeyword with a stacked bar chart

The minimal amount of buckets I found to kick the slowness is 200 (buckets) x 200 (breakdown).

With partition chart this is very easy to produce as bucketed operations can be nested up to 3 levels, so even smaller numbers can apply where 3 of them are used. The bigger problem here is that when the user hits one of these case with one chart type, the suggestion bar underneath will produce multiple chart types with similar effects producing big performance issues for the whole editor.

cc @markov00 @nickofthyme

[markov00]

I see how this can lead to a performance issue, but in any case I feel that rendering a piechart or a barchart with such high cardinality of breakdown categories is clearly a bad practice and is a solution that should not be reached by the user. A numerous things can be done to speed up the implementation, but I don't feel we should spend our efforts to improve this bad case. My proposal is instead to spend time in:

• reduce the number of possible "breakdown" value the user can have on a chart to an amount that can make sense (always remember that you probably are not able to clearly distinguish more than 7/10 different categories in a chart with colors
• provide a supportive description for this limitation and guide the user to a better practice (group 10 + others for example)
• improve our suggestions and never propose bad practices, limit the number of series to valid configurations, detatch the suggestion values from the user values, just create suggestions with the fields not the quantities

[markov00]

A chart in the suggestions is rendered in an area of 132x82 pixels.

A possible set of preferable configurations for the suggestions:

• line/area: limit the number of series to a max of 3 to 5. limit the number of time or numerical buckets to 10/20 (we don't need precision here, just quick rendering and possible suggestions). We can also reduce the number of visual elements like the grid, reduce the line width and any point size)
• bars: if categorical reduce it to a max of 7 to 10 values x max of 3 to 5 series (less is more). Follow a similar strategy of line/area if the bars are used as histograms
• pie: no need to suggest a donut IMHO. max of slice of 7 to 10, always remember to keep the other bucket because is fundamental to the partition concept. Don't suggest wrong aggregations per "size by" like average/mean, use only count/sum. Don't suggest multi-layer.
• treemap: we can increase the number of sectors, but I believe a max of 7 to 10 is already enough. Keep other.
• mosaic: 3x3 is fine
• waffle: also here max of 7 to 10 series
• gauge/bullet: at that size it will be represented as metric, the only care here is to format "compact"ly the number