Use OffScreenCanvas API for big data live charts

[ahmadalibaloch]

One-line summary [问题简述]

I am creating an application which shows years of historic data and also keeps plotting new live data from a web socket connection, I am facing real performance issues and UI thread goes blocking, I have been searching on it and I think the solution is this. Make use of OffScreenCanvas API to render live charts with big data smoothly. It is really easy to use with just one line amendment. canvasElement.transferControlToOffscreen()

Expected behaviour [期望结果]

Live charts with big data should work smoothly.

[deqingli]

Thank you for your feedback and suggestions.

[100pah]

If possible, can we have your example/demo for detailed discussion?

[ahmadalibaloch]

I customized this large bar chart from echarts gallery and made it dynamic so you can have a look, copy and past following code into an echart example. While this chart is rendering in real time try to scroll the code area scrollbar and you will feel UI thread blocking at every interval.

var dataCount = 5e5;
var data = generateData(dataCount);

var option = {
    title: {
        text: echarts.format.addCommas(dataCount) + ' Data',
        left: 10
    },
    toolbox: {
        feature: {
            dataZoom: {
                yAxisIndex: false
            },
            saveAsImage: {
                pixelRatio: 2
            }
        }
    },
    tooltip: {
        trigger: 'axis',
        axisPointer: {
            type: 'shadow'
        }
    },
    grid: {
        bottom: 90
    },
    dataZoom: [{
        type: 'inside'
    }, {
        type: 'slider'
    }],
    xAxis: {
        data: data.categoryData,
        silent: false,
        splitLine: {
            show: false
        },
        splitArea: {
            show: false
        }
    },
    yAxis: {
        splitArea: {
            show: false
        }
    },
    series: [{
        type: 'bar',
        data: data.valueData,
        // Set `large` for large data amount
        large: true
    }]
};
var baseValue = Math.random() * 1000;
function next1(idx) {
        smallBaseValue = idx % 30 === 0
            ? Math.random() * 700
            : (smallBaseValue + Math.random() * 500 - 250);
        baseValue += Math.random() * 20 - 10;
        return Math.max(
            0,
            Math.round(baseValue + smallBaseValue) + 3000
        );
    }
setInterval(()=>{
    option.series[0].data.push(next1(1));
        myChart.setOption(option);
},1000);

function generateData(count) {
    var baseValue = Math.random() * 1000;
    var time = +new Date(2011, 0, 1);
    var smallBaseValue;

    function next(idx) {
        smallBaseValue = idx % 30 === 0
            ? Math.random() * 700
            : (smallBaseValue + Math.random() * 500 - 250);
        baseValue += Math.random() * 20 - 10;
        return Math.max(
            0,
            Math.round(baseValue + smallBaseValue) + 3000
        );
    }

    var categoryData = [];
    var valueData = [];

    for (var i = 0; i < count; i++) {
        categoryData.push(echarts.format.formatTime('yyyy-MM-dd\nhh:mm:ss', time));
        valueData.push(next(i).toFixed(2));
        time += 1000;
    }

    return {
        categoryData: categoryData,
        valueData: valueData
    };
}

[skortchmark9]

I was able to get drawing working in a web worker using this feature:

// main.js
const offscreen = elt.transferControlToOffscreen();
const worker = new Worker('worker.js');
worker.postMessage({ canvas: offscreen }, [offscreen]);

// worker.js
onmessage = function(evt) {
    const canvas = evt.data.canvas;

    echarts.setCanvasCreator(function () {
        return new OffscreenCanvas(32, 32);
    });

    const plot = echarts.init(canvas);
        plot.setOption(...);
}

Certain features didn't work, but it's a good start.

[CarterLi]

Got some progress.

Like what @skortchmark9 said, Echarts itself has very good worker-env support, but no interaction.

Three major problems:

1. Echarts won't bind any mouse events in worker-env. I didn't find any solution other then modifying the source code.

2. Echarts assumes the global object - window's existence at some place. We can work around this by adding window property on self.

3. Of course browser won't trigger any mouse events in worker-env. We have to dispatch those events from UI thread.

Working example: https://github.com/CarterLi/echarts-with-offscreencanvas

[timothymarois]

Has this been implemented in the source yet? I would also like the performance to be improved on large data sets, was there any progress on this aside from @CarterLi forked version? Forgive me if I misunderstood this thread, but it seems very useful as I've been testing large data sets with dozens of series lines, it starts to drop FPS pretty quick.

[tihuan]

Bumping this thread as our project is running into the same performance bottleneck now 🙏

Anyone found a good get around solution in the meantime that supports interactions?

Thank you!

[uykusuz]

@tihuan I don't know when it was introduced, but you can do it now with something like this echarts.init(undefined, undefined, { renderer: "svg", ssr: true, width: <width>, height: <height> }) and then invoke renderToSVGString. This is how I got it working inside a web worker. See init, the option ssr

Then you get an SVG at least. Of course you won't be able to do the usual interactions with that echarts supports, but at least you have an object with a DOM.

[tihuan]

Ahh that's a great solution for offloading it to a web worker! Thanks so much for sharing 🤓🙌!

[iyashjayesh]

can anyone help me to implement web worker?

Main Code:

dataset = "dataset" + document.getElementById("dataset").value
      const config = {
        xAxis: {
          type: 'category',
          data: Object.keys(window[dataset])
        },
        yAxis: {
          type: 'value'
        },
        series: [
          {
            data: Object.values(window[dataset]),
            type: 'line'
          }
        ]
      };
      const worker1 = new Worker('worker/echarts1.worker.js');
      const offscreenCanvas = document.getElementById("offscreenCanvas1").transferControlToOffscreen();

      worker1.postMessage({ canvas: offscreenCanvas, config }, [offscreenCanvas]);

Web Worker (echarts1.worker.js)

importScripts('https://cdn.jsdelivr.net/npm/echarts@5.3.3/dist/echarts.js');

onmessage = function (event) {
    console.log("Started Worker");
    const { canvas, config } = event.data;
    const chart = echarts.init(canvas);
    chart.setOption(config);

    console.log("Finished Worker");
};

While Executing I'm facing the below error in the console.

[messaoudi-mounir]

@iyashjayesh did you make it work ?

[yassilah]

In case anyone is still looking for a solution, I ended up making a small package that does just that: https://github.com/yassilah/echarts-worker; there's support for mouse and custom events as well as devicePixelRatio.

[sev-b]

Any updates on this becoming available in core echarts?

I am using vue-echarts so can't easily integrate @iyashjayesh package