microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.22k stars 2.87k forks source link

[Performance] [webgl]bad performance of webgl #13986

Open vacing opened 1 year ago

vacing commented 1 year ago

Describe the issue

image I struggle for a long time to modify my model to fit onnx-webgl, but found it 3x slower than wasm or xnnpack. Hoping to get some advice on it.

9.134.253.57-1671091049474.log

To reproduce

        async function runSqueezenetModel(preprocessedData) {
            session.startProfiling()
            var [results, inferenceTime] = await runInference(session, preprocessedData);
            session.endProfiling()
            return [results, inferenceTime];
        }

        async function runInference(session, preprocessedData) {
            const start = new Date();
            const feeds = {
                src: preprocessedData[0],
                r1i: preprocessedData[1],
                r2i: preprocessedData[2],
                r3i: preprocessedData[3],
                r4i: preprocessedData[4],
            }

            const outputData = await session.run(feeds);
            const end = new Date();
            const inferenceTime = (end.getTime() - start.getTime());
            const fgr = preprocessedData[0]
            const pha = outputData["res"];
            const r1o = outputData["r1o"];
            const r2o = outputData["r2o"];
            const r3o = outputData["r3o"];
            const r4o = outputData["r4o"];

            return [[fgr, pha, r1o, r2o, r3o, r4o], inferenceTime];
        }

        // 默认初始化为 0
        const r1i_arr = new Float32Array(r1_size[0]*r1_size[1]*r1_size[2]*r1_size[3]);
        r1i = new ort.Tensor(type=precision, r1i_arr, dims=r1_size);
        const r2i_arr = new Float32Array(r2_size[0]*r2_size[1]*r2_size[2]*r2_size[3]);
        r2i = new ort.Tensor(type=precision, r2i_arr, dims=r2_size);
        const r3i_arr = new Float32Array(r3_size[0]*r3_size[1]*r3_size[2]*r3_size[3]);
        r3i = new ort.Tensor(type=precision, r3i_arr, dims=r3_size);
        const r4i_arr = new Float32Array(r4_size[0]*r4_size[1]*r4_size[2]*r4_size[3]);
        r4i = new ort.Tensor(type=precision, r4i_arr, dims=r4_size);

Urgency

very agent to get advice, is it a normal result?

Platform

Web Browser

OS Version

cenos-7.9

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

c9edc01c0b02f2d31020ac4bc06631f755403d3b

ONNX Runtime API

JavaScript

Architecture

X64

Execution Provider

Other / Unknown

Execution Provider Library Version

No response

Model File

No response

Is this a quantized model?

No

liuyingbin123 commented 1 year ago

have you fixed this?i meet the same problem too