I found that `preferredOutputLocation` only used in wasm/wasm-core-impl.ts. But webgl only use backend-onnxjs.ts

[langhuihui]

          I found that `preferredOutputLocation` only used in wasm/wasm-core-impl.ts. But webgl only use backend-onnxjs.ts

export * from 'onnxruntime-common';
import * as ort from 'onnxruntime-common';
export default ort;

import {registerBackend, env} from 'onnxruntime-common';
import {version} from './version';

if (!BUILD_DEFS.DISABLE_WEBGL) {
  const onnxjsBackend = require('./backend-onnxjs').onnxjsBackend;
  registerBackend('webgl', onnxjsBackend, -10);
}

if (!BUILD_DEFS.DISABLE_WASM) {
  const wasmBackend = BUILD_DEFS.DISABLE_TRAINING ? require('./backend-wasm-inference').wasmBackend :
                                                    require('./backend-wasm-training').wasmBackend;
  if (!BUILD_DEFS.DISABLE_WEBGPU) {
    registerBackend('webgpu', wasmBackend, 5);
    registerBackend('webnn', wasmBackend, 5);
  }
  registerBackend('cpu', wasmBackend, 10);
  registerBackend('wasm', wasmBackend, 10);
}

Object.defineProperty(env.versions, 'web', {value: version, enumerable: true});

Originally posted by @langhuihui in https://github.com/microsoft/onnxruntime/issues/16452#issuecomment-1970826430

[fs-eire]

Yes, you are right. Currently preferredOutputLocation only support cpu(default) or gpu-buffer. webgl texture is not yet supported.

[langhuihui]

@fs-eire However, in #16452 you give the example

// when create session, specify we prefer 'image_output:0' to be stored on GPU as texture
const session = await InferenceSession.create('./my_model.onnx', {
  executionProviders: [ 'webgl' ],
  preferredOutputLocation: { 'image_output:0': 'texture' }
});

[fs-eire]

Change #16452 is the API change. In this change, it includes all required code changes to support IO binding for both webgl texture and webgpu buffer from API level.

To actually support IO-binding from execution, it requires separated changes. For WebGPU, #17480 is the PR that actually implement the backend support for IO-binding. We haven't started the work for WebGL texture IO binding, because (1) we don't yet see real usage for that, and (2) we are mainly focusing on WebGPU instead of WebGL now.

[langhuihui]

WebGL is more useful because most user's browser support WebGL not WebGPU

[fs-eire]

WebGL is more useful because most user's browser support WebGL not WebGPU

It takes time for browsers like Safari to support WebGPU. This is why we still keep the WebGL backend, which works well with the old CNN models.

However, to support latest models (eg. transformer based ones), there are a few technical difficulties for WebGL to work with them. The unpacked kernel implementation uses actually 4x than necessary GPU memories, making it hard to accommodate the weights for LLMs; the lack of compute shaders, the inefficiency of the OpenGL APIs (comparing to modern Graphcis APIs) makes it unlikely to achieve a good enough performance; and having to use the rendering pipeline also makes it more difficult to implement kernels.

Because of the above considerations, we will put limited efforts on WebGL. We welcome community contribution if anyone is interested in it.

[github-actions[bot]]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[langhuihui]

我想完善一下 webgl 这块的逻辑，目前测试还没跑通，输入纹理方式，在getOrCreateTextureData函数里面会调用创建纹理，实际上不需要创建，只需要用传入的纹理，结果报错了，因为创建纹理时没有数据（使用fromTexture创建的 Tensor）

[DennisSmolek]

WebGL is more useful because most user's browser support WebGL not WebGPU

It takes time for browsers like Safari to support WebGPU. This is why we still keep the WebGL backend, which works well with the old CNN models.

However, to support latest models (eg. transformer based ones), there are a few technical difficulties for WebGL to work with them. The unpacked kernel implementation uses actually 4x than necessary GPU memories, making it hard to accommodate the weights for LLMs; the lack of compute shaders, the inefficiency of the OpenGL APIs (comparing to modern Graphcis APIs) makes it unlikely to achieve a good enough performance; and having to use the rendering pipeline also makes it more difficult to implement kernels.

Because of the above considerations, we will put limited efforts on WebGL. We welcome community contribution if anyone is interested in it.

Just as an example usage of WebGL is that of using models for image processing. I have created Denoiser: https://github.com/DennisSmolek/Denoiser

which is a port of the OIDN denoiser using tensorflow.js

Tensorflow has worked well for me but community feedback is that tensorflow is declining in popularity/support so I’m looking at other platforms.

In general I agree that webGPU is the future but as of right now it only really works in chrome and only on ANGLE backends.

Without a way to pass the WebGLTexture to a system like ThreeJS, Babylon, etc using ONNX is blocked.

the PRs I’ve read to get to this issue implied that accessing the data was already possible. If that’s not the case what would be needed to make that happen?

I’ll stick with tfjs for now but if some more webGPU support opens up I’ll probably be back!

[DennisSmolek]

Wait. I’m confused, this api shows the texture as a property of tensor.

Can I just access it that way?