[Feature request] Deno webgpu support

[jlucaso1]

Describe the feature request

Support to running onnx models in webgpu using deno

Describe scenario use case

Actually not working because this error: error: Uncaught (in promise) Error: no available backend found. ERR: [webgpu] backend not found.

Debug console.log(await navigator.gpu.requestAdapter())

GPUAdapter {
  features: GPUSupportedFeatures [
    "depth-clip-control",
    "timestamp-query",
    "indirect-first-instance",
    "shader-f16",
    "depth32float-stencil8",
    "texture-compression-bc",
    "rg11b10ufloat-renderable",
    "bgra8unorm-storage",
    "float32-filterable",
    "texture-format-16-bit-norm",
    "texture-adapter-specific-format-features",
    "pipeline-statistics-query",
    "timestamp-query-inside-passes",
    "mappable-primary-buffers",
    "texture-binding-array",
    "buffer-binding-array",
    "storage-resource-binding-array",
    "sampled-texture-and-storage-buffer-array-non-uniform-indexing",
    "uniform-buffer-and-storage-texture-array-non-uniform-indexing",
    "partially-bound-binding-array",
    "multi-draw-indirect",
    "multi-draw-indirect-count",
    "push-constants",
    "address-mode-clamp-to-zero",
    "address-mode-clamp-to-border",
    "polygon-mode-line",
    "polygon-mode-point",
    "conservative-rasterization",
    "vertex-writable-storage",
    "clear-texture",
    "spirv-shader-passthrough",
    "multiview",
    "shader-f64",
    "shader-i16",
    "shader-primitive-index",
    "shader-unused-vertex-output"
  ],
  limits: GPUSupportedLimits {
    maxTextureDimension1D: 16384,
    maxTextureDimension2D: 16384,
    maxTextureDimension3D: 2048,
    maxTextureArrayLayers: 2048,
    maxBindGroups: 8,
    maxBindingsPerBindGroup: 1000,
    maxBufferSize: 2147483647,
    maxDynamicUniformBuffersPerPipelineLayout: 16,
    maxDynamicStorageBuffersPerPipelineLayout: 8,
    maxSampledTexturesPerShaderStage: 8388606,
    maxSamplersPerShaderStage: 8388606,
    maxStorageBuffersPerShaderStage: 8388606,
    maxStorageTexturesPerShaderStage: 8388606,
    maxUniformBuffersPerShaderStage: 8388606,
    maxUniformBufferBindingSize: 2147483648,
    maxStorageBufferBindingSize: 2147483648,
    minUniformBufferOffsetAlignment: 32,
    minStorageBufferOffsetAlignment: 32,
    maxVertexBuffers: 16,
    maxVertexAttributes: 32,
    maxVertexBufferArrayStride: 2048,
    maxInterStageShaderComponents: 128,
    maxColorAttachments: 8,
    maxColorAttachmentBytesPerSample: 32,
    maxComputeWorkgroupStorageSize: 65536,
    maxComputeInvocationsPerWorkgroup: 1024,
    maxComputeWorkgroupSizeX: 1024,
    maxComputeWorkgroupSizeY: 1024,
    maxComputeWorkgroupSizeZ: 1024,
    maxComputeWorkgroupsPerDimension: 65535
  },
  info: GPUAdapterInfo {
    vendor: "4098",
    architecture: "",
    device: "29695",
    description: "AMD Radeon RX 6600 (RADV NAVI23)"
  },
  isFallbackAdapter: false
}

[fs-eire]

Thank you for reporting this issue. since navigator.gpu is supported in deno, it should work. could you share me reproduce steps (or a repo) so that I can debug it?

[jlucaso1]

@fs-eire Try the following:

import { InferenceSession } from "npm:onnxruntime-web";

await InferenceSession.create(
  "https://huggingface.co/briaai/RMBG-1.4/resolve/main/onnx/model_quantized.onnx",
  {
    executionProviders: ["webgpu"],
  }
);

I'm using deno 2.0.0-rc.10, but deno stable version get the same error.

[fs-eire]

I am not sure about this “npm:onnxruntime-web” syntax. In a nodejs based web app project I used “onnxruntime-web/webgpu” (“webgpu” is the export name that defined in package.json). If I just import “onnxruntime-web” it doesn’t load webgpu support.

[jlucaso1]

@fs-eire I understand. I got a different error when i use import { InferenceSession } from "npm:onnxruntime-web/webgpu"; on stable versions of Deno:

failed to asynchronously prepare wasm: both async and sync fetching of the wasm failed
Aborted(both async and sync fetching of the wasm failed)
error: Uncaught (in promise) Error: no available backend found. ERR: [webgpu] RuntimeError: Aborted(both async and sync fetching of the wasm failed). Build with -sASSERTIONS for more info.

The release candidate version of Deno throw the same error:

error: Uncaught (in promise) Error: no available backend found. ERR: [webgpu] backend not found.
    at resolveBackendAndExecutionProviders (file:///run/media/jlucaso/secondary/projects/webgpu/node_modules/.deno/onnxruntime-common@1.19.2/node_modules/onnxruntime-common/dist/esm/backend-impl.js:120:15)
    at async Function.create (file:///run/media/jlucaso/secondary/projects/webgpu/node_modules/.deno/onnxruntime-common@1.19.2/node_modules/onnxruntime-common/dist/esm/inference-session-impl.js:180:52)

[fs-eire]

It seems that Emscripten does not have well support for Deno. The generated javascript glue file didn't work well in Deno:

• The feature detection may be working incorrectly.

  var ENVIRONMENT_IS_WEB = typeof window == "object";
  var ENVIRONMENT_IS_WORKER = typeof importScripts == "function";
  var ENVIRONMENT_IS_NODE = typeof process == "object" && typeof process.versions == "object" && typeof process.versions.node == "string";
  var ENVIRONMENT_IS_SHELL = !ENVIRONMENT_IS_WEB && !ENVIRONMENT_IS_NODE && !ENVIRONMENT_IS_WORKER;

  There is no special treatment for Deno so further operation may be unexpected.

• Loading from file URL is explicitly disallowed:

  if (isFileURI(wasmBinaryFile)) {
    err(`warning: Loading from a file URI (${wasmBinaryFile}) is not supported in most browsers. See https://emscripten.org/docs/getting_started/FAQ.html#how-do-i-run-a-local-webserver-for-testing-why-does-my-program-stall-in-downloading-or-preparing`);
  }
  

• (after I fixed the above 2 locally) For some reason, the wasm factory Promise does not return. I have no idea how to fix this one. I think there is not much that I can do before Emscripten officially support Deno.

[fs-eire]

Update: OK I hacked the Emscripten glue JS and finally get into Deno's webgpu. And now I am blocked with a bug in Deno's webgpu implementation:

adapter.limits.maxBufferSize's value is 18446744073709551615n. This is not expected (adapter.limits.maxBufferSize should be number but not BigInt) and will cause an error later:

Cannot convert a BigInt value to a number

link to the issue: https://github.com/denoland/deno/issues/22029

[fs-eire]

I don't know why maxBufferSize on my devbox is bigint. will later change to a different machine and try.

[fs-eire]

changed another PC and the value of maxBufferSize is still 18446744073709551615n.

[mattvr]

According to the WebGPU spec, maxBufferSize is an unsigned long long, so I believe returning a BigInt is correct.

[jlucaso1]

In google chrome adapter.limits.maxBufferSize returns a number:

adapter.limits.maxBufferSize
2147483648
typeof adapter.limits.maxBufferSize
'number'

Chrome: 129.0.6668.100 OS: Linux

[fs-eire]

I am not sure about the details but at least the implementation of Deno WebGPU is incorrect.

It maxBufferSize is considered to be number, then adapter.limits.maxBufferSize being bigint is incorrect.

It maxBufferSize is considered to be bigint, then using it in adapter.requestDevice( { requiredLimits : { maxBufferSize: adapter.limits.maxBufferSize }}) should not run into error Cannot convert a BigInt value to a number.

[jlucaso1]

@fs-eire what deno version you tested early? I'm using deno 2.0.0 stable and getting a normal number in adapter.limits.maxBufferSize

const adapter = await navigator.gpu.requestAdapter();

console.log(adapter?.limits.maxBufferSize); // 2147483647

const device = await adapter?.requestDevice( { requiredLimits : { maxBufferSize: adapter.limits.maxBufferSize! }})

console.log(device) // GPUDevice with no error

[fs-eire]

I tried both 1.46.3 and 2.0.0

deno repl --unstable-webgpu

Deno 1.46.3
exit using ctrl+d, ctrl+c, or close()
> const adapter = await navigator.gpu.requestAdapter();
undefined
> console.log(adapter?.limits.maxBufferSize);
18446744073709551615n
undefined

deno repl --unstable-webgpu

Deno 2.0.0
exit using ctrl+d, ctrl+c, or close()
> const adapter = await navigator.gpu.requestAdapter();
undefined
> console.log(adapter?.limits.maxBufferSize);
18446744073709551615n
undefined
>