[Web] Feature Request: workaround for bundlers when using onnxruntime-web

[fs-eire]

Summary

This issue is created to track the status of onnxruntime-web's workaround for bundlers.

What is workaround for bundlers and why?

workaround for bundlers means extra codes that required to make onnxruntime-web as a library consumable by a web app or other libraries, while those extra codes are supposed to be unnecessary but exists because of various reasons.

What are the known issues:

1. import.meta.url as a runtime JavaScript expression, is usually rewritten into a static string starts with "file://" by WebPack as a default behavior. A few other bundlers followed this behvior. This means it is very difficult to use ESM dynamic import in the implementation.

2. Vite development mode does not support "real" ESM. It uses some complicated steps to transcript the ESM sources into IIFE/cjs. https://github.com/microsoft/onnxruntime/pull/22196 tries to fix this but the fix is incompatible with Vite release mode.

3. Some framework (eg. SvelteKit) always load onnxruntime-web in a Node.js context even if SSR is disabled . https://github.com/microsoft/onnxruntime/issues/22361#issuecomment-2439411518

4. Emscripten is not able to generate both ESM/commonJS glue file in a build. (https://github.com/emscripten-core/emscripten/issues/21899)

Why is it difficult to fix:

It's all about dynamic loading with bundler involved.

onnxruntime-web has 4 dynamic loading target:

• onnxruntime-web itself as an entry of WebWorker, used in proxy worker feature
• The WebAssembly file (.wasm suffix, generated by Emscripten)
• The MJS file (.mjs suffix, generated by Emscripten as JavaScript glue)
• The same MJS file above as an entry of WebWorker, used in WebAssembly Multi-threading

Considering the following requirements:

• support ESM
• support CSP
• support non-bundler usage. (eg. import ort.min.js from CDN in HTML script tag)
• support major bundlers/frameworks. ("support" means user should be able to use onnxruntime-web as a dependency of their web app when using the bundlers, with no or little extra bundler config)

It's not hard to conclude that:

• the number of combinations of usage variants is large.
• in some usage, certain requirements may be conflicting with others.

For example:

• UMD format cannot be supported in CSP. This is because:

  • CSP does not allow any dynamic loading. So in order to support CSP, onnxruntime-web need to offer a way to export an "all-in-one" bundle JS that does not dynamic import anything.
  • A UMD export of onnxruntime-web need to import an ESM JavaScript (because of known issue No.4), so dynamic import is required for UMD.

• proxy worker is not well supported when using with multi-threading. This is because:

  • proxy worker is loaded using new Worker(new URL("ort.min.mjs", import.meta.url), {type: "module"}), and webpack will use its builtin worker loader to load the worker as a static loaded module.
  • when the glue JS called from generated proxy worker module tries to create another worker, it failed to load. (detailed reason is still investigating. It's very likely because of WebPack still does not fully support ESM)

There are more case that makes onnxruntime-web not working out-of-box in different combinations of dev frameworks, bundlers, dev/prod modes and so on, and one fix for a specific usage may even break a specific existing working usage.

It seems that there is no way to make every use scenario work. Need to track issues and find workaround for each.

[ollema]

there are workarounds for issue 3 - but they involve patching this package to remove the "node": null entries so I am not sure if that can be called a workaround: https://github.com/ollema/g-split/commit/15f616cb8c10b3a32243ca4c872c1bce0ff44f8e