Missing modules for `onnxruntime-web` when loaded from jspm.io

jamesbeith commented 4 weeks ago

I'm trying to get @imgly/background-removal to work in the browser using an importmap and jspm.io. However, it appears some modules for onnxruntime-web are missing from jspm.io.

Here's the experimental document.

<!doctype html>
<html>
<head>
  <meta charset="utf-8">
  <title>@imgly/background-removal</title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
   <script type="importmap">
  {
    "imports": {
      "@imgly/background-removal": "https://ga.jspm.io/npm:@imgly/background-removal@1.5.5/dist/index.mjs"
    },
    "scopes": {
      "https://ga.jspm.io/": {
        "iota-array": "https://ga.jspm.io/npm:iota-array@1.0.0/iota.js",
        "is-buffer": "https://ga.jspm.io/npm:is-buffer@1.1.6/index.js",
        "ndarray": "https://ga.jspm.io/npm:ndarray@1.0.19/ndarray.js",
        "onnxruntime-web": "https://ga.jspm.io/npm:onnxruntime-web@1.18.0-esmtest.20240428-624c681c83/dist/ort.min.mjs",
        "onnxruntime-web/webgpu": "https://ga.jspm.io/npm:onnxruntime-web@1.18.0-esmtest.20240428-624c681c83/dist/ort.webgpu.min.mjs",
        "zod": "https://ga.jspm.io/npm:zod@3.23.8/lib/index.mjs"
      }
    }
  }
  </script>
  <script type="module">
    import { removeBackground } from "@imgly/background-removal"
    const imageSrc = "https://unsplash.com/photos/a-QH9MAAVNI/download?force=true&w=1920"
    const config = {
      debug: true,
      device: "cpu", // "cpu" | "gpu"
      proxyToWorker: true // Whether to proxy the calculations to a web worker. (Default true)
    }
    removeBackground(imageSrc, config).then((blob) => {
      // The result is a blob encoded as PNG. It can be converted to an URL to be used as HTMLImage.src
      const url = URL.createObjectURL(blob)
      console.log(url)
    })
  </script>
</body>
</html>

Loading in a browser results in the following error.

Failed to fetch dynamically imported module: https://ga.jspm.io/npm:onnxruntime-web@1.18.0-esmtest.20240428-624c681c83/dist/ort.proxy.min.mjs?import=1".

The URL mentioned returns 404 on jspm.io.

All four configurations of the device and proxyToWorker config result in 404s for their respected files.

device: "cpu", proxyToWorker: true
https://ga.jspm.io/npm:onnxruntime-web@1.18.0-esmtest.20240428-624c681c83/dist/ort.proxy.min.mjs?import=1

device: "cpu", proxyToWorker: false
https://ga.jspm.io/npm:onnxruntime-web@1.18.0-esmtest.20240428-624c681c83/dist/ort-wasm-simd-threaded.mjs

device: "gpu", proxyToWorker: true
https://ga.jspm.io/npm:onnxruntime-web@1.18.0-esmtest.20240428-624c681c83/dist/ort.webgpu.proxy.min.mjs?import=1

device: "gpu", proxyToWorker: false
https://ga.jspm.io/npm:onnxruntime-web@1.18.0-esmtest.20240428-624c681c83/dist/ort-wasm-simd-threaded.jsep.mjs

guybedford commented 2 weeks ago

I tried this out with a newly generated map and the code you provided and am not seeing any error like the one you mention.

Here's the sandbox link:

https://jspm.org/sandbox#H4sIAAAAAAAAA7VVbXPaOBD+zq/Y+mZuyBTbQF4GGMwlnWSStDhNC6RzH2VZYCW25JFkCOn1v9/KNoQwDffpPvhF2tWzz66elYYfYknNOmeQmCwdNYabDyPxqAEwzJghQBOiNDOBU5i523NKg+EmZaOZKL/x0K/G2yWCZCxwlpytcqmMA1QKwwRCrHhskiBmS06ZWw5awAU3nKSupiRlQQcDDP2KwTCS8bpE/eC6+AH4PLkP4ZoJpoiRCm4ziw8hyUvrVcwNzL6PB5iPyfXA9xcbV+9R55nHpf/HrPuQ/H396ekybKv15OXzZNZJaPgYTunHvHPV/zINT7+Oe3L5Mlt+H8/U3cukP365uVp1Lu8fwtXsYn4aGcpPLjCi61pyMNRU8dyArWTg8JJTRvKyUj9LYvWkdgb1BE6d82yRrv2I0KeFkoWIXcUyuSQpOjlb/mRLXOTZ4N015x3v1Dv1Y66Nz0XMnr3sUTtlqF+tioKmMmdvGPwmyI7ZspaGuEQpsj7A6dUJSbS9tm8n0MNp7QBpNyrmc6YO4Wx8EKbjndVpvMUR8X+xqT1KKp2+Xw/3YKQQz6pA8WYoQhYdgNvztNR6XttlOjNMG6/b7p60T7o996x7Qs96Hdo7rvYAN9vLuCh34f3APj6LvPif4lfgv6PxIuMDMdF6fux1j72en/JoX04oqMbmbZ+hX6nfir3uVLiaQCjjImUwSXimoVI/YE9oyGW6nvM01XX3bJqH6LWgoBUN3uPFtJuVqK62qLYWVm5lvns2u99AldRaKr7gInCIkGKdyUI7ox3Gjf3mrUDKzoUN7Z9QNhn7tO06+AVzJbNDPVwC4LGnDcKQBZsoCsFryQuh85ToxKMy8/NEGql94n676YcXFw93t34sVyKVJP5rLhVlgVEF+3MVdPrd9i4yvud8gbibno1ZVCwGYN1b2yl71uJuUxRaC3y/+oN/wLHSq71yJZ/XU/lDqiemKgDr+iNhJmEKjKw8AEeAxzQtUmI4crAWAqg0WJVLPWhesjkpUlNiHDVeBbNfxOamLq06jyMP0UWzGaUyOoJgtM0KiUwxrmLa4nKNEa0PMEFlzGKUDtzfXXtwixUhAiJmAZdMGbRZfsLeCfYPLYWuFtxMw/GtJeCh5Oo4VVELlWJFcYVHFSOGfY0eGTU4rojt+MqUealcNHFFnejRm4YY+tX9hddZea/+CyChcNpvBwAA

Do let me know what I can do to replicate the issue further?

guybedford commented 2 weeks ago

Ah I was able to replicate this locally. I think an override to ensure the file is explicitly available will work, will share shortly.

guybedford commented 2 weeks ago

I added an override in https://github.com/jspm/overrides/commit/bde5bd26e3116a91791f9f9afcf589db75ed96e6 to extend these files into the public exports of the package, and this seems to have made some progress. Note this is specific to this exact version since the package files are still not stable on this major, and will need to be updated for the next version until the file structure stabilizes.

I'm now getting this error:

index.mjs:43 Uncaught (in promise) Error: Failed to create session: "Error: no available backend found. ERR: [wasm] TypeError: Failed to resolve module specifier './ort-wasm-simd-threaded.mjs'". Please check if the publicPath is set correctly.
    at index.mjs:43:84178
    at async createOnnxSession (index.mjs:43:84124)
    at async initBase (index.mjs:43:88458)
    at async initInference (index.mjs:43:88556)
    at async removeBackground (index.mjs:43:89102)

This seems like it should be fixable via providing some expected publicPath configuration? Let me know if I can assist further.

jamesbeith commented 1 week ago

Thanks for your help. I'll take another look.

jspm / project

Missing modules for `onnxruntime-web` when loaded from jspm.io #343