[Web] ort-wasm-simd.wasm can't be loaded in Electron renderer (using webpack)

eisneim commented 1 year ago

Describe the issue

I‘m building a electron app, the ort.js is loaded using script tag, but the wasm file is not loaded in webpack-dev-server
electron packed into an App won't load either.

  <head>
      <base href="../../"> <!-- without this line, the issue still remains -->
      <title>my app</title>
      <script type="text/javascript" src="apps/videoAnalyzer/js/ort.js"></script>
    </head>

i'm sure i have copyed .wasm file near the ort.js file

the path is incorrect http://ort-wasm-simd.wasm ort-web.min.js?e269:6 GET http://ort-wasm-simd.wasm/ net::ERR_ABORTED 502 (Bad Gateway)

after some testing i guess the problem is in the "ort-wasm.js" file, but it's been minified so i can't do further test; or maybe it is because Electron's issue

any help would be appreciated

To reproduce

electron app, using webpack boundle and set target to "electron-renderer" test electron version: 22.0.0 and 16.0.6 both version won't work properly

Urgency

No response

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.12.0

Execution Provider

WASM

fs-eire commented 1 year ago

not sure if this is an Electron specific issue.

a workaround is to set the ort.env.wasm.wasmPaths to specify the path of wasm files

eisneim commented 1 year ago

@fs-eire thanks for your reply that solve the problem of loading wasm file! but the file finding related function is still not working, now i'm stuck at TypeError: Worker is not a constructor which means when importing the working.js file, it can't find the js file

caused by this function in ort-wasm-threaded.js function ma(a){return w.locateFile?w.locateFile(a,E):E+a}var na,F,G,fs,H,J;

i guess the locateFile() function is implemented without testing in Electron environment

const ort = require('onnxruntime-web')
async initializeWorkers() {
    ort.env.wasm.wasmPaths = this.wasmPath
   let session = await ort.InferenceSession.create("path to my xx.onnx", { executionProviders: ["wasm"] }); 
   // it stoped here
}

eisneim commented 1 year ago

i give up use wasm in web front end, and switch to Nodejs with onnxruntime-node, and it works great! but i'll have to do multi threading myself not like using wasm.

Daniel-Kelvich commented 1 year ago

@eisneim any updates on this?

eisneim commented 1 year ago

@eisneim any updates on this?

currently i'm using it in Nodejs with onnxruntime-node, it works good

FangliangBai commented 1 year ago

@eisneim 以前都是看你视频来的，很高兴在 github 上碰到你！我最近想用 nextron 在 electron里运行一个 nextjs 然后跑一个模型，也遇到了和你一模一样的问题，困扰了好几天都没有想到解决办法。我也想打算放弃了转 onnxruntime-node。可是我完全没有头绪。想冒昧问一下，如果你的项目开源，我是否可以参考以下看看是怎么实现的吗？如果不开源，是否可以分享一些你觉得有用的相关教程？非常感谢！！

conceptualspace commented 1 year ago

@eisneim do you have it working with electron in production? just curious what packager you use? there are issues with electronforge+webpack and maybe you can spare me from going too far down that rabbit hole :smile:

eisneim commented 1 year ago

@eisneim do you have it working with electron in production? just curious what packager you use? there are issues with electronforge+webpack and maybe you can spare me from going too far down that rabbit hole 😄

i'm using electron-builder, with no hack! here are the versions i use:

"electron": "^16.0.6",
"electron-builder": "^22.14.10",
"onnxruntime-node": "^1.13.1",

in production, you have to build onnx-runtime binding for differient platforms, darwin, win64, win32 etc.

@FangliangBai 这也回答了你的问题

FangliangBai commented 1 year ago

Hi, @eisneim Thanks for the node packages. I was wondering how to use onnxruntime-node in electron backend nodejs and communicate with the frontend. But I eventually managed to do so.

When you say building from different platforms, what do you specifically mean? I am asking because I didn't build anything but just install the package and it works. For what purpose do I need to build the onnx-runtime binding?

fs-eire commented 1 year ago

The onnxruntime-node package comes with pre-built binaries for 6 environments:

win/x64
win/arm64
linux/x64
linux/arm64
macos/x64
macos/arm64

This should have already covered most of the usage. You don't need to build if your target environment is in this list.

eisneim commented 1 year ago

The onnxruntime-node package comes with pre-built binaries for 6 environments:

win/x64

win/arm64

linux/x64

linux/arm64

macos/x64

macos/arm64

This should have already covered most of the usage. You don't need to build if your target environment is in this list.

but in production, electron only pack specific binding on specific platform, for example, when i pack it on mac arm64, send this app to Mac/x64 the app won't start, it will say can't find node binding!

this is very annoying for software update, so i have to build different electron ASAR package for differient platforms; @FangliangBai

FangliangBai commented 1 year ago

Got it! Thanks @fs-eire @eisneim !

fs-eire commented 1 year ago

The onnxruntime-node package comes with pre-built binaries for 6 environments:

win/x64

win/arm64

linux/x64

linux/arm64

macos/x64

macos/arm64

This should have already covered most of the usage. You don't need to build if your target environment is in this list.

but in production, electron only pack specific binding on specific platform, for example, when i pack it on mac arm64, send this app to Mac/x64 the app won't start, it will say can't find node binding!

this is very annoying for software update, so i have to build different electron ASAR package for differient platforms; @FangliangBai

could you share me a reproducing steps (ideally a repo) that reproduce the issue? I would like to take a look

eisneim commented 1 year ago

@fs-eire just pack it on windows10 and sent this app to mac and run, you'll see the error

"electron": "^16.0.6",
"electron-builder": "^22.14.10",
"onnxruntime-node": "^1.13.1",

fs-eire commented 1 year ago

electron-builder works as expected on my Windows machine. It has everything in the dist folder (after npx electron-builder):

(base) D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3>dir /s /b
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\darwin
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\linux
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\darwin\arm64
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\darwin\x64
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\darwin\arm64\libonnxruntime.1.14.0.dylib
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\darwin\arm64\onnxruntime_binding.node
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\darwin\x64\libonnxruntime.1.14.0.dylib
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\darwin\x64\onnxruntime_binding.node
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\linux\arm64
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\linux\x64
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\linux\arm64\libonnxruntime.so.1.14.0
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\linux\arm64\onnxruntime_binding.node
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\linux\x64\libonnxruntime.so.1.14.0
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\linux\x64\onnxruntime_binding.node
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32\arm64
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32\x64
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32\arm64\onnxruntime.dll
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32\arm64\onnxruntime_binding.node
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32\arm64\onnxruntime_providers_shared.dll
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32\x64\onnxruntime.dll
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32\x64\onnxruntime_binding.node
D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3\win32\x64\onnxruntime_providers_shared.dll

(base) D:\eire\electron-quick-start\dist\win-unpacked\resources\app.asar.unpacked\node_modules\onnxruntime-node\bin\napi-v3>

packages version is slightly different. not sure if this is the reason.

  "devDependencies": {
    "electron": "^23.1.0",
    "electron-builder": "^23.6.0"
  },
  "dependencies": {
    "onnxruntime-node": "^1.14.0"
  }

compilelife commented 1 year ago

if you're already using onnxruntime-web, a tricky workaround would be:

import { env } from "onnxruntime-web";
  env.wasm.wasmPaths = window.location.protocol === 'file:' ? window.location.href.substring(0, window.location.href.indexOf('app.asar')) : ''

copy wasm like this (electron builder):

"extraResources": [
    {
      "from": "node_modules/onnxruntime-web/dist/ort-wasm-simd-threaded.wasm",
      "to": ".",
    }
  ],

will be placed to and load from: release/linux-unpacked/resources/ort-wasm-simd-threaded.wasm

manuelernestog commented 11 months ago

env.wasm.wasmPaths = window.location.protocol === 'file:' ? window.location.href.substring(0, window.location.href.indexOf('app.asar')) : ''

This worked for me!

arpieb commented 2 months ago

@eisneim do you have it working with electron in production? just curious what packager you use? there are issues with electronforge+webpack and maybe you can spare me from going too far down that rabbit hole 😄

This electronforge+webpack appears to still be an issue (running into it atm trying to stand up an electron + react + onnxruntime-node project). Anyone know if/where this is being tracked, having no luck searching for it... Thanks!

microsoft / onnxruntime