silvia-odwyer / photon

⚡ Rust/WebAssembly image processing library
https://silvia-odwyer.github.io/photon
Apache License 2.0
2.69k stars 152 forks source link

Initializing PhotonImage multiple times would run into OOM in Node v20 #175

Open CYBAI opened 5 months ago

CYBAI commented 5 months ago

When initializing a PhotonImage instance from base64 multiple times, it could run into memory leak status and node would be stopped due to oom exception.

In my use case, I need to initialize instances for different images but, for demo purpose, I use same image base64 here.

const fs = require('node:fs');
const photon = require("@silvia-odwyer/photon-node");

// read file, then convert to base64
const base64 = fs.readFileSync(`input.png`, { encoding: 'base64' });
const data = base64.replace(/^data:image\/(png|jpg);base64,/, "");

for (let i = 0; i < 500; i++) {
  // convert base64 to PhotonImage
  const phtn_img = photon.PhotonImage.new_from_base64(data);
  console.log(i + 1);
}

output: it got oom-ed around the 174 instance on my end.

...
172 :
173 :
wasm://wasm/0052704a:1

RuntimeError: unreachable
    at wasm://wasm/0052704a:wasm-function[544]:0xc9986
    at wasm://wasm/0052704a:wasm-function[1135]:0xf45bb
    at wasm://wasm/0052704a:wasm-function[595]:0xcf163
    at wasm://wasm/0052704a:wasm-function[1201]:0xf6d0e
    at PhotonImage.new_from_base64 (/Users/cybai/codespace/playground/js/photon-oom/node_modules/.pnpm/@silvia-odwyer+photon-node@0.3.3/node_modules/@silvia-odwyer/photon-node/photon_rs.js:3802:24)
    at Object.<anonymous> (/Users/cybai/codespace/playground/js/photon-oom/index.js:10:39)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Module._load (node:internal/modules/cjs/loader:1022:12)

Environment:

CYBAI commented 5 months ago

I'm investigating this with @austinried and we wonder this issue could happen because the 0.3.3 photon-node was not built with --weak-refs so the instances are not gc-ed automatically.

However, after trying to build photon with --weak-refs --reference-types locally, I can still reproduce the issue with same snippet.

Then, it brings us to find out the FinalizationRegistry is not really working at the moment when it's run synchronously; related issues at https://github.com/rustwasm/wasm-bindgen/issues/3917 (and possibly https://github.com/rustwasm/wasm-bindgen/issues/3854?).

So, before the issue is fixed, photon wasm users may need to call .free() manually to avoid memory leak.

note that we can fix the issue by calling .free() manually.