MAXIM Denoising not working as intended?

[Skyknight47]

Hey, So I've recently started using additional models (aside from the upscaling). Specifically, the MAXIM Denoising model for node.js

For me however, the only noise it removes is like 0.5% (not even joking.

For reference;

This is obviously the example shown, with the Input/Ouputand here's my Output after utilising the code shown below;

 const denoiser = new Upscaler({
        DenoiserModel
    })
 let image = await jimp.read(attachment)
    await image.writeAsync(`./results/${interaction.user.id}/predenoised_${attachment.name}`)
   let toDenoise = tf.node.decodeImage(fs.readFileSync(`./results/${interaction.user.id}/predenoised_${attachment.name}`), 3)
   const tensor = await denoiser.upscale(toDenoise)

   const denoisedTensor = await tf.node.encodePng(tensor)
   fs.writeFileSync(`./results/${interaction.user.id}/denoised_${attachment.name}`, denoisedTensor)

Any idea why, and more importantly, how to avoid/fix this?

[thekevinscott]

I think there's a few things to try:

1. Does it work in the browser for you? Check this link (https://codesandbox.io/p/devbox/upscalerjs-demo-maxim-denoising-msfsy8?embed=1&file=%2Findex.js%3A13%2C4) - and check out the console. It will probably take some time to complete, but you should see a denoised image after a few minutes.
2. Do other models work, or is it just the denoising one?
3. Does supplying a patch size (i.e., { patchSize: 64, padding: 2, progress: console.log }) have any effect on the output?

That'll help start narrowing down where the issue is.

[Skyknight47]

Hey there,

1. Yep, after about 5 minutes and a few time getting the notification that "This webpage isn't responding", I've gotten the expected result. Although (maybe irrelevant) progress isn't printed out into the console.
2. Nope. I tried the Maxim-deblurring which doesn't work. And I obviously use the default upscaling model which does in fact works as intended)
3. Nope. It unlike the browser version does print out the progress, though, it has no effect on the actual image result.

[thekevinscott]

Appreciate the testing, and sorry you're running into this.

Let's try a different tack. Try cloning this repo (https://github.com/thekevinscott/upscaling-noise-bug), running npm install and then npm run dev, and see what it writes to output.png. It should write a denoised image.

The model is the same on browser and node, so given that it's working in the browser, I would expect it to be identical in Node, which would imply some sort of issue in your code. However, I don't see any problems in the code you originally shared.

If the linked repo works, then we can figure out what's different between your code and the linked repo; if it doesn't, well then, I've no idea! (In that case, please share your Node version, hardware configuration, etc. as maybe that'll provide a clue.)

[Skyknight47]

Unfortunately, this didn't work.

After running npm run dev I'm getting the following error;

> upscaling-test@1.0.0 dev
> node index.js

node:internal/modules/cjs/loader:1327
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: The specified module could not be found
\\?\C:\Users\mikeu\Desktop\Coding\upscaling-test\node_modules\@tensorflow\tfjs-node\lib\napi-v8\tfjs_binding.node
    at Module._extensions..node (node:internal/modules/cjs/loader:1327:18)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Module._load (node:internal/modules/cjs/loader:938:12)
    at Module.require (node:internal/modules/cjs/loader:1115:19)
    at require (node:internal/modules/helpers:130:18)
    at Object.<anonymous> (C:\Users\mikeu\Desktop\Coding\upscaling-test\node_modules\@tensorflow\tfjs-node\dist\index.js:72:16)
    at Module._compile (node:internal/modules/cjs/loader:1241:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Module._load (node:internal/modules/cjs/loader:938:12) {
  code: 'ERR_DLOPEN_FAILED'
}

Node.js v20.9.0

I've tried about anything (reinstalling, updating node, updating npm, rebuilding), pretty much anything in similar topics like this, or this, or this one. Unfortunately all, without result.

Node version: V20.9.0 NPM version: 10.8.0 CPU: AMD Ryzen 5 5600X (which supports AVX instructions, AVX and AVX2, just not AVX-512 apparently) GPU: MSI NVIDIA GeForce GTX 1050Ti Windows 10

P.S If you're wondering why the path in the error is "upscaling-test" rather than "upscaling-noise-bug", according to one of the possible solutions, creating it in a new folder, and rebuilding the entire project in there could help.

[thekevinscott]

Just to clarify, if you do something like:

const tf = require('@tensorflow/tfjs-node');
const image = tf.node.decodeImage(fs.readFileSync('./fixture.png'), 3);

It gives you the above issue? (I'm unclear whether the error originates in TFJS or in Upscaler).

If it's an issue with TFJS, it's probably above my pay grade and better asked on the TFJS repo. FWIW I also ran into a bunch of problems getting the latest TFJS Node to build, but I'm on a Mac and it was related to Rosetta not playing nicely with the arm build. Doesn't sound like what you're seeing.

You could try going back to the original repo you posted with, use the installed node_modules from there and just replace the index.js file with the above code, assuming it's still working.