Could not load the "sharp" module using the win32-x64 runtime when using webpack

Markos-Vasalos commented 7 months ago

I've installed the c2pa-node library through npm, but as soon as i try to build my node app (i'm using webpack for this), i get the following error: "Could not load the "sharp" module using the win32-x64 runtime". (i'm running on windows 11)

The c2pa-node version currently installed is 0.5.20.

Thanks a lot in advance!

dkozma commented 7 months ago

Hi @Markos-Vasalos, would you be able to post any related logs? I'm seeing some related issues in the sharp module (we had to recently upgrade to the latest version due to an issue we were having on our automation).

Additionally, does c2pa-node@0.5.19 work for you?

Markos-Vasalos commented 7 months ago

Hello @dkozma and thank you for your quick reply!

First of all webpack gives me these warnings (not sure if they are related)

WARNING in ./node_modules/sharp/lib/libvips.js 60:13-54 Module not found: Error: Can't resolve '@img/sharp-libvips-dev/include' in 'C:\Users\mvasalos\Documents\My-projects\ProjectDenn\code_repos\backend\node_modules\sharp\lib' @ ./node_modules/sharp/lib/utility.js 10:32-52 @ ./node_modules/sharp/lib/index.js 14:0-20 @ ./node_modules/c2pa-node/dist/js-src/lib/thumbnail.js 15:32-48 @ ./node_modules/c2pa-node/dist/js-src/index.js 13:20-46 @ ./src/domain/media/media-service.js 6:0-72 31:15-30 52:25-41 53:17-27 @ ./src/application/controller/media-controller.js 5:0-59 26:4-13 @ ./src/application/configuration/routes.js 4:0-61 9:37-52 @ ./src/index.js 4:0-69 9:0-16

WARNING in ./node_modules/sharp/lib/libvips.js 69:11-54 Module not found: Error: Can't resolve '@img/sharp-libvips-dev/cplusplus' in 'C:\Users\mvasalos\Documents\My-projects\ProjectDenn\code_repos\backend\node_modules\sharp\lib' @ ./node_modules/sharp/lib/utility.js 10:32-52 @ ./node_modules/sharp/lib/index.js 14:0-20 @ ./node_modules/c2pa-node/dist/js-src/lib/thumbnail.js 15:32-48 @ ./node_modules/c2pa-node/dist/js-src/index.js 13:20-46 @ ./src/domain/media/media-service.js 6:0-72 31:15-30 52:25-41 53:17-27 @ ./src/application/controller/media-controller.js 5:0-59 26:4-13 @ ./src/application/configuration/routes.js 4:0-61 9:37-52 @ ./src/index.js 4:0-69 9:0-16

WARNING in ./node_modules/sharp/lib/sharp.js 24:12-25 Critical dependency: the request of a dependency is an expression @ ./node_modules/sharp/lib/constructor.js 10:0-18 @ ./node_modules/sharp/lib/index.js 6:14-38 @ ./node_modules/c2pa-node/dist/js-src/lib/thumbnail.js 15:32-48 @ ./node_modules/c2pa-node/dist/js-src/index.js 13:20-46 @ ./src/domain/media/media-service.js 6:0-72 31:15-30 52:25-41 53:17-27 @ ./src/application/controller/media-controller.js 5:0-59 26:4-13 @ ./src/application/configuration/routes.js 4:0-61 9:37-52 @ ./src/index.js 4:0-69 9:0-16

WARNING in ./node_modules/sharp/lib/utility.js 72:17-54 Module not found: Error: Can't resolve '@img/sharp-wasm32/versions' in 'C:\Users\mvasalos\Documents\My-projects\ProjectDenn\code_repos\backend\node_modules\sharp\lib' @ ./node_modules/sharp/lib/index.js 14:0-20 @ ./node_modules/c2pa-node/dist/js-src/lib/thumbnail.js 15:32-48 @ ./node_modules/c2pa-node/dist/js-src/index.js 13:20-46 @ ./src/domain/media/media-service.js 6:0-72 31:15-30 52:25-41 53:17-27 @ ./src/application/controller/media-controller.js 5:0-59 26:4-13 @ ./src/application/configuration/routes.js 4:0-61 9:37-52 @ ./src/index.js 4:0-69 9:0-16

And ends up with:

webpack 5.90.1 compiled with 23 warnings in 2282 ms webpack-internal:///./node_modules/sharp/lib/sharp.js:114 throw new Error(help.join('\n')); ^

Error: Could not load the "sharp" module using the win32-x64 runtime Possible solutions:

Ensure optional dependencies can be installed: npm install --include=optional sharp yarn add sharp --ignore-engines
Ensure your package manager supports multi-platform installation: See https://sharp.pixelplumbing.com/install#cross-platform
Add platform-specific dependencies: npm install --os=win32 --cpu=x64 sharp
Consult the installation documentation: See https://sharp.pixelplumbing.com/install at eval (webpack-internal:///./node_modules/sharp/lib/sharp.js:114:9)

I've tried various solutions like installing sharp separately or using specific --os and --cpu flags like the ones specified above but the issue persists. If you need specific logs let me know.

Also i tried c2pa-node@0.5.19 but i got the same results.

dkozma commented 7 months ago

I'm thinking it may have to do with this change: https://github.com/lovell/sharp/issues/3750

Which package manager are you using? I see that the author recommends the latest version of yarn if you are using that by any chance.

Markos-Vasalos commented 7 months ago

Unfortunately i'm only using npm, version 10.4.0 and my node is 20.10.0

Markos-Vasalos commented 6 months ago

@dkozma After checking further, the issue is not with the c2pa but with the webpack. If i add webpack in a new project and run it, it produces the same error, even though without webpack is working fine. Steps to reproduce are the following:

create a new empty node project and add the following code inside index.js

` import { createC2pa } from 'c2pa-node'; import { readFile } from 'node:fs/promises';

const c2pa = createC2pa();

async function read(path, mimeType) { const buffer = await readFile(path); const result = await c2pa.read({ buffer, mimeType });

if (result) {
    const { active_manifest, manifests, validation_status } = result;
    console.log(active_manifest);
} else {
    console.log('No claim found');
}

}

await read('signed.jpg', 'image/jpeg'); `

If you install the c2pa-node through npm and add inside the project a c2pa signed image with the name "signed.jpg" and run it using node index.js it will work as intended.

For the above error, i've installed webpack doing npm i webpack webpack-cli -D and created the webpack.config.js file with the following content:

`const path = require('path')

module.exports={ target: 'node', mode: 'development', devtool: 'eval-source-map', entry: './index.js', output: { path: path.resolve('./dist'), filename: "bundle.cjs" } }`

Then run it with the script

webpack && node ./dist/bundle.cjs

Is any other way c2pa-node can work with webpack?

SteveMaier-IRT commented 6 months ago

You can also have it run using this optional version of sharp. Made it work for me locally on my Win64 machine.

npm i @img/sharp-win32-x64 --save-optional

dkozma commented 6 months ago

Hi @Markos-Vasalos, I'm not used to seeing Node packages being used with Webpack that often, so I'm not sure what its doing under the hood, however I wonder if it is trying to post-process some of the packages and causing issues.

What happens if you try to add sharp to the externals as mentioned here? https://sharp.pixelplumbing.com/install#webpack

Markos-Vasalos commented 6 months ago

@SteveMaier-IRT tried it but unfortunately it doesn't work with webpack

@dkozma Tried it and it changed the error. Now it complaints that it cannot find the module. Surely webpack does optimizations in dependencies under the hood so it probably breaks something. I might work with a different bundler if necessary instead of webpack. Thank you very much for your effort so far

joshh20 commented 4 months ago

Also experiencing this issue. Did you ever figure out a solution?

Markos-Vasalos commented 4 months ago

@joshh20 I just stopped using webpack. It wasn't necessary for my project eventually.

joshh20 commented 4 months ago

@Markos-Vasalos Thank you.

contentauth / c2pa-node

Could not load the "sharp" module using the win32-x64 runtime when using webpack #69