Error: Could not load the "sharp" module using the darwin-arm64 runtime

pvlvstepan commented 8 months ago

Possible install-time or require-time problem

[x] I have read and understood all of the documentation relating to installation.
[x] I have searched for known bugs relating to this problem in my choice of package manager.

Possibly related to #3960

You must confirm both of these before continuing.

Are you using the latest version of sharp?

[x] I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.

If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.

Are you using a supported runtime?

[x] I am using Node.js with a version that satisfies ^18.17.0 || ^20.3.0 || >=21.0.0
[ ] I am using Deno
[ ] I am using Bun

If you cannot confirm any of these, please upgrade to the latest version and try again before opening an issue.

Are you using a supported package manager and installing optional dependencies?

[ ] I am using npm >= 9.6.5 with --include=optional
[x] I am using yarn >= 3.2.0
[ ] I am using pnpm >= 7.1.0 with --no-optional=false
[ ] I am using Deno
[ ] I am using Bun

If you cannot confirm any of these, please upgrade to the latest version of your chosen package manager and ensure you are allowing the installation of optional or multi-platform dependencies before opening an issue.

What is the complete error message, including the full stack trace?

yarn dev
yarn run v1.22.21
$ next dev
/Users/pvlvstepan/Documents/GitHub/stepanpavlov.com/node_modules/sharp/lib/sharp.js:114
  throw new Error(help.join('\n'));
        ^

Error: Could not load the "sharp" module using the darwin-arm64 runtime
Possible solutions:
- Ensure optional dependencies can be installed:
    npm install --include=optional sharp
    yarn add sharp --ignore-engines
- Ensure your package manager supports multi-platform installation:
    See https://sharp.pixelplumbing.com/install#cross-platform
- Add platform-specific dependencies:
    npm install --os=darwin --cpu=arm64 sharp
- Consult the installation documentation:
    See https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (/Users/pvlvstepan/Documents/GitHub/stepanpavlov.com/node_modules/sharp/lib/sharp.js:114:9)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)
    at Module.require (node:internal/modules/cjs/loader:1235:19)
    at mod.require (/Users/pvlvstepan/Documents/GitHub/stepanpavlov.com/node_modules/next/dist/server/require-hook.js:65:28)
    at require (node:internal/modules/helpers:176:18)
    at Object.<anonymous> (/Users/pvlvstepan/Documents/GitHub/stepanpavlov.com/node_modules/sharp/lib/constructor.js:10:1)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)

Node.js v21.2.0
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

What is the complete output of running `npm install --verbose --foreground-scripts sharp` in an empty directory?

``` npm install --verbose --foreground-scripts sharp npm verb cli /Users/pvlvstepan/.nvm/versions/node/v21.2.0/bin/node /Users/pvlvstepan/.nvm/versions/node/v21.2.0/bin/npm npm info using npm@10.2.3 npm info using node@v21.2.0 npm verb title npm install sharp npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp" npm verb logfile logs-max:10 dir:/Users/pvlvstepan/.npm/_logs/2024-02-18T05_00_30_017Z- npm verb logfile /Users/pvlvstepan/.npm/_logs/2024-02-18T05_00_30_017Z-debug-0.log npm http fetch GET 200 https://registry.npmjs.org/sharp 1010ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/semver 99ms (cache updated) npm http fetch GET 200 https://registry.npmjs.org/color 901ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/detect-libc 919ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-wasm32 1506ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-x64 1507ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-x64 1509ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-x64 1511ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-arm64 1510ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-arm64 1509ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-x64 1509ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-arm64 1513ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm 1510ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-arm64 1512ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm64 1511ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-x64 1512ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm 1519ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-x64 2055ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-s390x 2067ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-ia32 2072ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm64 2088ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-x64 2088ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-s390x 3638ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/lru-cache 45ms (cache updated) npm http fetch GET 200 https://registry.npmjs.org/color-convert 57ms (cache revalidated) npm http fetch GET 200 https://registry.npmjs.org/color-string 995ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@emnapi%2fruntime 1494ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/tslib 38ms (cache revalidated) npm http fetch GET 200 https://registry.npmjs.org/color-name 35ms (cache revalidated) npm http fetch GET 200 https://registry.npmjs.org/color-name 44ms (cache revalidated) npm http fetch GET 200 https://registry.npmjs.org/simple-swizzle 379ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/is-arrayish 30ms (cache revalidated) npm http fetch GET 200 https://registry.npmjs.org/yallist 32ms (cache revalidated) npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-win32-x64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-win32-ia32 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-wasm32 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@emnapi/runtime npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/tslib npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-linuxmusl-x64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-linuxmusl-arm64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-linux-x64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-linux-s390x npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-linux-arm64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-linux-arm npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-libvips-linuxmusl-x64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-libvips-linuxmusl-arm64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-libvips-linux-x64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-libvips-linux-s390x npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-libvips-linux-arm64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-libvips-linux-arm npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-libvips-darwin-x64 npm verb reify failed optional dependency /Users/pvlvstepan/Documents/GitHub/sharp-debug/node_modules/@img/sharp-darwin-x64 npm http fetch GET 200 https://registry.npmjs.org/color-string/-/color-string-1.9.1.tgz 40ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/simple-swizzle/-/simple-swizzle-0.2.2.tgz 40ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/color/-/color-4.2.3.tgz 41ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/detect-libc/-/detect-libc-2.0.2.tgz 43ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/is-arrayish/-/is-arrayish-0.3.2.tgz 47ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/semver/-/semver-7.6.0.tgz 48ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/sharp/-/sharp-0.33.2.tgz 83ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@emnapi/runtime/-/runtime-0.45.0.tgz 318ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img/sharp-libvips-darwin-arm64/-/sharp-libvips-darwin-arm64-1.0.1.tgz 627ms (cache miss) npm http fetch GET 200 https://registry.npmjs.org/@img/sharp-darwin-arm64/-/sharp-darwin-arm64-0.33.2.tgz 1171ms (cache miss) npm info run sharp@0.33.2 install node_modules/sharp node install/check > sharp@0.33.2 install > node install/check npm info run sharp@0.33.2 install { code: 0, signal: null } added 13 packages in 9s 3 packages are looking for funding run `npm fund` for details npm verb exit 0 npm info ok ```

What is the output of running `npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp`?

npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp

  System:
    OS: macOS 14.2.1
    CPU: (8) arm64 Apple M1
    Memory: 215.59 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 21.2.0 - ~/.nvm/versions/node/v21.2.0/bin/node
    Yarn: 1.22.21 - ~/.nvm/versions/node/v21.2.0/bin/yarn
    npm: 10.2.3 - ~/.nvm/versions/node/v21.2.0/bin/npm
    pnpm: 8.15.3 - ~/.nvm/versions/node/v21.2.0/bin/pnpm
  npmPackages:
    sharp: ^0.33.2 => 0.33.2

Additional details

I found that running yarn add sharp --ignore-engines fixes the issue temporary but I haven't found a way to reproduce how it is coming back.

sharp is a dependency of a library that I use - velite, but I also have sharp as a dependency in my project.

lovell commented 8 months ago

Hi, you've ticked the box that says "I am using yarn >= 3.2.0" but it looks like you're using yarn 1.22.21. You appear to have seen from the installation documentation that the --ignore-engines flag is required when using it.

I simply cannot recommend highly enough to those that have selected yarn as their package manage of choice that they upgrade to the latest version.

KillerCodeMonkey commented 7 months ago

We got same errors but for darwin-x64 and linux-x64. We changed nothing on our build-pipeline. We are using electron 28 with the matching node v18 version required by electron to install fitting native dependencies. We are using electron-builder for creating the final package.

We are building it for years on github without any problems with sharp (thanks @lovell for the outstanding work here!).

But now we are getting this errors Error: Could not load the "sharp" module using the darwin-x64|linux-x64 runtime on startup with latest sharp releases (i will try to test it out if it is working with 0.32. just to be sure).

I tried building with for different mac and ubuntu versions, but same result.

It is working fine when we developing all deps are there. But when we finally bundle it, something is missing i guess.

Any hint on that?

lovell commented 7 months ago

@KillerCodeMonkey For Electron please see https://sharp.pixelplumbing.com/install#electron and for cross-platform see https://sharp.pixelplumbing.com/install#cross-platform

If you still require help, please open a new installation issue and answer all of the questions.

lovell commented 7 months ago

I'm going to close this as the --ignore-engines flag for those who continue to use the maintenance-mode-only yarn v1 is documented. You will run into exactly the same problem with an increasing number of native dependencies such as esbuild, parcel, swc etc.

I continue to recommend people migrate away from the use of yarn v1. It takes minutes yet could save you hours.

https://mastodon.social/@lovell/111726943907221407

KillerCodeMonkey commented 7 months ago

@lovell thanks for you super fast responses... seems like i really missed that the @img/** things are now in the asarUnpack array. *facepalm

coreyward commented 5 months ago

@lovell Seems like V***** still doesn't support versions of Yarn above v1, so upgrading isn't an option for many people. https://vercel.com/docs/deployments/builds/package-managers

lovell commented 5 months ago

@coreyward This issue is unrelated to the company beginning with V (I have edited your post for anti-SEO purposes to prevent future comment spam).

If you have a question for them you'll need to ask their support team. I suspect the page you've linked to is out of date as I have been made aware that the opposite is true, in that yarn v1 support is likely to soon be removed.

coreyward commented 5 months ago

Interesting. I can confirm that Yarn v4 does not work on V*****, so I don't know what you heard but perhaps they're unaware of this issue. The experimental corepack flag triggers this error when the package manager directive is set to 3.7 or 4.x:

It is possible to use 4.x locally but not enable the experimental corepack flag on V*****, but that results in Yarn 1.22 being used, which seems to behave differently w/r/t which packages get loaded. In my case all builds fail due to the types being resolved differently than I get locally.

lovell commented 5 months ago

@coreyward Perhaps this is https://github.com/yarnpkg/berry/issues/5831? You'll need to ask the support team at the company that must not be named (and thank you for not doing so).

coreyward commented 5 months ago

@lovell I took your advice and reached out to support. It didn’t really help. For posterity, here’s the full reply:

I’ll see if I can find an existing discussion for this, but as of the moment there isn’t really a way for users to upgrade to Yarn 4 on this particular popular infrastructure provider.

lovell commented 5 months ago

https://github.com/search?q=ENABLE_EXPERIMENTAL_COREPACK&type=discussions

lforst commented 3 months ago

Currently running into this issue locally and on Vercel. Hoping I am not holding anything wrong.

I continue to recommend people migrate away from the use of yarn v1. It takes minutes yet could save you hours.

@lovell I think yarn v1 is one of the most stable package managers and it handles optionalDeps perfectly fine. Esbuild, swc, and others work just fine with it. I wrote a post on our engineering blog that outlines potential approaches on how to effectively distribute binaries inside npm packages: https://sentry.engineering/blog/publishing-binaries-on-npm (ignore the title, I think you're much more capable and not the perfect target audience 😂) We use this approach for @sentry/cli and it has reduced our inbox to 0. Maybe you can re-evaluate your approach?

lovell commented 3 months ago

@lforst Thanks, if you hadn't seen it, https://github.com/lovell/sharp/issues/4130 should improve things for yarn v1 users.

lforst commented 3 months ago

@lovell awesome, I did not! Thanks for the heads up. :)

JohnMichaelangelo20 commented 2 weeks ago

I'm getting this error all of a sudden. Any solution?

my npm version is 10.9.0

lovell / sharp