search

payloadcms / payload

Payload is the open-source, fullstack Next.js framework, giving you instant backend superpowers. Get a full TypeScript backend and admin panel instantly. Use Payload as a headless CMS or for building powerful applications.
https://payloadcms.com
MIT License
24.88k stars 1.58k forks source link

Error installing sharp #1303

Closed florinmtsc closed 2 years ago

florinmtsc commented 2 years ago

Bug Report

Sharp error: "Cannot find module '.../build/Release/sharp-linux-arm64v8.node'"

Current Behavior

I am running payload cms inside nestjs in a docker image FROM 'node:18'. It used to work but suddenly I am getting this sharp error (I am running on an M1 max processor) - but it used to work...

Expected Behavior

To build correctly

Possible Solution

Not sure, I spent 4 hours on this matter and no success.

Steps to Reproduce

  1. Run payload inside a Dockerfile that has "FROM node:18" on an m1 machine.
  2. Not sure if it matters but I am using payload inside nestjs (again, it used to work...)
  3. You will get sharp library error

Detailed Description

Screenshot 2022-10-27 at 23 30 17
jmikrut commented 2 years ago

Hey @binaryhubio — this is actually an issue with Sharp and Docker rather than Payload. We've run into this before and there are some helpful issues / threads out there that can help you fix this.

https://github.com/lovell/sharp/issues/2875 https://github.com/lovell/sharp/issues/1990 https://github.com/lovell/sharp/issues/2815

I'm going to close this for now but we do have an official docker-compose file that now ships with create-payload-app:

@denolfe can you share that docker-compose file here and then close the issue?

denolfe commented 2 years ago

Here is the working docker-compose.yml. Modify as needed.

version: '3'

services:

  payload:
    image: node:18-alpine
    ports:
      - "3000:3000"
    volumes:
      - .:/home/node/app
      - node_modules:/home/node/app/node_modules
    working_dir: /home/node/app/
    command: sh -c "yarn install && yarn dev"
    depends_on:
      - mongo
    environment:
      MONGODB_URI: mongodb://mongo:27017/payload
      PORT: 3000
      NODE_ENV: development
      PAYLOAD_SECRET: TESTING

  mongo:
    image: mongo:latest
    ports:
      - "27017:27017"
    command:
      - --storageEngine=wiredTiger
    volumes:
      - data:/data/db
    logging:
      driver: none

volumes:
  data:
  node_modules:
omavi commented 1 year ago

Hey @binaryhubio — this is actually an issue with Sharp and Docker rather than Payload. We've run into this before and there are some helpful issues / threads out there that can help you fix this.

lovell/sharp#2875 lovell/sharp#1990 lovell/sharp#2815

I'm going to close this for now but we do have an official docker-compose file that now ships with create-payload-app:

@denolfe can you share that docker-compose file here and then close the issue?

@jmikrut I think the Dockerfile in the create-payload-app could use some improvement to avoid this error. I encountered the same issue and realized the Dockerfile is copying our local node_modules into the image then running npm install instead of npm ci (not sure what would happen with yarn). Instead of dependencies being installed from scratch, the image reused binaries meant for my dev machine.

npm ci did indeed solve this issue but it didn't work with legacy-peer-deps. So I'm hoping https://github.com/payloadcms/payload/issues/2564 is resolved soon in order for me to produce my ideal build. My current workaround is to add a .dockerignore containing **/node_modules.

omavi commented 1 year ago

Confirming the following Dockerfile does solve the issue, without the need for legacy-peer-deps, on Payload 1.8.3:

FROM node:18.8-alpine as base

FROM base as builder

ENV NODE_ENV=build

WORKDIR /home/node/app
COPY package*.json ./

COPY . .
RUN npm ci
RUN npm run build

FROM base as runtime

ENV NODE_ENV=production
ENV PAYLOAD_CONFIG_PATH=dist/payload.config.js

WORKDIR /home/node/app
COPY package*.json  ./

RUN npm ci
COPY --from=builder /home/node/app/dist ./dist
COPY --from=builder /home/node/app/build ./build

EXPOSE 4000

CMD ["node", "dist/server.js"]
Shakib448 commented 1 year ago

@florinmtsc Did you solve the issue?

Jarmos-san commented 12 months ago

Well I'm leaving this comment for anyone in the future (myself included) as a reminder what I just learned (coding definitely doesn't fail to surprise me with something new to learn every other day for sure!).

So, if you pass the --ignore-scripts flag to your npm commands, the sharp builds will fail (see this comment for reference)! Never in all my years of running npm commands did I ever realise the flag's behaviour will cascade down to other dependencies as well.

Regardless, lesson learned and I will now be quite wary of running npm commands like that in the future as well!

tommydangerous commented 11 months ago

I’m getting this in my deployment on Vercel now.

…erateFileData.js
- /var/task/node_modules/payload/dist/collections/operations/create.js
- /var/task/node_modules/payload/dist/collections/operations/local/create.js
- /var/task/node_modules/payload/dist/collections/operations/local/index.js
- /var/task/node_modules/payload/dist/payload.js
- /var/task/.next/server/pages/api/access.js
- /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js
- /var/task/___next_launcher.cjs
    at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
    at /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:11:25599
    at Module._load (node:internal/modules/cjs/loader:922:27)
    at r.<computed>.e._load (/var/task/___vc/__launcher/__launcher.js:14:2516)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at w.require (/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:11:25815)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/var/task/node_modules/payload/dist/uploads/generateFileData.js:16:55)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/var/task/node_modules/payload/dist/uploads/generateFileData.js',
    '/var/task/node_modules/payload/dist/collections/operations/create.js',
    '/var/task/node_modules/payload/dist/collections/operations/local/create.js',
    '/var/task/node_modules/payload/dist/collections/operations/local/index.js',
    '/var/task/node_modules/payload/dist/payload.js',
    '/var/task/.next/server/pages/api/access.js',
    '/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js',
    '/var/task/___next_launcher.cjs'
  ]
}
Error: Cannot find module 'sharp'
Require stack:
- /var/task/node_modules/payload/dist/uploads/generateFileData.js
- /var/task/node_modules/payload/dist/collections/operations/create.js
- /var/task/node_modules/payload/dist/collections/operations/local/create.js
- /var/task/node_modules/payload/dist/collections/operations/local/index.js
- /var/task/node_modules/payload/dist/payload.js
- /var/task/.next/server/pages/api/access.js
- /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js
- /var/task/___next_launcher.cjs
    at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
    at /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:11:25599
    at Module._load (node:internal/modules/cjs/loader:922:27)
    at r.<computed>.e._load (/var/task/___vc/__launcher/__launcher.js:14:2516)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at w.require (/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:11:25815)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/var/task/node_modules/payload/dist/uploads/generateFileData.js:16:55)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/var/task/node_modules/payload/dist/uploads/generateFileData.js',
    '/var/task/node_modules/payload/dist/collections/operations/create.js',
    '/var/task/node_modules/payload/dist/collections/operations/local/create.js',
    '/var/task/node_modules/payload/dist/collections/operations/local/index.js',
    '/var/task/node_modules/payload/dist/payload.js',
    '/var/task/.next/server/pages/api/access.js',
    '/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js',
    '/var/task/___next_launcher.cjs'
  ]
}
Error: Runtime exited with error: exit status 1
Runtime.ExitError
nathanclevenger commented 11 months ago

Yes - I'm seeing the same thing. I can't get any new next-payload projects to deploy on Vercel, even though they run fine locally.

perations/create.js
- /var/task/node_modules/payload/dist/collections/operations/local/create.js
- /var/task/node_modules/payload/dist/collections/operations/local/index.js
- /var/task/node_modules/payload/dist/payload.js
- /var/task/.next/server/pages/api/[collection]/[id].js
- /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js
- /var/task/___next_launcher.cjs
    at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
    at /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:11:25599
    at Module._load (node:internal/modules/cjs/loader:922:27)
    at r.<computed>.e._load (/var/task/___vc/__launcher/__launcher.js:14:2516)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at w.require (/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:11:25815)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/var/task/node_modules/payload/dist/uploads/generateFileData.js:16:55)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/var/task/node_modules/payload/dist/uploads/generateFileData.js',
    '/var/task/node_modules/payload/dist/collections/operations/create.js',
    '/var/task/node_modules/payload/dist/collections/operations/local/create.js',
    '/var/task/node_modules/payload/dist/collections/operations/local/index.js',
    '/var/task/node_modules/payload/dist/payload.js',
    '/var/task/.next/server/pages/api/[collection]/[id].js',
    '/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js',
    '/var/task/___next_launcher.cjs'
  ]
}
Error: Cannot find module 'sharp'
Require stack:
- /var/task/node_modules/payload/dist/uploads/generateFileData.js
- /var/task/node_modules/payload/dist/collections/operations/create.js
- /var/task/node_modules/payload/dist/collections/operations/local/create.js
- /var/task/node_modules/payload/dist/collections/operations/local/index.js
- /var/task/node_modules/payload/dist/payload.js
- /var/task/.next/server/pages/api/[collection]/[id].js
- /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js
- /var/task/___next_launcher.cjs
    at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
    at /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:11:25599
    at Module._load (node:internal/modules/cjs/loader:922:27)
    at r.<computed>.e._load (/var/task/___vc/__launcher/__launcher.js:14:2516)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at w.require (/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js:11:25815)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/var/task/node_modules/payload/dist/uploads/generateFileData.js:16:55)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/var/task/node_modules/payload/dist/uploads/generateFileData.js',
    '/var/task/node_modules/payload/dist/collections/operations/create.js',
    '/var/task/node_modules/payload/dist/collections/operations/local/create.js',
    '/var/task/node_modules/payload/dist/collections/operations/local/index.js',
    '/var/task/node_modules/payload/dist/payload.js',
    '/var/task/.next/server/pages/api/[collection]/[id].js',
    '/var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js',
    '/var/task/___next_launcher.cjs'
  ]
}
Error: Runtime exited with error: exit status 1
Runtime.ExitError
0xPT commented 11 months ago

Same thing! Can we re-open this?

Ajay01103 commented 5 months ago

Same error here while running a build

mxchinegod commented 3 months ago

This is a big problem during build...

zr3 commented 3 months ago

It's not the most efficient solution, but basing on the 18-slim image worked for me:

FROM node:18-slim

WORKDIR /home/node/app

COPY . .
RUN yarn install
RUN yarn build

ENV NODE_ENV=production
ENV PAYLOAD_CONFIG_PATH=dist/payload.config.js

EXPOSE 3000

CMD ["node", "dist/server.js"]

Please take care not to let this scoop up .env files or other secrets in the project folder going this route. I was running into a similar issue while trying to build and deploy this on Fargate -- not using the alpine image + using yarn ended up working well enough for my small, intermittently active project.

I know this issue is closed, but I think it's one of the top search results for the error :sweat_smile:

github-actions[bot] commented 1 month ago

This issue has been automatically locked. Please open a new issue if this issue persists with any additional detail.