Next JS with Sharp in docker container

[albinlang]

Possible bug

Is this a possible bug in a feature of sharp, unrelated to installation?

• [x] Running npm install sharp completes without error.
• [x] Running node -e "require('sharp')" completes without error.

If you cannot confirm both of these, please open an installation issue instead.

Are you using the latest version of sharp?

• [x] I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.

If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

System: OS: Windows 11 10.0.22631 CPU: (16) x64 AMD Ryzen 9 4900HS with Radeon Graphics Memory: 10.61 GB / 31.42 GB Binaries: Node: 18.18.2 - C:\Program Files\nodejs\node.EXE npm: 9.8.1 - C:\Program Files\nodejs\npm.CMD npmPackages: sharp: ^0.33.0 => 0.33.0

The issue

I'm trying to use sharp in my conatainer with Next JS:

My Dockerfile:

FROM node:18-alpine AS base

# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app

# Install dependencies based on the preferred package manager
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
RUN \
  if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
  elif [ -f package-lock.json ]; then npm ci; \
  elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && pnpm i --frozen-lockfile; \
  else echo "Lockfile not found." && exit 1; \
  fi

# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules

COPY . .

# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
ENV NEXT_TELEMETRY_DISABLED 1

RUN yarn build

# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app

ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public

# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 3000

ENV PORT 3000
# set hostname to localhost
ENV HOSTNAME "0.0.0.0"

# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD ["node", "server.js"]

My Package.json

{
  "name": "historiska-vandringar",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "classnames": "^2.3.2",
    "clsx": "^2.0.0",
    "embla-carousel-react": "^8.0.0-rc15",
    "mapbox-gl": "^2.15.0",
    "next": "13.5.6",
    "react": "^18",
    "react-dom": "^18",
    "react-map-gl": "^7.1.6",
    "sharp": "^0.33.0",
    "tailwind-merge": "^2.0.0",
    "zustand": "^4.4.3"
  },
  "devDependencies": {
    "@types/node": "^20",
    "@types/react": "^18",
    "@types/react-dom": "^18",
    "autoprefixer": "^10",
    "eslint": "^8",
    "eslint-config-next": "13.5.6",
    "postcss": "^8",
    "tailwindcss": "^3",
    "typescript": "^5"
  }
}

After running my container on this image I get the following error:

Error: 'sharp' is required to be installed in standalone mode for the image optimization to function correctly. Read more at: https://nextjs.org/docs/messages/sharp-missing-in-production

[lovell]

It looks like you're using yarn v1:

$ docker run -it --rm node:18-alpine yarn --version
1.22.19

Please see https://github.com/lovell/sharp/issues/3871

[albinlang]

@lovell Thank you for a quick response. However I'm not sure how to solve my problem. Would you mind explaining how I can solve this issue in my project? In my mind I use NPM and not yarn except in my Dockerfile on line 30: "RUN yarn build". I've tried replacing this with "RUN npm run build" but I still have the error.

[lovell]

Make sure you're not copying node_modules into the container from outside. Perhaps you could simplify this Dockerfile so it can only use npm e.g. by removing all reference to yarn and pnpm lockfiles.

[albinlang]

I replaced:

COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
RUN \
  if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
  elif [ -f package-lock.json ]; then npm ci; \
  elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && pnpm i --frozen-lockfile; \
  else echo "Lockfile not found." && exit 1; \
  fi

with this:

COPY package.json package-lock.json* ./
RUN npm ci

"Make sure you're not copying node_modules into the container from outside." What do I have to change and to what in order to do this?

[lovell]

The COPY . . directive looks like it might be copying files into the container from outside.

These questions relate to Docker and Node.js rather than being specific to sharp, so might be more suitable for a more general site such as StackOverflow.

[albinlang]

@lovell It's from Next JS example Dockerfile.

[albinlang]

However... locking sharp to version 0.32.6 seems to have solved the problem.

[lovell]

I tried running https://github.com/vercel/next.js/tree/canary/examples/with-docker locally and can reproduce - the problem is due to https://github.com/vercel/nft/issues/371

[albinlang]

I see... so it wasn't just me then. I'm somewhat pleased to hear that 😅 Would you say sticking to version 0.32.6 for now is a stable choice?

[lovell]

The docs you need are here - https://nextjs.org/docs/pages/api-reference/next-config-js/output

Either switch off tracing or specify outputFileTracingIncludes - see https://github.com/lovell/sharp/issues/3870#issuecomment-1838024154

I'll close this as there's already one sharp issue tracking this and there's nothing else we can do.

[99lalo]

However... locking sharp to version 0.32.6 seems to have solved the problem.

Where did you install sharp? I tried locking it on dependencies and i'm still getting the same issues on my container. I'm also using the nextjs official dockerfile example.

FROM node:20.10-alpine AS base

# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app

# Install dependencies based on the preferred package manager
COPY package.json package-lock.json* ./
RUN npm ci

# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=deps /root/.npm /root/.npm
COPY . .

# Generate Prisma Files
RUN npx prisma generate

ENV NEXT_SHARP_PATH=/tmp/node_modules/sharp

RUN npm run build

# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app

ENV NODE_ENV production
ENV NEXT_SHARP_PATH=/tmp/node_modules/sharp

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public

# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
COPY --from=builder --chown=nextjs:nodejs /app/prisma ./prisma

USER nextjs

EXPOSE 3000

ENV PORT 3000
# set hostname to localhost
ENV HOSTNAME "0.0.0.0"

# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD ["npm", "run", "start:migrate:prod"]

Package.json

{
  "name": "gamefinder",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "start:migrate:prod": "npx prisma migrate deploy && node server.js",
    "lint": "next lint",
    "migrate": "npx prisma migrate",
    "migrate:deploy": "npx prisma migrate",
    "prisma:seed": "npx prisma db seed",
    "docker:compose:up": "docker-compose up",
    "docker:compose:down": "docker-compose down",
    "docker:compose:restart": "npm run docker:compose:down && npm run docker:compose:up"
  },
  "prisma": {
    "seed": "ts-node --compiler-options {\"module\":\"CommonJS\"} prisma/seed.ts"
  },
  "engines": {
    "node": "20.10"
  },
  "dependencies": {
    "@auth/prisma-adapter": "^1.0.4",
    "@emotion/react": "^11.11.1",
    "@emotion/styled": "^11.11.0",
    "@mui/icons-material": "^5.14.14",
    "@mui/material": "^5.14.14",
    "@prisma/client": "^5.7.0",
    "axios": "^1.6.1",
    "bcrypt": "^5.1.1",
    "moment": "^2.29.4",
    "next": "^14.0.2",
    "next-auth": "^4.24.5",
    "react": "^18",
    "react-dom": "^18",
    "react-slick": "^0.29.0",
    "sharp": "0.32.6",
    "slick-carousel": "^1.8.1"
  },
  "devDependencies": {
    "@types/bcrypt": "^5.0.2",
    "@types/node": "^20.9.0",
    "@types/react": "^18",
    "@types/react-dom": "^18",
    "@types/react-slick": "^0.23.12",
    "eslint": "^8",
    "eslint-config-next": "13.5.6",
    "prisma": "^5.7.0",
    "sass": "^1.69.4",
    "ts-node": "^10.9.1",
    "typescript": "^5.2.2"
  }
}

[david7378]

Same problem here...

[deadcoder0904]

@99lalo did you solve this problem?

i'm kinda doing the same thing & have already installed sharp in package.json but docker container can't find it. it is probably bcz my node_modules in production doesn't include sharp as it strips imports that are not used.

[b1rdex]

@deadcoder0904

# using numeric uid/gid becuase of buildx bug (chmod+link can't use names for chmod): https://github.com/docker/buildx/issues/1408
COPY --from=builder --chown=1001:1001 --link /app/.next/standalone ./

# workaround for next.js standalone build bug
COPY --from=builder --chown=1001:1001 --link /app/node_modules/@img ./node_modules/@img
COPY --from=uploader --chown=1001:1001 --link /app/.next/static ./.next/static

So basically you just need to manually copy node_modules/@img to the docker image

[deadcoder0904]

@b1rdex what is @img?

i got an error:

failed to solve: failed to compute cache key: failed to calculate checksum of ref c6860692-8a86-49a4-a4d4-a83a55e135db::i2g9a4bzelnitjlbi179s2w6a: "/app/node_modules/@img": not found

[srigi]

I found a root cause in my case: It was a discrepancy between dependencies of @img on my developer machine (macOS - darwin) and in the container (node:20-alpine - linux).

I'm kinda surprised that no one mentioned the binary compatibility of dependencies. By looking at Sharp's package.json the error starts becoming evident. My developer workflow persisted the darwin variants of @img into the lock file. What I need for container is a linux and linuxmusl.

The fix is super easy: do the proper dependencies installation inside the Docker with the target platform. Today's package managers are clever enough to install the correct binary variant, even if the lock file is followed.

[lovell]

@srigi

My developer workflow persisted the darwin variants of @img into the lock file. What I need for container is...

If you're using npm this is probably due to https://github.com/npm/cli/issues/4828

Please see https://sharp.pixelplumbing.com/install#cross-platform for more information if you are attempting to share lockfiles and node_modules directories across multiple platforms.

For anyone else reading this and still having problems, please note there were bugs in previous versions of Next.js relating to its "standalone" build feature that prevented some dependencies from being included. Please ensure you are using the latest version of all your dependencies, including both Next.js and sharp. Finally please avoid yarn v1 and its lockfiles.