vercel / turborepo

Build system optimized for JavaScript and TypeScript, written in Rust
https://turbo.build/repo/docs
MIT License
26.45k stars 1.84k forks source link

[turborepo] Using turbo prune via Dockerfile doesn't generate turbo out folder #6344

Open erkand-imeri opened 1 year ago

erkand-imeri commented 1 year ago

What version of Turborepo are you using?

1.9.3

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Mac

Describe the Bug

Hello,

I am in the process of migrating from Yarn to PNPM and have written a generic Dockerfile to reuse for Next.js applications. I am facing an issue when running the project via Docker, where the turbo out folder is not being generated. Based on my understanding, this should happen when we run the following command:

RUN turbo prune ${APP_NAME} --docker
ARG NODE=bitnami/node:20.9.0

FROM ${NODE} as builder
WORKDIR /app
ARG APP_NAME
COPY . .
RUN npm install -g pnpm
RUN npm install -g turbo
RUN echo $(turbo --version)
RUN turbo prune ${APP_NAME} --docker

FROM builder as installer
ARG CACHE_CLEAN=true
ARG APP_NAME
WORKDIR /app/apps/${APP_NAME}
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
RUN pnpm install && \
    if [ "$CACHE_CLEAN" = "true" ]; then pnpm store prune; fi
ENV PATH /app/apps/${APP_NAME}/node_modules/.bin:$PATH

# Build the project and its dependencies
COPY --from=builder /app/out/full/ .
COPY turbo.json turbo.json
ENV NEXT_TELEMETRY_DISABLED 1
RUN turbo run build --filter=${APP_NAME}...

FROM installer as production
ARG APP_NAME
ARG PORT
ENV NEXT_TELEMETRY_DISABLED 1
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

COPY --from=installer /app/apps/${APP_NAME}/ ./
USER nextjs
EXPOSE ${PORT}
ENV PORT ${PORT}

CMD node --max-old-space-size=200 --optimize_for_size --gc_interval=100 server.js

For local development, I run it via docker-compose and pass the APP_NAME variable like this:

app-1:
    container_name: app1
    build:
      context: .
      dockerfile: ./tools/docker/nextjs/Dockerfile
      target: installer
      args:
        APP_NAME: app1
        PORT: 3000
        CACHE_CLEAN: "false"
    working_dir: /app
    restart: always
    environment:
      - NODE_ENV=development
    volumes:
      - .:/app
    ports:
      - "3000:3000"
    command: pnpm --filter app1 run dev

Here is my turbo.json configuration:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "outputs": ["dist/**", ".next/**", "!.next/cache/**", "public/dist/**"],
      "dependsOn": ["^build"]
    },
    "test": {
      "outputs": ["coverage/**"],
      "dependsOn": []
    },
    "lint": {
      "dependsOn": ["^build"]
    },
    "dev": {
      "cache": false,
      "persistent": true
    },
    "clean": {
      "cache": false
    }
  }
}

Any help or guidance on resolving this issue would be greatly appreciated.

Expected Behavior

out folder should be generated on the root of monorepo

To Reproduce

Build the docker image and run the container.

Reproduction Repo

No response

chris-olszewski commented 1 year ago

Hi @erkand-imeri, did the issue only pop up once you switched from Yarn to PNPM?

Can you verify that /app/out isn't being populated at all by just checking it's existence right after the RUN turbo prune ${APP_NAME} --docker? If we're failing to populate it we should be producing some error text which would also be helpful to have.

tknickman commented 1 year ago

Hey @erkand-imeri I tried to isolate this using our create-turbo starter, and with the following docker details:

Dockerfile:

ARG NODE=bitnami/node:20.9.0

FROM ${NODE} as builder
WORKDIR /app
ARG APP_NAME
COPY . .
RUN npm install -g pnpm
RUN npm install -g turbo
RUN echo $(turbo --version)
RUN turbo prune ${APP_NAME} --docker

Build the container:

docker build --build-arg APP_NAME=docs -t prune-test .

Running the container:

docker run -i -t prune-test /bin/bash

I see the out directory as expected:

root@97ddab231bef /app/out git:(main) ±4
ls
full  json  pnpm-lock.yaml  pnpm-workspace.yaml

Can you try the above and share the error you are getting? That might shed some additional light on where the problem is.

erkand-imeri commented 1 year ago

Hey @chris-olszewski , @tknickman , thanks for the comments, my current Dockerfile looks like this:

ARG NODE=bitnami/node:20.9.0

# BUILDER STAGE, TURBO DOCKER PRUNE
FROM ${NODE} as builder
WORKDIR /app
ARG APP_NAME
RUN npm install -g pnpm && \
    npm install -g turbo
COPY . .
RUN turbo prune ${APP_NAME} --docker

# INSTALL DEPENDENCIES STAGE
FROM builder as installer
ARG CACHE_CLEAN=true
ARG APP_NAME
WORKDIR /app
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
RUN pnpm --filter ${APP_NAME} install
# Build the project and its dependencies
COPY --from=builder /app/out/full/ .
COPY turbo.json turbo.json
RUN turbo run build --filter=${APP_NAME}

# PRODUCTION DEPLOYMENT STAGE
FROM ${NODE} as production_runner
ARG APP_NAME
ARG PORT
ENV NEXT_TELEMETRY_DISABLED 1
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

# Copy only the necessary app files for runtime
COPY --from=installer /app/apps/${APP_NAME}/next.config.js ./
COPY --from=installer /app/apps/${APP_NAME}/package.json ./package.json
COPY --from=installer /app/apps/${APP_NAME}/.next/standalone ./
COPY --from=installer /app/apps/${APP_NAME}/.next/static ./

USER nextjs
# When using sidecar/istio-proxy you need to tweak --max-old-space-size and --gc_interval for optimal memory management, usually at the moment
# one istio-proxy is taking 150Mi, when cloudops allow us to create sidecar objects we need to define namespaces from whom we use service mesh,
# that will reduce the memory footprint of istio-proxy, since currently one pod app starts with 20-25Mi while one istio-proxy pod starts with 100-150Mi
# That 100-150Mi should be taken in consideration when defining K8 resource limits.
# Usually in node.js you define 70-80% of what resource memory limit you define in Kubernetes, 
# but that's not the golden rule (tweak lower or higher values for optimal management), also substract 150Mi (istio-proxy memory) from that 70-80%. 
CMD node --max-old-space-size=200 --optimize_for_size --gc_interval=100 server.js

but i am facing this error

Error: Cannot find module 'next/dist/server/next-server'
Require stack:
- /app/server.js
 at Module._resolveFilename (node:internal/modules/cjs/loader:1048:15)
 at Module._load (node:internal/modules/cjs/loader:901:27)
 at Module.require (node:internal/modules/cjs/loader:1115:19)
at require (node:internal/modules/helpers:130:18)
 at Object.<anonymous> (/app/server.js:2:20)
at Module._compile (node:internal/modules/cjs/loader:1241:14)
 at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
at Module.load (node:internal/modules/cjs/loader:1091:32)
 at Module._load (node:internal/modules/cjs/loader:938:12)
 at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [ '/app/server.js' ]

Trying to figure out why is this the case, am i missing something when i copy the folders.

My .dockerignore looks like this

# Dependency directories
node_modules

# Log files
yarn-error.log
*.log

# Version control
.git

# System files
.DS_Store
Thumbs.db

# Editor directories
.idea/
.vscode/
*.sublime-workspace

# Build outputs
dist/
build/
out/
.coverage/

# Documentation & Tests
*.md
LICENSE
test/
tests/
__tests__/

# Environment configs
.env*
!.env.production

.pnpm-store

AFAIK, .next/standalone doesn't require anything, it bundles the Next.js app and makes it ready for running.

To be clear, i can run this locally fine, the issue is the production_runner stage (which i use it in staging/production Kubernetes cluster deployment, while locally i target up to installer stage), when i try the .next/standalone.

app1:
    container_name: app1
    build:
      context: .
      dockerfile: ./tools/docker/nextjs/Dockerfile
      target: installer
      args:
        APP_NAME: app1
        PORT: 3003
        CACHE_CLEAN: "false"
    working_dir: /app
    restart: always
    environment:
      - NODE_ENV=development
    volumes:
      - .:/app
      - "pnpm-store:/root/.pnpm-store"
    ports:
      - "3003:3003"
    command: pnpm --filter app1 run dev
erkand-imeri commented 1 year ago

Ok, so if i add in production_runner stage

FROM ${NODE} as production_runner
ARG APP_NAME
ARG PORT
ENV NEXT_TELEMETRY_DISABLED 1
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

# Copy only the necessary app files for runtime
COPY --from=installer /app/apps/${APP_NAME}/next.config.js ./
COPY --from=installer /app/apps/${APP_NAME}/package.json ./package.json
COPY --from=installer /app/apps/${APP_NAME}/.next/standalone ./
COPY --from=installer /app/apps/${APP_NAME}/.next/static ./
RUN npm install -g pnpm && pnpm install --prod 

USER nextjs

CMD node --max-old-space-size=200 --optimize_for_size --gc_interval=100 server.js
RUN npm install -g pnpm && pnpm install --prod 

It doesn't show the error, but yet nothing is up and running. IDK, something is wrong with the way production_runner copies the folders from installer, it doesn't sum it up. I guess i am missing something, also pnpm install shouldn't be necessary at all, but i added just for debugging purposes.

wigsnes commented 1 year ago

I am having a similar issues when I try to install using yarn 4.0.1. But I am missing next, and you next-server. Not sure if it is related.

2023-11-07 12:52:00 node:internal/modules/cjs/loader:1051
2023-11-07 12:52:00   throw err;
2023-11-07 12:52:00   ^
2023-11-07 12:52:00 
2023-11-07 12:52:00 Error: Cannot find module 'next'
2023-11-07 12:52:00 Require stack:
2023-11-07 12:52:00 - /app/apps/pio-app/server.js
2023-11-07 12:52:00     at Module._resolveFilename (node:internal/modules/cjs/loader:1048:15)
2023-11-07 12:52:00     at Module._load (node:internal/modules/cjs/loader:901:27)
2023-11-07 12:52:00     at Module.require (node:internal/modules/cjs/loader:1115:19)
2023-11-07 12:52:00     at require (node:internal/modules/helpers:130:18)
2023-11-07 12:52:00     at Object.<anonymous> (/app/apps/pio-app/server.js:23:1)
2023-11-07 12:52:00     at Module._compile (node:internal/modules/cjs/loader:1241:14)
2023-11-07 12:52:00     at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
2023-11-07 12:52:00     at Module.load (node:internal/modules/cjs/loader:1091:32)
2023-11-07 12:52:00     at Module._load (node:internal/modules/cjs/loader:938:12)
2023-11-07 12:52:00     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12) {
2023-11-07 12:52:00   code: 'MODULE_NOT_FOUND',
2023-11-07 12:52:00   requireStack: [ '/app/apps/pio-app/server.js' ]
2023-11-07 12:52:00 }
2023-11-07 12:52:00 
2023-11-07 12:52:00 Node.js v20.9.0

I have gotten it to work with pnpm, this might help you:

FROM node:20-alpine AS base

RUN npm install -g pnpm
RUN npm install -g turbo

FROM base AS builder
RUN apk add --no-cache libc6-compat
RUN apk update

# Set working directory
WORKDIR /app
COPY . .
RUN turbo prune @pio/app --docker

FROM base AS installer
RUN apk add --no-cache libc6-compat
RUN apk update
WORKDIR /app

COPY turbo.json turbo.json
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
RUN pnpm install

COPY --from=builder /app/out/full/ .
RUN turbo run build --filter=@pio/app...

FROM base AS runner
WORKDIR /app

ENV NODE_ENV production

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
USER nextjs

COPY --from=installer /app/apps/pio-app/next.config.js .
COPY --from=installer /app/apps/pio-app/package.json .

COPY --from=installer --chown=nextjs:nodejs /app/apps/pio-app/.next/standalone ./
COPY --from=installer --chown=nextjs:nodejs /app/apps/pio-app/.next/static ./apps/pio-app/.next/static
COPY --from=installer --chown=nextjs:nodejs /app/apps/pio-app/public ./apps/pio-app/public

EXPOSE 3000

ENV NEXT_TELEMETRY_DISABLED 1

CMD ["node", "apps/pio-app/server.js"]

Feels like turbo is not made for yarn berry,

erkand-imeri commented 1 year ago

Hey @wigsnes i will try your pnpm Dockerfile, thanks.

A question, why you prune @pio/app like this when your app folder name is apps/pio-app. Is there any labeling being done in pnpm-workspace.yaml?

Also, what i am trying to achieve is in addition to not copy all monorepo apps code into the pod except for the target one. Since i want to deploy them separately, as separate deployments and services but sharing the same repository.

The way i see, how you copied files and folders, it might work as you say, but i cannot make sense of it, how you pass the standalone in current working dir and you execute the server.js in apps/pio-app. I will give it a shot and notify you. Thanks.

wigsnes commented 1 year ago

@pio/app is the name in our package.json. I assumed it did not matter if you use the folder name or workspace name. I will check.

Seems to only work with @pio/app that is the name given in the package.json file for that app. No labeling is being done just what it is called.

Works for me using pnpm, but if I try to do the same with yarn berry it fails as stated above.

erkand-imeri commented 1 year ago

Well, i could not make it work with your example.

I have this error

Error: Cannot find module '/app/apps/app1/server.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1048:15)
    at Module._load (node:internal/modules/cjs/loader:901:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
    at node:internal/main/run_main_module:23:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

@wigsnes can you show me what do you have in .dockerignore, do you put node_modules or no there?

wigsnes commented 1 year ago

My .dockerignore

node_modules
**/node_modules
.git
.gitignore
*.md
dist
out

do you see server.js in your folder? Do you also call you folder apps?

My folder structure:

monorepo
    apps
        pio-app
            package.json with name @pio/app
    packages
        ui/
        types/
    Dockerfile
    package.json
    .dockerignore

My pnpm-workspace.yaml file:

packages:
  - 'packages/*'
  - 'apps/*'
erkand-imeri commented 1 year ago

Here is my Dockerfile mimicking yours.

ARG NODE=bitnami/node:20.9.0

# BUILDER STAGE, TURBO DOCKER PRUNE
FROM ${NODE} as builder
WORKDIR /app
ARG APP_NAME
RUN npm install -g pnpm && \
    npm install -g turbo
COPY . .
RUN turbo prune ${APP_NAME} --docker

# INSTALL DEPENDENCIES STAGE
FROM builder as installer
ARG CACHE_CLEAN=true
ARG APP_NAME
WORKDIR /app

COPY turbo.json turbo.json
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
RUN pnpm install

COPY --from=builder /app/out/full/ .
RUN turbo run build --filter=${APP_NAME}

# PRODUCTION DEPLOYMENT STAGE
FROM installer as production_runner
ARG APP_NAME
ENV NEXT_TELEMETRY_DISABLED 1
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

# Copy only the necessary app files for runtime
COPY --from=installer /app/apps/${APP_NAME}/next.config.js .
COPY --from=installer /app/apps/${APP_NAME}/package.json .
COPY --from=installer /app/apps/${APP_NAME}/.next/standalone ./
COPY --from=installer /app/apps/${APP_NAME}/.next/static ./apps/${APP_NAME}/.next/static

USER nextjs

CMD node --max-old-space-size=200 --optimize_for_size --gc_interval=100 apps/${APP_NAME}/server.js

.dockerignore

# Dependency directories
node_modules
**/node_modules

# Log files
yarn-error.log
*.log

# Version control
.git

# System files
.DS_Store
Thumbs.db

# Editor directories
.idea/
.vscode/
*.sublime-workspace

# Build outputs
dist/
build/
out/
.coverage/

# Documentation & Tests
*.md
LICENSE
test/
tests/
__tests__/

# Environment configs
.env*
!.env.production

.pnpm-store
monorepo
    apps
        app1
            package.json with name app1
    packages
        ui/
        types/
   tools/docker/nextjs/Dockerfile
    package.json
    .dockerignore
   pnpm-workspace.yaml

So, technically the same, but it fails. Interesting.

wigsnes commented 1 year ago

Do you see server.js in your files on docker when you run the app?

image
erkand-imeri commented 1 year ago

It crashes, so cannot see the container file structure, i tried with this

# This is a generic Dockerfile for all apps which uses variable names and based on the execution context it takes the context of the apps
# APP_NAME and PORT locally are taken from docker-compose.yaml and in pipeline it's passed via the yaml file 
# of the repository host/DevSecOps like Giltab with .gitlab-ci.yaml
ARG NODE=bitnami/node:20.9.0

# BUILDER STAGE, TURBO DOCKER PRUNE
FROM ${NODE} as builder
WORKDIR /app
ARG APP_NAME
RUN npm install -g pnpm && \
    npm install -g turbo
COPY . .
RUN turbo prune ${APP_NAME} --docker

# INSTALL DEPENDENCIES STAGE
FROM builder as installer
ARG CACHE_CLEAN=true
ARG APP_NAME
WORKDIR /app

COPY turbo.json turbo.json
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
RUN pnpm install

COPY --from=builder /app/out/full/ .
RUN turbo run build --filter=${APP_NAME}

RUN mkdir -p runtime/.next/static && \
    mv apps/${APP_NAME}/next.config.js runtime/ && \
    mv apps/${APP_NAME}/package.json runtime/ && \
    mv apps/${APP_NAME}/.next/standalone/* runtime/ && \
    mv apps/${APP_NAME}/.next/static/* runtime/.next/static

# PRODUCTION DEPLOYMENT STAGE
FROM ${NODE} as production_runner
ARG APP_NAME
ENV NEXT_TELEMETRY_DISABLED 1
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

# Copy only the necessary app files for runtime
COPY --from=installer /app/runtime/ ./

USER nextjs
# When using sidecar/istio-proxy you need to tweak --max-old-space-size and --gc_interval for optimal memory management, usually at the moment
# one istio-proxy is taking 150Mi, when cloudops allow us to create sidecar objects we need to define namespaces from whom we use service mesh,
# that will reduce the memory footprint of istio-proxy, since currently one pod app starts with 20-25Mi while one istio-proxy pod starts with 100-150Mi
# That 100-150Mi should be taken in consideration when defining K8 resource limits.
# Usually in node.js you define 70-80% of what resource memory limit you define in Kubernetes, 
# but that's not the golden rule (tweak lower or higher values for optimal management), also substract 150Mi (istio-proxy memory) from that 70-80%. 
CMD node --max-old-space-size=200 --optimize_for_size --gc_interval=100 server.js

I see server.js node_modules and all neccessary files for standalone but the container inside the pod keeps crashing and the pod is not healthy.

wigsnes commented 1 year ago

ok, I am a bit lost then. At what line does it crash?

My versions are:

"turbo": "^1.10.16",
"next": "^13.5.6",

Do you still have this error?

Error: Cannot find module '/app/apps/app1/server.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1048:15)
    at Module._load (node:internal/modules/cjs/loader:901:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
    at node:internal/main/run_main_module:23:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}
erkand-imeri commented 1 year ago

So, my version is

"turbo": "^1.9.3", "next": "13.1.5",

For now, i decided to use this Dockerfile which works perfectly fine for now, and not utilizing turbo for building the app

# This is a generic Dockerfile for all apps which uses variable names and based on the execution context it takes the context of the apps
# APP_NAME and PORT locally are taken from docker-compose.yaml and in pipeline it's passed via the yaml file 
# of the repository host/DevSecOps like Giltab with .gitlab-ci.yaml
ARG NODE=bitnami/node:20.9.0

# BUILDER STAGE
FROM ${NODE} as installer
WORKDIR /app
ARG APP_NAME
COPY apps/${APP_NAME}/ ./
RUN npm install -g pnpm && pnpm install

# INSTALL DEPENDENCIES STAGE
FROM installer as builder
WORKDIR /app
RUN pnpm run build

# PRODUCTION DEPLOYMENT STAGE
FROM ${NODE} as runner_production
WORKDIR /app
ARG PORT
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/package.json ./package.json
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

EXPOSE ${PORT}

ENV PORT ${PORT}

USER nextjs
# When using sidecar/istio-proxy you need to tweak --max-old-space-size and --gc_interval for optimal memory management, usually at the moment
# one istio-proxy is taking 150Mi, when cloudops allow us to create sidecar objects we need to define namespaces from whom we use service mesh,
# that will reduce the memory footprint of istio-proxy, since currently one pod app starts with 20-25Mi while one istio-proxy pod starts with 100-150Mi
# That 100-150Mi should be taken in consideration when defining K8 resource limits.
# Usually in node.js you define 70-80% of what resource memory limit you define in Kubernetes, 
# but that's not the golden rule (tweak lower or higher values for optimal management), also substract 150Mi (istio-proxy memory) from that 70-80%. 
CMD node --max-old-space-size=200 --optimize_for_size --gc_interval=100 server.js
CodeGetters commented 10 months ago

hi,I was also trying to use pnpm+turbo+Nextjs and I also ran into this problem (server.js not found). I built it using the official example (yarn+turbo) and found that yarn can generate both server.js and standalone folders. But using pnpm does not.😥

CodeGetters commented 10 months ago

hi,I was also trying to use pnpm+turbo+Nextjs and I also ran into this problem (server.js not found). I built it using the official example (yarn+turbo) and found that yarn can generate both server.js and standalone folders. But using pnpm does not.😥

I'm using :

wigsnes commented 9 months ago

We ended up going back to using yarn

Dockerfile:

# Install dependencies only when needed
FROM node:20-alpine AS deps
ENV YARN_VERSION 4.0.2

RUN corepack enable && corepack prepare yarn@${YARN_VERSION}
...
CodeGetters commented 9 months ago

We ended up going back to using yarn

Dockerfile:

# Install dependencies only when needed
FROM node:20-alpine AS deps
ENV YARN_VERSION 4.0.2

RUN corepack enable && corepack prepare yarn@${YARN_VERSION}
...

Here is my dockerfile if you need it, it will run successfully on windows🥰, but there may be some issues on mac.😥

This file is located under ./apps/blog

FROM node:20-alpine AS base
RUN corepack enable && corepack prepare pnpm@latest --activate
RUN npm i -g turbo

# -------------------------- stage builder ---------------------------------
# Add c library and update version
FROM base AS builder
RUN apk add --no-cache libc6-compat
RUN apk update

# Set working directory /app
WORKDIR /app
COPY . .
RUN turbo prune blog --docker

# -------------------------- stage installer ---------------------------------
# Add lockfile and package.json's of isolated subworkspace
FROM base AS installer
RUN apk add --no-cache libc6-compat
RUN apk update

WORKDIR /app
# First install the dependencies (as they change less often)
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml

# Build the project
COPY --from=builder /app/out/full/ .
COPY turbo.json turbo.json
RUN pnpm install
RUN pnpm turbo build --filter=blog...

# -------------------------- stage runner ---------------------------------
FROM base AS runner
WORKDIR /app

# Don't run production as root
# 添加一个名为 nodejs 的系统用户组,gid 为 1001
RUN addgroup --system --gid 1001 nodejs
# 添加一个名为 nextjs 的系统用户,uid 为 1001
RUN adduser --system --uid 1001 nextjs
# 切换用户为 nextjs
USER nextjs

COPY --from=installer /app/apps/blog/next.config.js .
COPY --from=installer /app/apps/blog/package.json .

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=installer --chown=nextjs:nodejs /app/apps/blog/.next/standalone ./
COPY --from=installer --chown=nextjs:nodejs /app/apps/blog/.next/static ./apps/blog/.next/static
COPY --from=installer --chown=nextjs:nodejs /app/apps/blog/public ./apps/blog/public

CMD node apps/blog/server.js

The file is located in the root directory(docker-compose.yml)

version: "3"

services:
  blog:
    container_name: blog
    build:
      context: .
      dockerfile: ./apps/blog/Dockerfile
    restart: always
    ports:
      - 3000:3000
einaralex commented 5 months ago

Thankfully, because of this thread, I now have a working Dockerfile for a Next app in Turborepo using PNPM.

I ran into a few issues on the way:

  1. Missing packages. First time running pnpm install I got a bunch of errors, this was due to some node dependencies depending on Python.

    - RUN apk add --no-cache libc6-compat
    + RUN apk add --no-cache libc6-compat python3 py3-pip make g++
  2. Not copying over pnpm-workspace.yaml

    COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
    + COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml
  3. Move the install below /out/full copy. I got an error that I was missing a dependecy from an internal package my web app is using.

    
    - RUN pnpm install
    -
    - COPY --from=builder /app/out/full/ .
  1. Next.config configuration Make sure your next.config includes the standalone output:
    
    // filename: next.config.mjs

/* @type {import('next').NextConfig} / import path from 'path'; import { fileURLToPath } from 'url';

const __dirname = path.dirname(fileURLToPath(import.meta.url));

const nextConfig = {

export default nextConfig;


## Working Dockerfile

```dockerfile
FROM node:20-alpine AS base

RUN corepack enable && corepack prepare pnpm@latest --activate
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN pnpm add -g turbo@^2

FROM base AS builder
WORKDIR /app
COPY . .

# Generate a partial monorepo with a pruned lockfile for a target workspace.
RUN turbo prune customer --docker

FROM base AS installer
RUN apk add --no-cache libc6-compat python3 py3-pip make g++
RUN apk update

WORKDIR /app

COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml
COPY --from=builder /app/out/full/ .

RUN pnpm install
RUN pnpm turbo run build --filter=customer...

FROM base AS runner
WORKDIR /app

# Don't run production as root
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
USER nextjs

# NOTE: I'm using `.mjs`
COPY --from=installer /app/apps/customer/next.config.mjs .
COPY --from=installer /app/apps/customer/package.json .

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=installer --chown=nextjs:nodejs /app/apps/customer/.next/standalone ./
COPY --from=installer --chown=nextjs:nodejs /app/apps/customer/.next/static ./apps/customer/.next/static
COPY --from=installer --chown=nextjs:nodejs /app/apps/customer/public ./apps/customer/public

EXPOSE 3000

ENV PORT 3000
CMD HOSTNAME="0.0.0.0" node apps/customer/server.js
docker build -f apps/customer/Dockerfile -t customer .
docker run -it -p 3000:3000 customer
andresgutgon commented 4 months ago

@einaralex your Dockerfile made mine at least build successfully 🎉

But dependencies inside the image in apps/web/node_modules point to broken symlinks image

Those pnpm folders does not exists

Looks like I'm missing passing node_modules to the last stage some how?

andresgutgon commented 4 months ago

I manage to make it run adding prop-deps

+FROM installer AS prod-deps
+WORKDIR /app
+RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --prod --frozen-lockfile

Then

-FROM base AS runner
+FROM base AS prod-runner

+COPY --from=prod-deps --chown=nextjs:nodejs /app/node_modules ./node_modules
+COPY --from=prod-deps --chown=nextjs:nodejs /app/apps/web/package.json ./apps/web/package.json