Dockerfile optimization and issues

[BraianPita]

Link to reproduction

No response

Describe the Bug

I have been working on a website solution using the website template on the repository, in which I built a containerized version of the website template, so I could more easily host it and deploy it. However, I have realized that the current Dockerfile is lacking in a lot of areas I had to fix for it to work properly. First, I had many issues with yarn and ended up replacing the yarn commands with "npm run build" and the "node dist/server.js" with "npm run serve".

I also have to mention that in my experience, there are many missing files on the dockerfile runner image created from the default dockerfile. I had to include both the ".next", the "nextjs.config.js", and its associated "csp.js" and "redirects.js" into my final docker image when copying from the builder image.

Finally, it is recommended that we change the order of some of the commands on the dockerfile for better caching of layers during build time, which significantly reduces build time during development, for example: As it is, any change to any file will change the state of the image before the "yarn install" (npm ci in my case) is run. If the COPY . . command is moved after yarn install we can ensure that the image state will remain identical unless the package or yarn.lock files are modified, which allows you to basically skip those steps unless you change the package*.json yarn.lock. I also made sure to create an env variable ENV BUILD_DOCKER_IMAGE=true to avoid unnecessary issues during development vs build. I needed to add some extra logic to the payload.config.js file to be able to build my images without having a connection to the database, using something like this:

await payload.init({
    ...(process.env.BUILD_DOCKER_IMAGE == 'true' ? {disableDBConnect: true} : {}),
   secret: process.env.PAYLOAD_SECRET,
   //... etc.
})

To Reproduce

Try to run the docker build command to create a standalone docker image for the website template.

Payload Version

2.0.7 (website template)

Adapters and Plugins

website template

[adamczykjac]

@BraianPita did you have issues with the lack of PAYLOAD_SECRET env var in the builder phase?

[BraianPita]

@BraianPita did you have issues with the lack of PAYLOAD_SECRET env var in the builder phase?

I don't think that was ever an issue for me. I did add a step in which I copy a '.env.production' file into '.env' before building though since I need some of those variables like the next_public ones that get hard coded during build time for the client side. I was working on removing my use of the 'NEXT_PUBLIC_SERVER_URL' as much as possible for this reason, since once built it can't be changed sadly

[adamczykjac]

would you mind sharing a fork @BraianPita ? I'm struggling with a bunch of stuff at RUN yarn build, which you seem to have already resolved.

[BraianPita]

would you mind sharing a fork @BraianPita ? I'm struggling with a bunch of stuff at RUN yarn build, which you seem to have already resolved.

I'm not at my computer right now but once I am I can set up a working example. For now what comes to mind is that I also had many issues with yarn and ended up switching to 'npm ci' and 'npm ci --omit=dev'

Another thing that comes to mind was that I needed to copy many other files from builder to the runtime (next.config.js + csp.js + redirects.js, also the .next folder, and I think that's it but might be missing something). Another thing is that 'npm run serve' seemed to give me less problems than directly calling 'node dist/serve.js'.

Let me know if any of this helps, otherwise I can look at my current working Dockerfile tomorrow

[BraianPita]

I used my current Dockerfile as reference, here are some suggestions:

FROM node:18.8-alpine as base

FROM base as builder

WORKDIR /home/node/app

COPY package*.json ./
RUN npm ci

# Only run the copy all files command AFTER `npm ci` for better cache
COPY . .
RUN npm run build

FROM base as runtime

# Not needed at it is set by the `npm run serve` command
# ENV NODE_ENV=production

# The config path was wrong in my case, this is the correct one
ENV PAYLOAD_CONFIG_PATH=dist/payload/payload.config.js

# I set this to true when I build a new image and use it to make sure the database is set to not connect as explained on my issue
ENV BUILD_DOCKER_IMAGE=false

WORKDIR /home/node/app
COPY package*.json  ./

RUN npm ci --omit=dev

# missing parts to copy, honestly on mine I just copy the whole repo again
COPY --from=builder /home/node/app/dist ./dist
COPY --from=builder /home/node/app/build ./build
COPY --from=builder /home/node/app/.next ./.next
COPY --from=builder /home/node/app/public ./public

# We also need next.config.js, csp.js, redirects.js ... (At least I was having issues without it) If you still have issues, I would try copying all files to the runtime image (COPY . .)
COPY *.js .

EXPOSE 3000

# This command was giving me issues
#CMD ["node", "dist/server.js"]

# I used this instead
CMD ["npm", "run", "serve"]

My actual dockerfile actually ended up having a single image (runtime) and I did not bother with npm ci --omit=dev since it was adding too much extra time to my development build when I was testing my Dockerfile, but this should work fine. If not, you can try also removing the builder stage.

[ExaltedJim]

Hey @BraianPita - trying to do something similar. What did you have to do to prevent connection to database during image build? I'm not quite sure what needs to change for this to happen, so any pointers appreciated, thanks!