Add recipe for Docker image #44

Open Skn0tt opened 4 years ago

What do you want and why?

There should be an easy way to package Blitz applications in a Docker image. Therefore, It'd be great to have some kind of well-thought-out, fully working Dockerfile available to users.

Possible implementation(s)

The Dockerfile could either be part of the blitz new template or be generated using installers.

Additional context

Some more thoughts:

The Dockerfile needs to reside in the root folder. It's a convention that enables things like GitLab's AutoDevOps to work "magically".
Obviously, we'd need documentation for this.

this is a possible solution that uses a multistep build to lower the final image size by only take dependencies necessary to run a production server from node_modules

FROM node:alpine as builder
RUN apk add --no-cache libc6-compat
WORKDIR /app

COPY package.json /app
COPY yarn.lock /app

# install production dependencies  
RUN yarn install --pure-lockfile --production
# Save production depenencies installed so we can later copy them in the production image
RUN cp -R node_modules /tmp/node_modules

# install all dependencies including devDependencies
RUN yarn install --pure-lockfile
COPY . .

ENV NODE_ENV production
RUN yarn build

FROM node:slim
WORKDIR /app
COPY --from=builder /tmp/node_modules ./node_modules
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/package.json ./

ENV PORT 3000
EXPOSE 3000

CMD [ "blitz", "start" "--production" ]

Ah yes, looks a lot like what I've thought of. I'd add the we shouldn't need ENV NODE_ENV production since that's already set by running yarn build and also, we should add a health check:

apk add --no-cache curl // (or package manager is used in node:slim)

...

HEALTHCHECK CMD curl -f http://localhost:8080/ || exit 1;

Working on this as an installer

@dajinchu any updates on this?

Got blocked on the Prisma generation stuff but I believe that's fixed now. Repo was here https://github.com/dajinchu/blitz-docker-installer I think lots has changed with installers since but can take a look this weekend

Ok cool, no problem :)

Can't seem to figure out installers. Getting SyntaxError: Cannot use import statement outside a module

Hmm, maybe better to wait till @aem makes more progress on recipes?

In the meantime, can you post the dockerfile here so folks can use it manually?

Hello

This is my Dockerfile:

# Install all node_modules and build the project
FROM mhart/alpine-node:12 as builder
WORKDIR /app

COPY package.json yarn.lock ./
RUN yarn install --pure-lockfile

COPY . .
RUN yarn build

# Install node_modules for production
FROM mhart/alpine-node:12 as production
WORKDIR /app

COPY package.json yarn.lock ./
RUN yarn install --pure-lockfile --production

# Copy the above into a slim container
FROM mhart/alpine-node:slim-12
WORKDIR /app

COPY . . 
COPY --from=production /app/node_modules ./node_modules
COPY --from=builder /app/.next ./.next

EXPOSE 3000

# If possible, run your container using `docker run --init`
# Otherwise, you can use `tini`:
# RUN apk add --no-cache tini
# ENTRYPOINT ["/sbin/tini", "--"]

CMD ["./node_modules/.bin/blitz", "start", "--production"]

On a blank blitz installation this comes out to 283MB.

Note: I had to move @types/react into dependencies

Re health checks: I would leave that to for example kubernetes - https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

Now that recipes are released this is ready to be worked on. While the example files shown here are a great start, they are missing some small things and there are some best practices that can be improved to get a complete multi-stage setup going on. I want to give this a try if it's okay for everyone.

The above Dockerfile did not work cross platform (armv7) so I used a crossplatform source

Here is my Docker recipie

I used the node user provided There is a docker-compose.yml file as well

My build process fails at yarn install or npm install though, with many NPM and YARN errors npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm"})

It's indeed always better to use the official Docker images. Those fs-events errors pop up on Linux but you can ignore those. I still have to complete this recipe but I've been very busy with a project. I started working on it but it's not quite ready yet. https://github.com/LoriKarikari/sample-blitz-docker-setup. I have to pick this up ASAP sorry for the wait.

👀

A little note. Because of the sodium-native package you'll need to add this line to a "builder" phase:

RUN apk --update add --no-cache curl git python alpine-sdk \
  bash autoconf libtool automake

Without it sodium-native won't compile and install fails.

For whoever wants to work on this, the render recipe can by copied and modified for the docker recipe because it's doing the same thing (adding a file).

hello, i am the one from Discord who said i could look at this (so hopefully no conflict of interests).

i got this working locally. i was testing against a VSCode remote container for development, and Heroku for production. there are still a few considerations i need to work out.

assuming sqlite will remain the default database provider in the near future, i need to give a little more thought into making sure a production deploy of sqlite is intuitive and seemless. if a production deploy of sqlite is configurable with DATABASE_URL, we are good to go. if it's needs to rely on a volume, there's a bit more tweaking.
non-Linux volumes are notoriously slow (see docker-sync). i'm not sure if this accounts for all of the slowness yarn/npm experience in Docker, but it accounts for most of it. this means that mounting node_modules in a volume is generally a bad experience. but without mounting node_modules, tools like VSCode cannot by default find your modules. to address this, the user either needs to:
- be docker-adept, and understand that while Docker installs node_modules for running applications, they must keep a separate local copy of node_modules maintained for development. this can be "solved" with documentation.
- be ide-adept, and understand how to run their ide in the context of a container. this can be solved with ide specific recipes, like .devcontainer for vscode.

i handle this problem in my own projects by running VSCode in a container (mac) or just mounting the node_modules (linux).

i'm not quite sure the best way to handle if they are installing the recipe on a non-fresh project. for instance, should the Docker recipe detect if the user is using sqlite or postgres, and adjust the generated files accordingly? "no" is kinda sad DX, "yes" expands the scope of this PR and may get hairy to maintain all possible configurations.

in short, it's working for me and my setup (default sqlite project, VSCode container, sqlite in dev, postgres DATABASE_URL in prod)... but i'm not sure those are sensible defaults. so before opening a PR i want to make sure i am not missing anything obvious. if i don't hear back i will open a PR this weekend with my best effort, and we can continue the discussion there. but i thought it might be helpful to open up the things that will likely require discussion before i create one.

Don't use sqlite for production. I think assume postgres for now. Perhaps we can name recipe docker-postgres and then we can add recipes for mysql too.

Also we have two use cases: Docker only for prod and Docker for local and prod. Does it make sense to have two separate recipes for these?

Don't use sqlite for production. I think assume postgres for now.

this makes things easier.

Perhaps we can name recipe docker-postgres and then we can add recipes for mysql too.

how do you feel about a single recipe that prompts for input? (database type (oiptional), username and password)? the variations between a sqlite, psql, and mysql recipe are almost insignificant.

Also we have two use cases: Docker only for prod and Docker for local and prod. Does it make sense to have two separate recipes for these?

unless i am missing anything obvious, which i have a tendency of, it seems like for the default project we can treat prod as a strict superset of dev in a multistage build:

FROM node:14-stretch-slim as base
WORKDIR /opt/app
RUN apt-get update && apt-get install openssl -y && rm -rf /var/lib/apt/lists/* 

FROM base as env
COPY package*.json .
COPY yarn.lock .
RUN yarn install --frozen-lockfile

FROM env as dev
COPY . .
RUN yarn blitz prisma generate
CMD yarn dev --port ${PORT}

FROM dev as build
RUN yarn next telemetry disable && npm run build

FROM base as prod
COPY --from=build /opt/app/package.json /opt/app/blitz.config.js ./
COPY --from=build /opt/app/node_modules ./node_modules
COPY --from=build /opt/app/public ./public
COPY --from=build /opt/app/.blitz ./.blitz
COPY --from=build /opt/app/.next ./.next
CMD yarn start --port ${PORT}

prod target provides the minimal environment for yarn start, and dev target provides the mininmal environment for yarn dev. does this answer your question?

this is a scary one to share because i have never seen 2 identical Dockerfiles. :) but i have to get over that eventually, so i might as well start collecting feedback if you know the right channels. :) thank you again!

how do you feel about a single recipe that prompts for input? (database type (oiptional), username and password)? the variations between a sqlite, psql, and mysql recipe are almost insignificant.

That's a great idea! Although I'm not sure if currently support installer prompts? 🤔

unless i am missing anything obvious, which i have a tendency of, it seems like for the default project we can treat prod as a strict superset of dev in a multistage build:

👍

I would be willing to take on this if I get it to work for myself first.

Recipes are hard when you've to modify code etc, but a simple recipe could just add the dockerfile and not much else?

@agustif Do you have an update on your Dockerfile progress? I'm having issues generating the prisma client and wondering how you solved it.

EDIT: I solved it. Brandon informed me that the environment variable needs to be set correctly. I set NODE_ENV to production, and it picked up the production details correctly. Hiding comment.

More details: My Dockerfile is

FROM node:14
WORKDIR /app
COPY . .
RUN yarn
RUN npm i -g blitz@0.33.1 --legacy-peer-deps

RUN blitz prisma generate
RUN blitz build

EXPOSE 3000

CMD blitz start

When I run the container, I get this error. My connection string points to a digital ocean database in my .env files, so I'm unsure where the localhost:5432 string is coming from. EDIT: I am adding debug strings to the prisma client file to see where its coming from, so maybe I'll be able to identify this soon using that method

ERROR Error while processing the request

Error Can't reach database server at `localhost`:`5432`

Please make sure your database server is running at `localhost`:`5432`.

details:

{

clientVersion: '2.20.1',

errorCode: undefined

}

error stack:

• index.js:34868 cb

node_modules/@prisma/client/runtime/index.js:34868:17

• blitzjs-core-server.cjs.prod.js:998 getSessionKernel

node_modules/@blitzjs/core/server/dist/blitzjs-core-server.cjs.prod.js:998:28

• blitzjs-core-server.cjs.prod.js:662 getSession

node_modules/@blitzjs/core/server/dist/blitzjs-core-server.cjs.prod.js:662:23

• blitzjs-core-server.cjs.prod.js:626 <anonymous>

node_modules/@blitzjs/core/server/dist/blitzjs-core-server.cjs.prod.js:626:7

• blitzjs-core-server.cjs.prod.js:79 handleRequestWithMiddleware

node_modules/@blitzjs/core/server/dist/blitzjs-core-server.cjs.prod.js:79:5

• api-utils.ts:86 apiResolver

node_modules/next/next-server/server/api-utils.ts:86:5

• next-server.ts:1078 handleApiRequest

node_modules/next/next-server/server/next-server.ts:1078:5

• next-server.ts:952 fn

node_modules/next/next-server/server/next-server.ts:952:27

• router.ts:247 execute

node_modules/next/next-server/server/router.ts:247:24

This seems to be working as of Blitz 0.38.5 - a blend of the proposals from @benjick and @kn0ll. Without it, I was getting TypeScript errors for enums inferred from my Prisma Schema.

# Install all node_modules and build the project
FROM mhart/alpine-node:14 as builder
WORKDIR /app

COPY package.json yarn.lock ./
RUN apk add --no-cache make gcc g++ python3 libtool autoconf automake
RUN yarn install --pure-lockfile

COPY . .
RUN NODE_ENV=production yarn blitz prisma generate && yarn build

# Install node_modules for production
FROM mhart/alpine-node:14 as production
WORKDIR /app

COPY package.json yarn.lock ./
RUN apk add --no-cache make gcc g++ python3 libtool autoconf automake
RUN yarn install --pure-lockfile --production

# Copy the above into a slim container
FROM mhart/alpine-node:slim-14
WORKDIR /app

COPY . .
COPY --from=production /app/node_modules ./node_modules
COPY --from=builder /app/node_modules/.prisma ./node_modules/.prisma # Important, otherwise app will start but fail on first request. Doesn't look like Prisma is copied with @benjick's suggestion
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/.blitz ./.blitz

EXPOSE 3000
#
# If possible, run your container using `docker run --init`
# Otherwise, you can use `tini`:
# RUN apk add --no-cache tini

# ENTRYPOINT ["/sbin/tini", "--"]

CMD ["./node_modules/.bin/blitz", "start"]

This results in a 600MB file, which is pretty huge. Doesn't feel like I'm bringing in much as dependencies, just the Multi-tenancy customization and Stripe. Any ideas how I might cut this down?

EDIT: Looks like the production build step is bringing down and storing devDependencies which is not cool. Looking into it.

https://github.com/blitz-js/blitz/pull/2475 was merged and was ostensibly attempting to improve this but I'm still seeing issues with the size.

I cannot understand why the resulting image is so huge. @flybayer and crew any ideas? Making my deploys pretty huge right now.

Nope, no immediate ideas. node_modules will just be very big in a lot of cases, I don't know if there's a way around that. If this can't be fixed by some magic Dockerfile, and you still need tiny images, you may want to use something like webpack to inline & treeshake your dependencies into the bundle. I imagine that will be very finicky, though.

@tomhallam can you open a new issue specifically for the size issue?

Haven't had a chance to dig into it. I'll try to dig in as soon as I can, but would definitely appreciate anyone's help.

@tomhallam

COPY --from=builder /app/.blitz ./.blitz
...
CMD ["./node_modules/.bin/blitz", "start"]

is not working anymore as of the latest canary build.

With latest canary, copy .next. .blitz is no longer used

With latest canary, copy .next. .blitz is no longer used

What about .blitz.config.compiled.js @flybayer?

Also, it's worth noting that your custom server won't work if you just copy the .next folder in canary. ATM I'm copying all custom server source files into my production build docker container

@andreasasprou yes you will need .blitz.config.compiled.js. Currently that's not inside .next because the config is built, then .next is totally wiped. We should change that so we can store .blitz.config.compiled.js inside there.

Also, it's worth noting that your custom server won't work if you just copy the .next folder in canary. ATM I'm copying all custom server source files into my production build docker container

Hmm, it should. Because custom server is built to .next/. Could you open an issue for this case?

Is there anything I could test? Or even help with a Dockerfile that tries to reduce the size?

Is there anything I could test? Or even help with a Dockerfile that tries to reduce the size?

@kivi you can take a look at this repo from @LoriKarikari: https://github.com/LoriKarikari/blitz-docker. It has a few examples

Thanks @beerose, I already was discussing with Lori and others on Discord. I have this example running: https://github.com/kivi/blitz-docker-example/blob/main/Dockerfile

I was more likely referring to the topic of this issue: "Add a recipe for Docker(file)"

@kivi oops, I misunderstood. Maybe this docs page on writing/testing recipes will help: https://blitzjs.com/docs/writing-recipes?

blitz-js / legacy-framework