search

topheman / docker-experiments

Discover docker with a simple use case in development, production (local kubernetes) and CI
97 stars 11 forks source link
docker docker-compose kubernetes

readme

docker-experiments

CircleCI

This started as a simple use case to discover docker and docker-compose 🐳 :

I also setup deployments on a local kubernetes ☸️ and tests are running on CircleCI on each push.

TL;DR

You are a true developer? You don't RTFM? After all, this is why we have docker ... not to bother with all the boring setup/install steps ... 😉

git clone https://github.com/topheman/docker-experiments.git
cd docker-experiments
docker-compose up -d

You are good to go with a development server running at http://localhost:3000, with the front in react, the api in go and everything hot reloading. 👏

Try to take a few minutes to read the doc bellow ... 😇

Summary

Prerequisites

You need to have installed:

Setup

git clone https://github.com/topheman/docker-experiments.git

A Makefile is available that automates all the commands that are described bellow. For each section, you'll find the related commands next to the 🖊 emoji.

Just run make help to see the whole list.

Development

Launch development

docker-compose up -d

This will create (if not already done) and launch a whole development stack, based on docker-compose.yml, docker-compose.override.yml, api/Dockerfile and front/Dockerfile - following images:

Go to http://localhost:3000/ to access the frontend, you're good to go, the api is accessible at http://localhost:5000/.

🖊 make dev-start, make dev-start-d, make dev-stop, make dev-ps, make dev-logs, make dev-logs-front, make dev-logs-api

Tests

Launch tests

docker-compose run --rm -e CI=true front npm run -s test && docker-compose run --rm api go test -run ''

🖊 make test, make test-front, make test-api

Production - docker-compose

This section is about testing the production images with docker-compose 🐳 (check the deployment section to deploy with kubernetes locally).

Make sure you have built the frontend with docker-compose run --rm front npm run build, then:

docker-compose -f ./docker-compose.yml -f ./docker-compose.prod.yml up --build

Note: make sure to use the --build flag so that it will rebuild the images if anything changed (in the source code or whatever), thanks to docker images layers, only changes will be rebuilt, based on cache (not the whole image).

This will create (if not already done) and launch a whole production stack:

Access http://localhost and you're good to go.

🖊 make prod-start, make prod-start-d, make prod-start-no-rebuild, make prod-start-d-no-rebuild, make prod-stop, make prod-ps, make prod-logs, make prod-logs-front, make prod-logs-api

Deployment - kubernetes

This section is about deploying the app locally with kubernetes ☸️ (not tested with a cloud provider). To stay simple, there aren't any TLS termination management (only port 80 exposed).

Local kubernetes server and client:

The files descripting the deployments are stored in the deployments folder. You will find two files, each containing the deployment and the service.

Deploy with kubernetes

1) If you haven't built the frontend, run docker-compose run --rm front npm run build

2) Build the production images:

docker build ./api -t topheman/docker-experiments_api_production:1.0.1
docker build . -f Dockerfile.prod -t topheman/docker-experiments_nginx:1.0.1

Note: They are tagged 1.0.1, same version number as in the deployments files (want to put an other version number ? Don't forget to update the deployment files). For the moment, I'm not using Helm that let's you do string interpolation on yml files.

3) Create your pods and services

Make sure nothing is up on port 80, then:

kubectl create -f ./deployments/api.yml -f ./deployments/front.yml

You're good to go, check out http://localhost

To stop and delete the pods/services you created:

kubectl delete -f ./deployments/api.yml -f ./deployments/front.yml

They won't stop right away, you can list them and see their status with:

kubectl get pods,services

More commands

🖊 make kube-start, make kube-start-no-rebuild, make kube-stop, make kube-ps

Notes

Docker Multi-stage builds

Thanks to docker multi-stage builds, the golang application is built in a docker golang:alpine image (which contains all the tooling for golang such as compiler/libs ...) and produces a small image with only a binary in an alpine image (small Linux distrib).

The targets for multi-stage build are specified in the docker*.yml config files.

The api/Dockerfile will create such a production image by default.

You can tell the difference of weight:

docker images
topheman/docker-experiments_api_production      latest  01f1b575fae6  About a minute ago  11.5MB
topheman/docker-experiments_api_development     latest  fff1ef3ec29e  8 minutes ago       426MB
topheman/docker-experiments_front_development   latest  4ed3aea602ef  22 hours ago        225MB

Docker networks / Kubernetes services

In development, the api server in golang is available at http://localhost:5000 and proxied onto http://localhost:3000/api (the same port as the front, thanks to create-react-app proxy).

In production mode, we only want the golang server to be available via /api (we don't want to expose it on it's own port).

To make it work:

That way, the nginx conf can work with both docker-compose AND kubernetes, proxying http://api - see nginx/site.conf.

Restart on failure

If your app exits with a failure code (greater than 0) inside the container, you'll want it to restart (like you would do with pm2 and node apps).

With docker-compose/production, the, directive restart: on-failure in the docker-compose.yml file will ensure that. You'll be able to check it by clicking on the "exit 1 the api server" button, which will exit the golang api. You'll see that the uptime is back counting from 0 seconds.

With kubernetes/deployment, I setup 2 replicas of the api server, so when you retrieve the infos, the hostname might change according of the api pod you're balance on.

Exiting one pod won't break the app, it will fallback on the remaining replica. If you exit the two pods, you'll get an error retrieving infos, until one of the pod is back up by kubernetes (check their status with kubectl get pods).

Commands

Docker commands

Don't want to use docker-compose (everything bellow is already specified in the docker*.yml files - only dropping to remember the syntax for the futur) ?

Kubernetes commands

kubectl Cheat Sheet

FAQ

CircleCI

How to use latest version of docker-compose / docker-engine

I had the following error on my first build:

ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version ("2.0", "2.1", "3.0", "3.1", "3.2") and place your service definitions under the services key, or omit the version key and place your service definitions at the root of the file to use version 1.

For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

The reason was because I'm using docker-compose file format v3.4, which doesn't seem to be supported by the version of docker-engine used on the default setup of CircleCI - see compatibility matrix.

With CircleCI, in machine executor mode, you can change/customize the image your VM will be running (by default: circleci/classic:latest) - see the list of images available. I simply changed the image to use:

version: 2
jobs:
  build:
-    machine: true
+    machine:
+      image: circleci/classic:201808-01

Checkout .circleci/config.yml

Note: Why use docker-compose file format v3.4 ? To take advantage of the target attribute.

Docker vs Machine executors

You can not build Docker within Docker.

To build/push docker images, you have two solutions on CircleCI:

Why does /api fallbacks to index.html in production

Service Worker

create-react-app ships with a service worker by default which implementation is based on sw-precache-webpack-plugin (a Webpack plugin that generates a service worker using sw-precache that will cache webpack's bundles' emitted assets).

It means that a service-worker.js file will be created at build time, listing your public static assets that the service worker will cache using a cache first strategy (on a request for an asset, will first hit the service worker cache and serve it, then call the network and update the cache - this makes the app fast and offline-first).

From the create-react-app doc:

On a production build, and in a browser that supports service workers, the service worker will automatically handle all navigation requests, like for /todos/42 or /api, by serving the cached copy of your index.html. This service worker navigation routing can be configured or disabled by ejecting and then modifying the navigateFallback and navigateFallbackWhitelist options of the SWPreachePlugin configuration.

What's next?

The next thing that will be comming are:

This is still in progress.

Resources

More bookmarks from my research:

Author

Christophe Rosset