[RFC] Docker Questions / Optimization

Hello Boltheads and Docker Friends,

i was using Bolt via composer setup in the past but in the last couple of years changed to completely dockerized servers. There are a couple of things about the current Docker setup for bolt that i feel could be optimized or should be changed.

I know that there are lot of use cases for docker setups, and unlike mentioned in the Bolt documentation, running a production site via Docker is not only possible but could be a very quick and handy way to setup Bolt-powered websites. Discussions about Docker can quickly become dogmatic, since there are many people that have pretty strong objections against it - while most of them never actually tried using docker containers, maybe those people could refrain from participating in the discussion.

i think its good if we could assume that docker containers CAN run production websites, because this is done a lot. even by very big companies. VPS servers and other cloud computing platforms that run Docker containers are becoming cheaper every year, so the classical "managed host with php" scenario may become less and less in thu future.

The last days i spend understanding the existing Bolt Docker setup, going through the Dockerfiles etc. My biggest ciriticism would be the lack of documentation and the complexity in which things are organized. Then and there are some things that i don't understand or don't think are very efficient:

The Container configures a NGinx to serve the static files and proxy PHP-FPM. But then the docker-compose.yml stack sets up ANOTHER NGinx proxy server in front of the other NGinx, just for SSL and http2. Maybe there is something i dont understand about this approach. i think it would be fairly easy to optionally setup the self signed cert and http2 for the existing Nginx, enabling & disabling options via the .env file. Personally i dont need either, since i have a central ssl offloading proxy server (HAProxy) for all my internet facing containers and i think this is what most people have. Would it no be enough to set up an unencrypted Nginx and make a note in the docs that people should look into a Proxy Servers and LetsEncrypt et al if they want to run this properly?
Minor thing, but why name the Bolt container "php", i guess it should be either "nginx-php-bolt" or similar or just bolt.
While i see the advantage of only having to clone the entire bolt/project directory from github, regardless of running the composer setup or via docker, there is a lot of "garbage" left over from this approach, which is simply not needed or used when running the docker.... i think it would be fair to argue for a seperate bolt/docker repo, which clones the required bolt/project files into the container during the built progress and not throws everyting together, duplicates it and leaves a big mess at the end.
i think a reasonably big share of bolt users might be OK with a SQLite based install for small project websites. or a quick tryout. the ones that need MariaDB / MySQL probably have a Database already running or know how to set one up. In any case it should be very easy and well documented how to switch between these two types of setup. This is not the case at the moment.
I guess the idea behind running a complete build process for everyone that wants to run the Bolt Docker is to optimize cross platform compatibility? I think this might be problematic for people running low spec servers (think RPi <=4)... There are other approaches, like automatic cross platfotm container builds using CI etc.. This is probably a lot of work to set up initially but then save EVERY user to go through the build process again.
Generally speaking, a more streamlined, simple and well documented approach Docker pre-built Container would encourage more people to give Bolt a try by just spinning it up quickly. With the level of complexity, the extra build steps and lack of documentation there is a risk that people just feel overwhelmed and give it a pass.
The default 2021 theme is built around tailwind.css which CAN be used as a complete CDN version, in which case its a hefty 3.3MB css file. But it really only makes sense if you run a build chain with postcss and compile it for YOUR use case. If Bolt stays with tailwind, the build chain (node.js / npm / postcss / autoprefixer / maybe nanocss) should be integrated in the Docker container as well, to make it usable as a complete starting point for building your sites. Either the build chain could be integrated into the nginx-php-bolt container or the docker-compose stack could bring in another container in the stack which takes care of that, similar to - https://hub.docker.com/r/hut6/postcss

Like i said, totally possible that i misunderstood a couple of the original ideas, in which case i would be happy if someone would enlighten me about what was intended.

Cheers, T.

@toofff

@tilllt I don't have much time now but I try to answer you as soon as possible.

In any case what I did is a first draft which can evolve and be improved. And the idea of my contribution was to have a functional docker for local development that can be easily deployed in production via a Helm chart which is still in the boxes. ;)

Hey, first of all thanks for your work. This is not at all urgent... I am not an expert of creating custom docker containers, I am just a classical lazy person that heavily relies on docker containers for my internal infrastructure.

That said, I have heard about Kubernetes and Helm, but I think that for personal or even a boutique agency use, Kubernetes is way too much overhead.

Half of my container is started via command line, half via docker compose. The only management tool I use is portainer.

I would assume that an organization that uses kubernetes & helm also has the insight and personell to adapt a container to their needs, while the "semi-pro" or small agency is rather turned off from too much complexity. So I don't know if Kubernetes etc is really a real-world use case for people that develop websites with bolt...

As discussed in Slack, i think adding the tailwind.css buildchain to the Docker would be a good move since the default theme is built on it. Without purging unused CSS, Tailwind is >3MB, reduced to the parts that are actually used in the Bolt Default Theme, its 47KB.

For me as an eternal web noob, the default themes have always been a good starting point on what to build on, so i think it would be a good idea if the build-chain was available. afaik the whole postcss process requires node.js / npm, so i modified the Dockerfile here:

`RUN apk add --no-cache \
        acl \
        fcgi \
        file \
        gettext \
        git \
        ttf-freefont \
        fontconfig \
        dbus \
        freetype-dev \
        libjpeg-turbo-dev \
        libpng-dev \
        npm`

and added

`# install postcss build chain for tailwind.css
RUN npm i -D tailwindcss postcss autoprefixer @fullhuman/postcss-purgecss cssnano`

then you can initialize the tailwind config like this:

docker exec -it -e NODE_ENV='production' dockername_php_1 /usr/bin/npx tailwindcss-cli@latest init ./public/theme/base-2021/tailwind.config.js

and build the production css like this:

docker exec -it -e NODE_ENV='production' dockername_php_1 /usr/bin/npx tailwindcss-cli@latest build -c ./public/theme/base-2021/tailwind.config.js ./tailwind_css/base-2021.css -o ./css/base-2021.css

Obviously you have to configure the tailwind..config.js to your needs, especialle enabling the purge feature and pointing it to the twig files.

`module.exports = {
  purge: [
  './**/*.twig',
  ],
  darkMode: false, // or 'media' or 'class'
  theme: {
    extend: {},
  },
  variants: {
    extend: {},
  },
  plugins: [],
}`

After unsusccesfully trying to install a specialized *.twig extractor for purgecss, i found out that the purge functions seems to work ok, just using HTML regex on twig files, but i really dont have a lot of experience with purgecss to judge if this is a bad idea.

As mentioned earlier, buidling Docker containers is not something i know anything about, so if these changes make sense would be i.E. for @toofff to judge.

Another issue: sometimes, and i dont know how to reproduce this, the /var directory disappears as a volume and only exists as a transient volume in the docker overlay. Which means: after building / pulling a new version of the Container, the site is gone, at least when using SQLite.

that is obviously a very undesirable thing to happen, i dont know how to reliably reproduce this, but i had it happening twice already....

There are a couple of things that might be related to this, but again, i dont know enough about Container creation to fully debug this:

Dockerfile

RUN set -eux; \
        mkdir -p var/cache var/log var/data; \
        composer dump-autoload --classmap-authoritative --no-dev; \
        chmod +x bin/console; sync
VOLUME /srv/bolt/var

docker-compose.yml

 # if you develop on Linux, you may use a bind-mounted host directory instead
      # - ./var:/srv/bolt/var:rw

i dont quite understand towards who the comment ("If you develop on linux...") is pointed: my assumption would be that 90% of the people using docker do this on a linux dev machine, so maybe linux options should be default and everything else (macOS / Windows) optional...

Furthermore: does this mean that in a default setup, the SQLite Database is stored in transient volumes, unless specifically told otherwise? imho, that is a very bad trap for people running bolt via docker, and should be the other way around.

Hi @tilllt

For the documentation, I added it in the README.md of the project, but feel free to ask me what you want me to add. And it can obviously be added in the official documentation with a little refacto on the installation and that it is the recommended method.

Here are the answers I can give to your first post:

1/ Here is an excerpt from the README.md file on the different services launched by docker compose.

For Docker, what is often recommended is never to use docker-compose in production but a container orchestrator such as Kubernetes or another SaaS solution. So I use docker-compose in development then we will build our images to push them into production. That's why I use services (db, h2-proxy and mailcatcher) and configures that in docker-compose.

Obviously there are plenty of other solutions to implement SSL, it's a pretty simple solution that I implemented but only for development.

If I have to put a Bolt project in production, I would have 2 docker images (php and nginx), then I will use a managed database, a dedicated service for setting up SSL (like ingress in kubernetes) and a dedicated email service.

2/ For me the name of the service must correspond to the technology used mainly in the container. We mainly rely on the PHP image to build our own. After that the name does not matter, but the simpler and more explicit it is, the better it will be for maintenance in production.

3/ Could you specify which files are bothering you and which are duplicated, it is surely an error on our part? For me, this project is doomed to disappear when we have finished creating flex recipes. Even part of the docker setup may be generated by flex revenue. But before that happens, this project may serve us well :smiley:

4/ Here is an excerpt from README.md file on the Docker part.

We ask to update the .env file, but we can very well leave the SQLite configuration. And if you don't want to keep MySQL, just remove the db service from the docker-compose.yml file

5/ I'm not sure if the Bolt Core Team is ready to do all of this work.

6/ Do you have an example of what you would like to see in the documentation and I will take care of it next week.

7/ I deliberately did not add a nodeJs container for the theme build for 2 reasons.

Because I don't have enough experience on Bolt development on the front end yet.
For me there is a problem with the build of themes by the core project and the fact that everything is in the same repo. So not to complicate this part, it would be interesting to make the changes in the core project before adding this part here. But that's another topic for me.

Hope this first piece of answer fits you? I will continue tomorrow noon on the rest of your posts

For the addition of tailwind.css buildchain in to the Docker, I recommend adding a nodeJs container, to have a separation between the 2 PHP and NodeJS services. This NodeJs container must be in direct dependency of the PHP container, because it will be necessary to wait on the PHP container to be run to be able to launch the run of the NodeJs container (the dependencies of Webpack Encore on the Symfony side must be installed and launched as soon as it is done, the container nodeJs can launch either the build or the watch with Webpack Encore)

By doing this, we can more easily control the build and the watch of the Node part (whether with npm or yarn) and above all this container will not be necessary in production.

Because in production we shouldn't need the NodeJs image in my opinion, we build all the docker images (so the different assets will already be available in the PHP and Nginx images) and we can just send the PHP and Nginx images to production. on a Kubernetes cluster for example.

Should see with @bobdenotter if he would be interested in opening a demo repo for Bolt with an example of deployment with Docker and Kubernetes directly set up from the Digital Ocean or Google Cloud Platform for example?

in Regards to 1/ i think it would be useful to define who the major target group of the bolt-docker and maybe Bolt in general is. People DO run production services from docker-compose, there are mechanisms for management (i.e. auto-update via ouroboros) and gui tools (portainer) for it. I never saw any comment that running Containers from the command line or docker-compose is discouraged for a productive use case,

i think it depends on your ability to monitor and administrate the containers you are running. If you have i.e. a Bolt website and maybe Nextcloud as the two services you are running for your business, Kubernetes is surely overkill to manage them EVEN if you are running those two Containers productively (vs. not for fun) for your business.

Kubernetes in my opinion is WAY beyond the interest and scope of the majority of people that will run bolt powered websites. I dont see Bolt as a CMS for large corporations, government institutions etc. but maybe i am getting a wrong impression there. Would be good to have input from Bob what he would see as the main target group of bolt.

If you have to monitor 150 containers, solutions like Graylog, Kubernetes, Nagios etc. are essential, but people running services at that scale usually have a good knowledge of changing & building containers for their needs, so i would simply neglect this target group and concentrate on the "low end" usecase for dockerized bolt.

as for 7/ The default theme uses tailwind.css which i didnt know before coming back to bolt this year, but i think its a great starting point for a default theme. The problem is that even the creators of tailwind.css are questioning its usefulness if not being used with postcss / purgcess. The non processed version of Tailwind is ~3.5 MB, when trimmed to the css actually used in the default theme, its 47KB. So i would argue that implementing the tailwind build chain and documenting how to use it is part of the decision to use it in the default theme. I can absolutely see the point that bringing it in as a seperate container is the "cleaner" solution, i was simply lacking the knowledge on how to do that, especially since it shoudl re-use all the layers that we brought in for the php / nginx containers already, to keep it efficient...

afaik it is not necessary to use webpack or make dependencies between the php and the buildchain container, you can just use postcss (autoprefix) & purgecss (optimally with specific twig extractors), let it analyze the used twig templates and then weed out and prefix the full tailwind.css.

I know, this is an older topic, but I just went through the process of setting up tailwind with purgeCSS and cssnano. This is a WIP, so things may not be perfect, but it feels quite good. tailwind.css is down from ~100k line to ~1000 lines of css (without cssnano of course). I did not check the sizes, but you get the idea.

Let me share my current setup (all files are relative to public/theme/my-theme/):

package.json:

{
    "name": "current",
    "version": "1.0.0",
    "description": "",
    "scripts": {
        "build-tailwind": "postcss css/tailwind/*.css -d css/",
        "watch": "chokidar '*.config.js' 'css/tailwind/*.css' '**/*.twig' -i 'node_modules' -c 'yarn build-tailwind' --initial"
    },
    "keywords": [],
    "author": "ANIMAL Design OG <studio@animal.at>",
    "license": "ISC",
    "devDependencies": {
        "autoprefixer": "^10.2.6",
        "chokidar-cli": "^2.1.0",
        "cssnano": "^5.0.6",
        "postcss-cli": "^8.3.1",
        "tailwindcss": "^2.2.2",
        "tailwindcss-scroll-snap": "^1.1.0"
    }
}

postcss.config.js:

module.exports = {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
    cssnano: {
      preset: "default",
    },
  },
};

tailwind.config.js: (I left some project specific stuff in, but the purgepart is the important one.)

module.exports = {
  purge: {
    mode: "all",
    enabled: true,
    preserveHtmlElements: false,
    options: {
      keyframes: true,
    },
    content: ["./**/*.twig"],
  },
  darkMode: false,
  theme: {
    extend: {},
    colors: {
      white: "#FDFDFD",
      dark: "#262525",
      semidark: "#313131",
      brand: "#FDEA34",
    },
    fontFamily: {
      sans: ["Gotham", "sans-serif"],
    },
  },
  variants: {
    scrollSnapType: ["responsive"],
    extend: {},
  },
  plugins: [require("tailwindcss-scroll-snap")],
};

css/tailwind/tailwind.css:

@tailwind base;
@tailwind components;
@tailwind utilities;

Now while development I run yarn watch. Similar to the default bolt theme, my CSS files live in css/tailwind/ and get "compiled" to css/ in my theme folder. The CSS is then imported as usual:

[...]
<link rel="stylesheet" href="{{ asset('css/tailwind.css') }}">
<link rel="stylesheet" href="{{ asset('css/main.css') }}">
[...]

I hope this helps someone, let me know if any questions or suggestions arise.

bolt / project

[RFC] Docker Questions / Optimization #51