Failed on docker compose up

[francesco-di-marcantonio]

Hi! i'm trying to follow your guide for deploying the infrastructure. First try: Deploying existing m2 The first try was building a docker image based on the current development that we have done. The infrastructure already exist with his environment but when we build the image we always receive error connection with redis or mysql. It looks like that the services are not comunicating each other.

Second try: magento-from scratch

there are some changes that i needed to make:

cloud build

composer require phoenix-media/magento2-cloud-build --no-update

became

composer require phoenix-media/magento2-cloud-build:1.0.29 --no-update

for downloading the latest version that is compatble with magento 2.4.5

then we try to run

docker build . -t xx/yy

and then

docker compose up

rabbit and redis looks running but magento return this log:

agento | ****************************** magento | ** nginx server_name SETUP *** magento | ****************************** magento | ****************************** magento | ******* POSTFIX SETUP ******** magento | ****************************** magento | ****************************** magento | ******* PHP SETUP ************ magento | ****************************** magento | 2024-01-10 04:03:53,753 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. magento | 2024-01-10 04:03:53,754 INFO supervisord started with pid 1 magento | 2024-01-10 04:03:54,759 INFO spawned: 'deploy' with pid 8 magento | 2024-01-10 04:03:54,761 INFO spawned: 'nginx' with pid 9 magento | 2024-01-10 04:03:54,762 INFO spawned: 'php-fpm' with pid 10 magento | 2024-01-10 04:03:54,764 INFO spawned: 'master' with pid 11 magento | /bin/bash: line 1: /var/www/html/bin/deploy: No such file or directory magento | 2024-01-10 04:03:54,764 INFO success: deploy entered RUNNING state, process has stayed up for > than 0 seconds (startsecs) magento | 2024-01-10 04:03:54,764 INFO success: master entered RUNNING state, process has stayed up for > than 0 seconds (startsecs) magento | 2024-01-10 04:03:54,764 INFO exited: deploy (exit status 127; not expected) magento | 2024/01/10 04:03:54 [emerg] 9#9: "server_names_hash_bucket_size" directive is not allowed here in /etc/nginx/conf.d/healthz.conf:1 magento | nginx: [emerg] "server_names_hash_bucket_size" directive is not allowed here in /etc/nginx/conf.d/healthz.conf:1 magento | 2024-01-10 04:03:54,773 INFO exited: nginx (exit status 1; not expected) magento | [10-Jan-2024 04:03:54] NOTICE: fpm is running, pid 10 magento | [10-Jan-2024 04:03:54] NOTICE: ready to handle connections magento | 2024-01-10 04:03:54,892 INFO exited: master (exit status 0; expected) magento | 2024-01-10 04:03:55,894 INFO spawned: 'nginx' with pid 93 magento | 2024-01-10 04:03:55,895 INFO success: php-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) magento | 2024/01/10 04:03:55 [emerg] 93#93: "server_names_hash_bucket_size" directive is not allowed here in /etc/nginx/conf.d/healthz.conf:1 magento | nginx: [emerg] "server_names_hash_bucket_size" directive is not allowed here in /etc/nginx/conf.d/healthz.conf:1 magento | 2024-01-10 04:03:55,914 INFO exited: nginx (exit status 1; not expected) magento | 2024-01-10 04:03:57,920 INFO spawned: 'nginx' with pid 94 magento | 2024/01/10 04:03:57 [emerg] 94#94: "server_names_hash_bucket_size" directive is not allowed here in /etc/nginx/conf.d/healthz.conf:1 magento | nginx: [emerg] "server_names_hash_bucket_size" directive is not allowed here in /etc/nginx/conf.d/healthz.conf:1 magento | 2024-01-10 04:03:57,927 INFO exited: nginx (exit status 1; not expected) magento | 2024-01-10 04:04:00,934 INFO spawned: 'nginx' with pid 95 magento | 2024/01/10 04:04:00 [emerg] 95#95: "server_names_hash_bucket_size" directive is not allowed here in /etc/nginx/conf.d/healthz.conf:1 magento | nginx: [emerg] "server_names_hash_bucket_size" directive is not allowed here in /etc/nginx/conf.d/healthz.conf:1 magento | 2024-01-10 04:04:00,945 INFO exited: nginx (exit status 1; not expected) magento | 2024-01-10 04:04:01,946 INFO gave up: nginx entered FATAL state, too many start retries too quickly it looks like that nginx is failing becose the default conf file is not working.

I created a repo for checking the status

link

[francesco-di-marcantonio]

i also try to run the command

vendor/bin/px-cloud-build-install

but i received this answer:

mkdir: impossible to create directory "..test2/src/vendor/phoenix-media/helm": File già esistente mkdir:mpossible to create directory "..test2/src/vendor/phoenix-media/m2-hotfixes": already existent cp: l'obiettivo '..test2/src/vendor/phoenix-media/dev/' its not a directory cp: l'obiettivo '/..test2/src/vendor/phoenix-media/bin/' its not a directory cp: impossible create regular file '/home/kimura/Scrivania/iride/Helm/test/test2/src/vendor/phoenix-media/app/etc/config.php': missing file or directory

part of the erros are because i directly copied from vendor/composer the skeletong as you mentioned in your post. But for example dev and bin directory are non existant. same with config.php

[francesco-di-marcantonio]

the last problem i think is related to this: MAGENTO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)" but if the script is under vendor/phoenix etc etc, it's not taking the right MAGENTO_ROOT

I'm also trying to fix the path for dev and bin but i dont understand why they are here. Same for

/vendor/bin/composer config apcu-autoloader true what is it doing? because if we run it, obviously it doesnt have the composer json file in the folder so the command is not running

[phoenix-bjoern]

@francesco-di-marcantonio The vendor/bin/px-cloud-build-install script tries to detect the MAGENTO_ROOT in order to copy the files in vendor/phoenix-media/magento2-cloud-build/dist to the intended location. If the script fails you can still copy the files manually without the script.

Regarding the docker-compose support I will do some testing, haven't used it quite a while.

[phoenix-bjoern]

@francesco-di-marcantonio One issue I've found during docker build is the missing Magento 2.4.6 support of the Composer package. Therefore the wrong version has been installed. That should be fixed with the 1.0.30 release of phoenix-media/magento2-cloud-build. The pipeline build of the Magento2 container image was successful, so you can also compare the files generated by the build.sh script with your code base.

Will perform more tests with docker-compose to update container images and configuration where necessary.

[phoenix-bjoern]

@francesco-di-marcantonio found an issue in the PHP base image and fixed it. Execute a docker pull phoenixmedia/nginx-php:8.1 and docker-compose up --force-recreate --build to ensure you're running on the latest container versions. Feel free to change the tag to "8.2" in the Dockerfile if you want to run Magento2 with the latest PHP version.

N.B. The nginx container doesn't have a TLS/SSL setup. When running the container with docker-compose you might have to change Magento's secure base URL to run in plain HTTP (bin/magento config:set "web/secure/base_url" "http://localhost/"; bin/magento cache:flush).

[francesco-di-marcantonio]

Just tested the workflow in magento2-build and it's working. It create a magento installation and push to dockerhub. The installation is without services connection (no env file). just a question. why you developed this workflow? if you would develop on this machine it became more complex. Is there some core information that i'm not getting?

So actually i should create a new docker image (from local environment) following your guide, publish to magento2 and then adapt the magento2 helm chart with the reference of the new created docker image.

In this context how do you manage sql difference? if you create a module, you need to launch setup:upgrade for updating the db. In which step the db is updated?

thank you so much for all this information. trying to understand the amazing job that you have done!

Thank you so much for sharing all this information.

[phoenix-bjoern]

@francesco-di-marcantonio great it worked now for you! The big picture is explained in my Medium series. You can read all three articles where I also explain how bin/magento setup:upgrade is executed in the deployment phase.

The whole work is about CI/CD and the question "How do we deploy Magento2?". On every "git push" a CI pipeline builds a new container image. You can adopt the Github action in your CI tool and checkout your M2 git repo instead of installing a clean M2 code base. The container image should then be pushed to a private container registry with a unique tag which corresponds to git commit (you can use the short commit hash as image tag). With that the CI process is done.

Now the CD (Continuous Delivery) process is of course not docker-composer up. This was only added to debug/verify the build process of the container image. Instead you can use the Magento2 Helm chart to deploy the new image to your target environment. The CD process comes down to a simple command like helm upgrade -i -n target-namespace -f image.repository=your-registry.com/project/magento -f image.tag=hash123 .. With that K8S should deploy Magento2 like any other native cloud application and update new images without noticeable downtime (incl. all the quirks like setup:upgrade and cache cleaning).

And this is basically the whole idea: Fully automate Magento2 deployments in a cloud native way - from A (build the image) to Z (deploy it to production). We use dynamic target namespaces and domain names to create new environments on demand whenever a new branch is pushed to git. The only trick necessary is to make the CD process a bit "smarter".

My colleague has created a cool video which shows how M2 can be deployed to GKE: https://www.youtube.com/watch?v=gJC70QHzzsU&t=23s. Hope you enjoy it!

[francesco-di-marcantonio]

It's working the workflow but stil the buildiin phase is not working.

The process stop here

RUN su -s /bin/bash nginx -c "/var/www/html/vendor/bin/ece-tools build:generate"

That is the same we had at the beginning. When we deploy on Dockerfile it's not able to find redis connection. How should be managed?

[phoenix-bjoern]

@francesco-di-marcantonio during docker build there shouldn’t be a connection to any service as the CI pipeline does not run within a hosting environment. I assume this doesn’t occur in a vanilla installation. Try to identify the module which connects to sessions or cache in that phase.

[francesco-di-marcantonio]

In the env.php and config.php in app/etc/ there are the configuration for the services. do you delete both of the file before copying to src? and in the different environment how can you set the right env file? Because at the end of the helm chart deploy the magento installation need both of them with the configuration of the services

[phoenix-bjoern]

@francesco-di-marcantonio app/etc/config.php should be in the git repo, env.php not. Remove it! The file is generated on the fly by ece-tools. Credentials are set per environment variables for each environment. It is described in the article as well. if you need additional settings in env.php customize src/bin/deploy. This way you can deploy the same image to any environment and inject credentials as needed.

[phoenix-bjoern]

@francesco-di-marcantonio https://medium.com/@bjoern.kraus/running-magento2-in-kubernetes-part-2-building-the-docker-image-8516c0ed7d48 explains the env.php generation in detail.