"Could not connect to Amqp server" error during Magento 2 bootstrap on M1 Mac

[smithi1]

I am using reward to try and get a Magento 2 local environment running on my 16GB M1 Mac Mini. In the Docker Desktop settings I have allocated 8GB RAM and 4 CPUs. Docker Desktop is the "Apple Silicon" version.

reward bootstrap exits with the following error (surplus whitespace removed):

Could not connect to the Amqp Server.
In InstallCommand.php line 274: Parameter validation failed

Running the RabbitMQ container separately gives:

% docker run rewardenv/rabbitmq:3.8
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
WARNING: 'docker-entrypoint.sh' generated/modified the RabbitMQ configuration file, which will no longer happen in 3.9 and later! (https://github.com/docker-library/rabbitmq/pull/424)

Generated end result, for reference:
------------------------------------
loopback_users.guest = false
listeners.tcp.default = 5672
------------------------------------
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

For completeness, .env file is:

REWARD_ENV_NAME=sa-local
REWARD_ENV_TYPE=magento2
REWARD_WEB_ROOT=/

TRAEFIK_DOMAIN=sa-local.test
TRAEFIK_SUBDOMAIN=
TRAEFIK_EXTRA_HOSTS=

REWARD_DB=1
REWARD_ELASTICSEARCH=1
REWARD_VARNISH=1
REWARD_RABBITMQ=1
REWARD_REDIS=1
REWARD_MERCURE=0

ELASTICSEARCH_VERSION=7.12
MARIADB_VERSION=10.4
NODE_VERSION=10
PHP_VERSION=7.4
RABBITMQ_VERSION=3.8
REDIS_VERSION=6.0
VARNISH_VERSION=6.5
COMPOSER_VERSION=2

REWARD_SYNC_IGNORE=

REWARD_ALLURE=0
REWARD_SELENIUM=0
REWARD_SELENIUM_DEBUG=0
REWARD_BLACKFIRE=0
REWARD_SPLIT_SALES=0
REWARD_SPLIT_CHECKOUT=0
REWARD_TEST_DB=0
REWARD_MAGEPACK=0

BLACKFIRE_CLIENT_ID=
BLACKFIRE_CLIENT_TOKEN=
BLACKFIRE_SERVER_ID=
BLACKFIRE_SERVER_TOKEN=

XDEBUG_VERSION=

[janosmiko]

Hi @smithi1!

Could you try to change RABBITMQ_VERSION to 3.9?

[smithi1]

I did test that while I was trying to sort get it to work. The result was identical.

[janosmiko]

Hi @smithi1 !

I added the build process for arm64 arch docker images.

Could you try to bootstrap an environment again?

[smithi1]

Many thanks for your help with this, @janosmiko. I deleted everything and started again and this time did not get the amqp error. RabbitMQ seems to be working. However, there is now a similar issue with Elasticsearch.

Bootstrap fails with: In SearchConfig.php line 81: Could not validate a connection to Elasticsearch. No alive nodes found in your cluster

When I run the image separately, I get:

% docker run rewardenv/elasticsearch:7.12
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Exception in thread "main" java.io.IOException: Cannot run program "/usr/share/elasticsearch/jdk/bin/java": error=0, Failed to exec spawn helper: pid: 242, exit value: 1
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1142)
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
    at org.elasticsearch.tools.launchers.JvmOption.flagsFinal(JvmOption.java:107)
    at org.elasticsearch.tools.launchers.JvmOption.findFinalOptions(JvmOption.java:81)
    at org.elasticsearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:38)
    at org.elasticsearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:135)
    at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:86)
Caused by: java.io.IOException: error=0, Failed to exec spawn helper: pid: 242, exit value: 1
    at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
    at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:313)
    at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1109)
    ... 6 more
% 

[janosmiko]

Hi @smithi1 !

Could you try again please? I added build process for elasticsearch/arm64 as well.

[smithi1]

This now appears fixed. There is a rather more subtle error now. The redis container is running, but not permitting connections:

Bootstrap gives:

read error on connection to redis:6379
                                      Error: exit status 1

When I try reward env exec redis redis-cli the command simply hangs.

As a note: for all of the reward and environment containers without an ARM binary, Docker Desktop has tagged them in the Containers listing with a warning: amd64: Image may have poor performance, or fail, if run via emulation.

UPDATE: I restarted the redis container, and was able to connect to it using the command above. Perhaps it had just got stuck somehow. Re-running bootstrap, and I will update again with the result.

FURTHER UPDATE: bootstrap displayed [Progress: 444 / 1673] Module 'Magento_Indexer': Running schema recurring... for a while, and then finished with the error above. Redis is now stuck again. I'm not sure my tinkering is adding much value so will stop for now :)

[janosmiko]

Hi @smithi1 !

I bumped Redis images to support ARM. Could you try again? Sorry for the step-by-step fixing, but I just have an Intel MBP and I can't test your case.

[smithi1]

Hi @janosmiko - no problem, I figured that this was probably the reason why we were taking an incremental approach!

With the new Redis image, the bootstrap completes. However, when I try to load the site in my browser, the content under /static (.png, .js, .woff, etc. files) is showing a lot of "failed to load resource" because of 502 errors. I assume this might be because of Varnish, but you would know better than me :)

[janosmiko]

I updated the Varnish containers too.

Could you retry it please @smithi1 ?

Note: only the following Varnish versions support ARM64:

• 6.0
• 6.5
• 6.6
• 7.0
• 7.1

Varnish 4.1 and 6.4 does not provide packages for ARM64.

[smithi1]

Thanks @janosmiko - I have switched the Varnish version to 6.6, and confirm that I am running the ARM native version. I am now getting the following in the browser, when I connect:

Error 503 Backend fetch failed

Backend fetch failed

Guru Meditation:

XID: 32779

Varnish cache server

I also have this, from the end of bootstrap:

INFO[2022-03-17T13:28:35Z] Syncing environment with mutagen...
Created session sync_z9gmwpJwuBXHnEWc4MWGWeyc9xDnOBKwMjOl2q1W15W

                                                   [ErrorException]
                    require(/var/www/html/vendor/ralouphie/getallheaders/src/getallheaders.php): failed to open stream: No such file or directory

followed by an exception trace. I suppose php-fpm might be next in our step-by-step, but I will also try deleting everything and doing it again from scratch.

[janosmiko]

Hi @smithi1 ,

I built the rest of the images (nginx, mariadb, php and more), to ARM. Could you try it again?

[smithi1]

Hi @janosmiko Thanks for that. I note that all of the environment containers are now ARM compatible, although not all of the reward svc containers.

I am now getting Varnish cache errors along the following lines.

Error 503 Backend fetch failed

Backend fetch failed

Guru Meditation:

XID: 18

Varnish cache server

The php-fpm container appears to be up and running, however.

I have connected to the varnish container and had a dig about. I found /etc/supervisor/conf.d/varnish.conf which seems to be how the Varnish service is being started. The first command in that file invokes gomplate to create /etc/varnish/default.vcl from a template /etc/varnish/default.vcl.template.

I found that /usr/local/bin/gomplate does not run. The container appears to be trying to run it as a shell script (error message comes from bash), although it's definitely a binary file. I assume that it is an incompatible binary, although the container doesn't contain the file command so I could not verify this. At any rate, due to this crash /etc/varnish/default.vcl is an empty file, which might explain why I can't get through to php.

[janosmiko]

Hi @smithi1 !

Thanks for the details, it helps a lot.

I changed the Dockerfiles to download gomplate binary directly instead of copying it from the hairyhenders/gomplate docker image. I believe the previously used docker copy method copied from the x86 docker image instead of the ARM. I hope downloading the binary directly will work.

Could you try again?

[smithi1]

hi @janosmiko - that version of gomplate doesn't work either - I tried a few in the nginx container, which at least doesn't crash, and this one worked - https://github.com/hairyhenderson/gomplate/releases/latest/download/gomplate_linux-armv6-slim.

I did a further experiment that might also be of interest. I tested this version of gomplate on an AWS t4g.nano instance. It ran fine, showing that t4g instances may be a reasonable proxy for a Docker container running on Apple M1. I would speculate that an ARM version of Reward might be able to run in a t4g.large (which has 8GB RAM). Anyway... :)

[janosmiko]

Hi @smithi1 !

First of all, thanks for the idea to use t4g instances in AWS. I already subscribed to m1 mac beta instances (in AWS) but sadly I didn't get the invitation yet.

I reached out the creator of gomplate, he will not fix it, however the non-slim version is working so I changed that.

I found that the libvmod-dynamic varnish module is sadly not working on m1mac (it's pretty weird, cause it's working in linux arm64). I decided to disable that when reward host is m1 mac.

Could you try again? And please let me know that if you face any other issues!

[smithi1]

@janosmiko It looks as though I have a working local environment now on M1.

Thanks so much for the excellent support for getting this working, and for creating it in the first place - it's a fantastic tool. Over time I think it would be good to migrate the core services across to M1 as well, as those containers are tagged in Docker Desktop with serious-looking warnings about poor performance and potential failures. However, it currently all seems to be working nicely!

Thanks again!

[janosmiko]

I'm really happy to hear that!

Over time I think it would be good to migrate the core services across to M1 as well, as those containers are tagged in Docker Desktop with serious-looking warnings about poor performance and potential failures.

That's definitely the next step, sadly most of those containers are built by external teams so that will be a bit larger effort to solve. Anyway, I close this issue now and I will open another one to track that.