search

magento / magento-cloud-docker

All Submissions you make to Magento Inc. (“Magento") through GitHub are subject to the following terms and conditions: (1) You grant Magento a perpetual, worldwide, non-exclusive, no charge, royalty free, irrevocable license under your applicable copyrights and patents to reproduce, prepare derivative works of, display, publically perform, sublicense and distribute any feedback, ideas, code, or other information (“Submission") you submit through GitHub. (2) Your Submission is an original work of authorship and you are the owner or are legally entitled to grant the license stated above. (3) You agree to the Contributor License Agreement found here: https://github.com/magento/magento2/blob/master/CONTRIBUTOR_LICENSE_AGREEMENT.html
Open Software License 3.0
253 stars 191 forks source link

No ARM images available for development on Macbook Pro M1 CPU #326

Open asim-blueprintprep opened 3 years ago

asim-blueprintprep commented 3 years ago

Preconditions

  1. Macbook Pro M1 cpu
  2. php 7.3
  3. redis 5
  4. elasticsearch 7.5 (attempted)
  5. varnish (attempted)

Steps to reproduce

  1. ./vendor/bin/ece-docker build:compose
  2. docker-compose up -d
  3. multiple issues arise

Expected result

  1. docker containers should be created, and all start with no errors

Actual result

  1. elasticsearch issue
    1. Error: could not find libjava.so Error: Could not find Java SE Runtime Environment
    2. No ARM elasticsearch image provided by magento here: https://registry.hub.docker.com/r/magento/magento-cloud-docker-elasticsearch
    3. workaround, to use elasticsearch own here, https://registry.hub.docker.com/_/elasticsearch?tab=tags&page=1&ordering=last_updated
      1. e.g image: 'elasticsearch:7.8.1'
  2. varnish issue
    1. varnish_1 | Error: varnish_1 | Running VCC-compiler failed, signal 5 varnish_1 | VCL compilation failed
    2. There are no ARM varnish images available in magento or varnishes own docker repos

Is there a known way of getting cloud docker to boot on a new Macbook Pro?

Or is there a way to disable varnish, and have the docker just go straight to the frontend node to serve the site without a FPC?

shiftedreality commented 3 years ago

@asim-blueprintprep thank you for this request.

We've created an internal issue https://jira.corp.magento.com/browse/MCLOUD-7989 to get it fixed. However, any help is appreciated!

asim-blueprintprep commented 3 years ago

@shiftedreality thanks for repsonding so swiftly, do you know of a way of disabling varnish through docker compose, and configuring it so the site loads without varnish? As this is currently blocking development work

drpayyne commented 3 years ago

Hi @shiftedreality, is there any ticket to track the dependencies of cloud-docker and their compatibility with the Apple M1 chip? Just so that it's visible on what's pending for us Magento developers to start using cloud-docker on our Apple M1 computers.

shiftedreality commented 3 years ago

@asim-blueprintprep please use --no-varnish option

shiftedreality commented 3 years ago

@drpayyne I'm aware only about this one. Is there anything else we missing?

drpayyne commented 3 years ago

Not that I know of, @shiftedreality. Thanks for the update. Let me ask the community to see if anyone caught anything else.

asim-blueprintprep commented 3 years ago

Hi @shiftedreality That created great, and allowed me to get docker-compose up -d booting all containers, I then run bash ./mutagen.sh And then try to run docker-compose run --rm deploy cloud-deploy

However, it errors: [2021-06-16 14:48:55] ERROR: [132] Can't get version of elasticsearch: cURL error 6: Could not resolve host: elasticsearch (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)

I can't help but wonder, if this is because I've changed the elasticsearch image to not be a magento-elasticsearch image?

asim-blueprintprep commented 3 years ago

I believe the issue above is resolved by having elasticsearch in your hosts file, after adding that and rebuilding that works. However, because docker desktop is having to emulate so many amd64 packages, it runs out of memory

https://github.com/docker/for-mac/issues/5204

When running docker-compose run --rm deploy cloud-deploy

It reaches: [2021-06-17 08:49:41] INFO: Running setup upgrade. [2021-06-17 08:49:58] ERROR: [126] The command "/bin/bash -c "set -o pipefail; php ./bin/magento setup:upgrade --keep-generated --ansi --no-interaction | tee -a /app/var/log/install_upgrade.log"" failed. mmap() failed: [12] Cannot allocate memory

This is with 6GB RAM set available for docker desktop, I tried increasing it to 12GB, and the same error happened (with qemu actually using 16GB RAM - taken from top command)

So far: 1) varnish & elastic are the 2 containers that don't compile properly, even under emulation of amd64 2) docker runs out of memory trying to emulate all of the containers as ARM instead of amd64

I am out of ideas on how to progress this any further, without there being any ARM builds available

Thanks,

drpayyne commented 3 years ago

Hi @shiftedreality! Is there any update available for public view on this? :)

WingmanImd commented 2 years ago

Hey @drpayyne I have a working docker-compose.yml file in case you still need it. You'll need to keep the Dockerfiles in the project(I don't plan on pushing them to docker hub) though

drpayyne commented 2 years ago

Awesome @WingmanImd! Do you have this in your fork?

WingmanImd commented 2 years ago

@drpayyne no, I had to resort to manually editing the docker-compose.yml file after it's created then use modified versions of Dockerfiles for all images. Gonna fork it again and edit the Readme with the instructions later today

antoinette-mkwapatira-at-hanes commented 2 years ago

@WingmanImd Can you share a little more info about your approach to getting set up with the M1 chip?

AlexanderKopyl commented 2 years ago

Hello guys any one resolved this issue?

BaDos commented 2 years ago

Hi! We're going to continue work on this issue soon.

self-diagnostics commented 2 years ago

Is there any news on this topic?

MaximGns commented 2 years ago

@shiftedreality any progress here?

mustdobetter commented 2 years ago

@BaDos I head up C3, an Adobe Commerce partner agency in the UK, and this is something I and my team need. I'm keen to know if I can be of help to move this forward - we have lots of Docker experience.

mustdobetter commented 2 years ago

I've just found a commit adding support: 50c77b0dc30ea33b86fe8d02224342d6c9484e93. Thank you @BaDos! 🙌

I've successfully built and run the PHP 7.4 FPM image on ARM (MacBook Pro Max), and all seems to be working well. I'll be building and testing some of the others shortly and will feed back on any issues etc.

ferroapp commented 1 year ago

Hi, I wonder if there is any progress on this, we are stuck in the same situation, version 2.4..4, any help will be much appreciated

BaDos commented 1 year ago

Hi @ferroapp Now we have prepared Dockerfiles to build images for arm64/M1, but we do not have images on docker hub.

ferroapp commented 1 year ago

@BaDos that's nice, what version should we try?? 1.3.3?? thank you

BaDos commented 1 year ago

@ferroapp yes. you can use 1.3.3

mamarx commented 1 year ago

really happy about the ARM64 support added in 1.3.3. thank you guys!

we updated to 1.3.3 and built a new docker setup but still AMD64 images. what do we have to update to use the ARM64? thanks in advance!

BaDos commented 1 year ago

@mamarx docker hub contains only amd64 images, you need to build you own arm64 images using M1 machine or any arm64

mamarx commented 1 year ago

@BaDos thank you!

ga3003 commented 1 year ago

Is there any update on this issue? I am using the Mac Pro M2 and I am facing the same issue where I got a warning for ARM images and when the setup is complete, the URL is showing an error message of not reachable.

Please help, if someone has any hack to solve this.

BaDos commented 1 year ago

Hi @ga3003 Please, see comments above. Now you need to build arm64 images yourself.

ga3003 commented 1 year ago

Thanks, @BaDos.

pawelskowronek commented 1 year ago

Here is an example of how you can build arm64 images for Magento Cloud Docker 1.3.4 and Magento 2.4.5-p1:

  1. Clone magento cloud docker repository. 
    git clone https://github.com/magento/magento-cloud-docker.git
  2. Change current working directory to magento cloud docker. 
    cd magento-cloud-docker

  3. Build docker images.

    docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-php:8.1-cli-1.3.4 ./images/php/8.1-cli

docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-opensearch:1.2-1.3.4 ./images/opensearch/1.2

docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-nginx:1.19-1.3.4 ./images/nginx/1.19

docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-varnish:6.6-1.3.4 ./images/varnish/6.6
IgorVitol commented 1 year ago

@mamarx docker hub contains only amd64 images, you need to build you own arm64 images using M1 machine or any arm64

@BaDos Hi! Is there any plans to build the cross-platform images and push those directly to Docker Hub to prevent any manual "build" actions by every dev? Same name/tag would be used on all platforms letting docker to download the proper platform image automatically.

This should be as simple as configuring your build pipeline to use proper BuildKit in dockerx, eg docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t <username>/<image>:latest --push

Note: You can see example in varnish/redis official images, which has inbuilt cross-platform support: image

joeshelton-wagento commented 10 months ago

Any updates here? I'm able to locally build arm64 images. But, I can't utilize them without a manual modification of the docker-compose.yml file. (And changing permissions on the executable files copied to the image.) These manual modifications are not easy to figure out, and don't persist when updating the tool. So, not ideal for new Mac users.

JonasWijne commented 7 months ago

writing on jan 22 2024, is there any update?

drpayyne commented 7 months ago

I wonder what's stopping the Adobe team from developing and publishing this? How are the 100s of internal Adobe developers working on Magento on their MacBook? Either everyone has the old chips, or they're all running these containers on emulation, or their multi-platform images are proprietary. :)

Morgy93 commented 6 months ago

What a great developer experience in 2024 with latest technology. Makes me want to jump on Magento Cloud right away!

Or that's what I'd like to say if this issue would not exist. Any news?