search

eddmann / serverless-fargate

Maintain long-running Fargate ECS tasks within your Serverless project
53 stars 22 forks source link

Not a bug! Wondering how you handle mixed dev. env's between Mac M-chip and x86... #47

Closed QAnders closed 4 months ago

QAnders commented 1 year ago

We've run into an issue as more developers are getting newer MacBooks with the M processor running ARM, while we also have Linux (Ubuntu) and Windows 11 (but all development is done through WSL2 (Ubuntu).

The Linux and Windows Laptops (and older MacBooks) are all Intel based, and thus we have always deployed to Fargate as x86 and never given it much thought but now we got issues when the newer MacBooks try to deploy and the deployment is "stalled" and hangs forever.

After investigating I found that Fargate gave the very helpful error message on the task as Exit code: 1, which led me to Stackoverflow and the fact that since the default in deploying is actually x86 which, obviously the M processors can't build.

What surprises me here, is that Docker doesn't complain, it happily builds an image that can't be run...

We started looking into the details and if we set architecture: ARM64 in serverless.yml the M MacBooks can deploy fine, and the container also runs fine in Fargate.

Then comes the problem, of course, that if we set architecture: ARM64 "hard" in serverless.yml the Linux and Windows/WSL guys can't deploy. Or rather they can try, but will end up in an infinite loop between Fargate and Cloudformation...

This now leads to that we have to change the architecture flag depending on who deploys (from what machine) which, of course will be forgotten and we've got the infinite loop and have to go and "kill" the Fargate deploy manually to get out of it (circuit breaker doesn't kick in here it seems).

Ideally we should deploy through a CI/CD tool, but we are using CodeShip (Pro) and they don't support running "Docker in Docker" so we can't build the image at deploy and we are stuck with deploying "manually". We are looking into using GitHub Actions instead, but haven't gotten there yet...

So, anyone else run into this... and how do you handle it (unless you use a CI/CD tool)?

jeremygiberson commented 10 months ago

FWIW: We're generally able to cope with this by always compiling our containers for AMD64 architecture and having our mac (apple silicon) devs run it via rosetta 2 feature in docker https://www.docker.com/blog/docker-desktop-4-25/

Rosetta for Linux GA: Docker now supports running x86-64 (Intel) binaries on Apple silicon with Rosetta 2. It’s no longer an experimental feature but a seamlessly integrated component of Docker Desktop.