search

nlewo / nix2container

An archive-less dockerTools.buildImage implementation
Apache License 2.0
552 stars 49 forks source link

Rootfs Issue with image deployed to AWS Fargate #98

Open dergraf opened 1 year ago

dergraf commented 1 year ago

Hello

I am not sure if this is an issue related to nix2container, devenv.sh, AWS Fargate or my wrong-doing (absolutely possible). I am using the container feature of devenv.sh which is based on nix2container.

AWS Fargate can't start a container due to a rootfs error:

CannotStartContainerError: ResourceInitializationError: container rootfs: fail to create /etc: mkdir /var/lib/two/volumes/xxxx/rootfs/etc: file exists

image

The issue can be reproduced using my gist https://gist.github.com/dergraf/10473c2f3c9c84ad9ea225f7ee3ae3b6 but requires a working AWS Fargate/ECR setup.

Despite the issue on AWS Fargate I can pull and run the image locally with Podman.

dergraf commented 1 year ago

In https://github.com/cachix/devenv/commit/00aed4abf0fdfe31ea0b2e557044e2e99006e178 the docker ca-certificates package gets added to the image by default. The package installs the CA certificates in /etc/... Resulting in a symlink pointing from /etc to the nix store. Apparently AWS Fargate tries to mount a folder to /etc which doesn't work and triggers the described issue. At this point I am not really sure who, where, how to fix the issue.

nlewo commented 1 year ago

Hello @dergraf

Is there any recommandation on how to build an image for fargate? Is it possible to run a standard container debian image for instance?

To debug further, i think it could be interesting to try running a minimal nix2container image:


nix run .#hello.copyTo SKOPEO-COPY-ARGS
``
dergraf commented 1 year ago

Hi!

Good point, a standard debian image is actually working... 🤯