search

project-machine / atomfs

Apache License 2.0
12 stars 6 forks source link

Cannot create an LXC OCI template with an atomfs rootfs #15

Open ariel-miculas opened 2 months ago

ariel-miculas commented 2 months ago 
$ sudo env PATH=$PATH build/src/lxc/tools/lxc-create --name my-atomfs-container -t oci -- --url oci:$HOME/work/cisco/stacker/oci:hello-stacker-squashfs
Copying blob bcfd1457bd9c skipped: already exists  
Copying blob 8d3a9c82f78d skipped: already exists  
Copying config 7e41ad5ab5 done   | 
Writing manifest to image destination
mfpath=/var/lib/lxc/my-atomfs-container/oci/blobs/sha256/510908f7430da725116f8abc73807c0d6be97cf8f9d2706571c9716a20f05a48 conf=/var/lib/lxc/my-atomfs-container/oci/blobs/sha256/7e41ad5ab5a2c56b99a1ecff25e711ef2534aa52657d743c8d9da5afca67a8a0
mediatype=application/vnd.stacker.image.layer.squashfs+zstd+verity
atomfs mount /var/lib/lxc/my-atomfs-container/oci:my-atomfs-container /var/lib/lxc/my-atomfs-container/rootfs
/usr/share/lxc/templates/lxc-oci: line 463: /var/lib/lxc/my-atomfs-container/rootfs/etc/hostname: Read-only file system
atomfs unmount /var/lib/lxc/my-atomfs-container/rootfs
lxc-create: my-atomfs-container: ../src/lxc/lxccontainer.c: create_run_template: 1628 Failed to create container from template
lxc-create: my-atomfs-container: ../src/lxc/tools/lxc_create.c: lxc_create_main: 318 Failed to create container my-atomfs-container

This is due to this commit which replaced the default writable mounts with default read-only mounts. The LXC template doesn't pass the --writeable mount option to atomfs.

We either change the default behavior of atomfs or we change the OCI LXC template. If we change the OCI template, there will still be older versions of LXC which won't work with atomfs.

There's also the PuzzleFS PR and it would be nice if we could keep the same MOUNT_HELPER semantics.

ariel-miculas commented 1 month ago

With the latest changes in [this lxc PR], creating an LXC container with at atomfs rootfs works, but when I start the container, it alternates between starting an erroring out:

❯ sudo env PATH=$PATH build/src/lxc/tools/lxc-create --name my-atomfs-container -t oci -- --url oci:$HOME/work/cisco/stacker/oci:hello-stacker-squashfs
Copying blob bcfd1457bd9c skipped: already exists
Copying blob 8d3a9c82f78d skipped: already exists
Copying config 7e41ad5ab5 done   |
Writing manifest to image destination
mfpath=/var/lib/lxc/my-atomfs-container/oci/blobs/sha256/510908f7430da725116f8abc73807c0d6be97cf8f9d2706571c9716a20f05a48 conf=/var/lib/lxc/my-atomfs-container/oci/blobs/sha256/7e41ad5ab5a2c56b99a1ecff25e711ef2534aa52657d743c8d9da5afca67a8a0
mediatype=application/vnd.stacker.image.layer.squashfs+zstd+verity
atomfs mount --writable /var/lib/lxc/my-atomfs-container/oci:my-atomfs-container /var/lib/lxc/my-atomfs-container/rootfs
atomfs unmount /var/lib/lxc/my-atomfs-container/rootfs
❯ sudo build/src/lxc/tools/lxc-start --name my-atomfs-container --logfile=/tmp/lxc-log --logpriority=debug --foreground /bin/sh

/bin/sh: can't access tty; job control turned off
/ #
❯ sudo build/src/lxc/tools/lxc-start --name my-atomfs-container --logfile=/tmp/lxc-log --logpriority=debug --foreground /bin/sh

lxc-start: my-atomfs-container: ../src/lxc/conf.c: run_buffer: 322 Script exited with status 1
lxc-start: my-atomfs-container: ../src/lxc/conf.c: lxc_setup_rootfs_prepare_root: 3989 Failed to run pre-mount hooks
lxc-start: my-atomfs-container: ../src/lxc/conf.c: lxc_setup: 4368 Failed to setup rootfs
lxc-start: my-atomfs-container: ../src/lxc/start.c: do_start: 1272 Failed to setup container "my-atomfs-container"
lxc-start: my-atomfs-container: ../src/lxc/sync.c: sync_wait: 34 An error occurred in another process (expected sequence number 3)
lxc-start: my-atomfs-container: ../src/lxc/start.c: __lxc_start: 2107 Failed to spawn container "my-atomfs-container"
lxc-start: my-atomfs-container: ../src/lxc/tools/lxc_start.c: lxc_start_main: 307 The container failed to start
lxc-start: my-atomfs-container: ../src/lxc/tools/lxc_start.c: lxc_start_main: 312 Additional information can be obtained by setting the --logfile and --logpriority options
❯ sudo build/src/lxc/tools/lxc-start --name my-atomfs-container --logfile=/tmp/lxc-log --logpriority=debug --foreground /bin/sh

/bin/sh: can't access tty; job control turned off
/ #
❯ sudo build/src/lxc/tools/lxc-start --name my-atomfs-container --logfile=/tmp/lxc-log --logpriority=debug --foreground /bin/sh

lxc-start: my-atomfs-container: ../src/lxc/conf.c: run_buffer: 322 Script exited with status 1
lxc-start: my-atomfs-container: ../src/lxc/conf.c: lxc_setup_rootfs_prepare_root: 3989 Failed to run pre-mount hooks
lxc-start: my-atomfs-container: ../src/lxc/conf.c: lxc_setup: 4368 Failed to setup rootfs
lxc-start: my-atomfs-container: ../src/lxc/start.c: do_start: 1272 Failed to setup container "my-atomfs-container"
lxc-start: my-atomfs-container: ../src/lxc/sync.c: sync_wait: 34 An error occurred in another process (expected sequence number 3)
lxc-start: my-atomfs-container: ../src/lxc/start.c: __lxc_start: 2107 Failed to spawn container "my-atomfs-container"
lxc-start: my-atomfs-container: ../src/lxc/tools/lxc_start.c: lxc_start_main: 307 The container failed to start
lxc-start: my-atomfs-container: ../src/lxc/tools/lxc_start.c: lxc_start_main: 312 Additional information can be obtained by setting the --logfile and --logpriority options