containerd / overlaybd

Overlaybd: a block based remote image format. The storage backend of containerd/accelerated-container-image.
Apache License 2.0
259 stars 58 forks source link

Userspace converted images may launch failed with "rwMode: dev" #285

Closed WaberZhuang closed 1 year ago

WaberZhuang commented 1 year ago

What happened in your environment?

Failed to create container, nerdctl run reports failed to create shim task: OCI runtime ...... no space left on device: unknown, but there's plenty of disk space.

What did you expect to happen?

create overlaybd container normally.

How can we reproduce it?

  1. Set snapshotter config rwMode field to "dev"
  2. Convert an image by userspace convertor and use option --mkfs (This option is turned on by default after release v1.0.0)
  3. Do rpull and run

What is the version of your Overlaybd?

What is your OS environment?

centos 7.8 (kernel-3.10)

Are you willing to submit PRs to fix it?

WaberZhuang commented 1 year ago

This is a resolved issue (fixed in #283) and I opened it just for the conclusion. Thanks to @jinteng123 who reported this bug.

Cause

The cause of the problem is that userspace convertor went wrong at step 'mkfs': it neglected to mark inode 1 in use in the bitmap. Inode 1 is a reserved inode, which is used to hold defective blocks.

What will happen

In simple words, the converted image will have some EXT4-fs errors. This may behave differently on different kernels, some will still work fine and some will just quit. Opening the file system as read-only should avoid this problem, so this issue will only happen if snapshotter is set up with "rwMode: dev" (rw file system) instead of "rwMode: overlayfs." (ro file system + overlayfs upper, default)

Any kernel version after this commit should be able to 'still work fine'.

Note

The converted layers before and after this fix differ from the digest (content has changed). If you are using layer deduplication, clearing the deduplication database is required.