User stories: some thoughts about common use cases

[joh-ku]

I try to translate my knowledge about Jails into the world of OCI, so here are some (not too technical) thoughts.

Are container engines implementing the runtime-spec like Podman indented to serve as an alternative to Jail managers such as BastilleBSD? Like e.g. As a user, I want to use container engines such as Podman for general Jail management.

Some Jail managers implement the concept of thin and thick Jails, where a thin Jail, contrary to a thick Jail, shares a FreeBSD base image with other Jails. Is this something the runtime-spec should reflect in any form? Like e.g. As a user, I want to be able to share one FreeBSD base image with multiple containers.

Not sure if this belongs to the runtime-spec: As a user, I want to create reusable containers and publish/download them to/from a container registry. (e.g. to install certain applications like NGINX or PostgreSQL or create my own)

[samuelkarp]

Are container engines implementing the runtime-spec like Podman indented to serve as an alternative to Jail managers such as BastilleBSD?

I'm not very familiar with BastilleBSD, but from a quick look at the website I think that's a somewhat higher-level tool. The runtime spec is more concerned with a single container (jail) at a time; I'd consider it more as an alternative to jail(8).

Some Jail managers implement the concept of thin and thick Jails, where a thin Jail, contrary to a thick Jail, shares a FreeBSD base image with other Jails. Is this something the runtime-spec should reflect in any form?

The runtime spec doesn't really know about images or whether anything is shared. At a high level, a compliant OCI runtime consumes a "bundle" that consists of a JSON file describing how the container should be configured and a root filesystem (generally presented as a directory). Higher-level tools built on top of OCI runtimes (such as containerd, Docker, etc) will manage images and determine whether sharing occurs; containerd and Docker typically use some form of a copy-on-write backend to implement this (overlayfs being the most common on Linux, but on FreeBSD zfs can be used).

The image spec (which is separate from the runtime spec) describes how images are formatted.

Not sure if this belongs to the runtime-spec: As a user, I want to create reusable containers and publish/download them to/from a container registry. (e.g. to install certain applications like NGINX or PostgreSQL or create my own)

Publishing and downloading images (aka pushing and pulling) is standardized as part of the distribution spec.

An OCI runtime does not (typically) implement anything other than the runtime spec. Higher-level tools will use an OCI runtime and implement the image spec and/or distribution spec themselves.

Hopefully that helps!

[dfr]

This is mostly covered elsewhere in the image and distribution specification as @samuelkarp mentions. How the image is stored locally is somewhat dependent on the container engine's implementation. For instance podman on FreeBSD supports two storage drivers, ZFS which uses cloned datasets to make something like what I think you mean by 'thin jails' and VFS which just copies directory trees, which its sort of like a 'thick jail'.

I would also like to note that it is possible to avoid using images altogether although I think this is probably rarely used. With podman, you can uses the --rootfs argument to podman create and podman run to reference an existing directory tree:

# d=$(mktemp -d)
# tar -C / -cf - rescue | tar -C $d -xf -
# podman run -ti --rm --rootfs $d /rescue/sh
Cannot read termcap database;
using dumb terminal settings.
# /rescue/ls
dev etc rescue  var
# exit
# rm -rf $d

[joh-ku]

I'm not very familiar with BastilleBSD, but from a quick look at the website I think that's a somewhat higher-level tool. The runtime spec is more concerned with a single container (jail) at a time; I'd consider it more as an alternative to jail(8).

Thanks for the clarification. Yes, correct, BastilleBSD is a higher-level jail manager. My general thought was if, when we think about the future, e.g. podman in combination with an OCI-compliant runtime can serve as a tool for general purpose jail management (like e.g. BastilleBSD). From what I understood now, this can be assumed "yes". However, the runtime-spec itself, which this working group is about, will in that sense "only" form the foundation for high-level tooling (together with other specifications like the image and distribution spec). Hope this makes sense.

[joh-ku]

For instance podman on FreeBSD supports two storage drivers, ZFS which uses cloned datasets to make something like what I think you mean by 'thin jails' and VFS which just copies directory trees, which its sort of like a 'thick jail'.

Thin jails are basically read-only nullfs mounts, whereas thick jails independently maintain there own directory trees. For example:

/usr/local/bastille/releases/14.1-RELEASE /usr/local/bastille/jails/testjail/root/.bastille nullfs ro 0 0

The major advantage of thin jails is that upgrades of the base release across multiple jails become extremely simple (and fast).

[dfr]

I'm not very familiar with BastilleBSD, but from a quick look at the website I think that's a somewhat higher-level tool. The runtime spec is more concerned with a single container (jail) at a time; I'd consider it more as an alternative to jail(8).

Thanks for the clarification. Yes, correct, BastilleBSD is a higher-level jail manager. My general thought was if, when we think about the future, e.g. podman in combination with an OCI-compliant runtime can serve as a tool for general purpose jail management (like e.g. BastilleBSD). From what I understood now, this can be assumed "yes". However, the runtime-spec itself, which this working group is about, will in that sense "only" form the foundation for high-level tooling (together with other specifications like the image and distribution spec). Hope this makes sense.

The way I think about this is that the runtime specificiation, including whatever we agree on for a FreeBSD extension defines the interface between higher level tooling (e.g. Podman, containerd, CRI-O) to the low-level tooling (e.g., runj, ocijail) and clarifies the features offered by those low-level tools. This abstraction layer gives a lot of flexibility on how to implement the container and on Linux there is a wide variety of runtimes ranging from lightweight kernel-level virtualisation (FreeBSD equivalent is jail) to full virtual machines. This is similar to what you wrote above so I think we are on the same page.

[joh-ku]

The way I think about this is that the runtime specificiation, including whatever we agree on for a FreeBSD extension defines the interface between higher level tooling (e.g. Podman, containerd, CRI-O) to the low-level tooling (e.g., runj, ocijail) and clarifies the features offered by those low-level tools. This abstraction layer gives a lot of flexibility on how to implement the container and on Linux there is a wide variety of runtimes ranging from lightweight kernel-level virtualisation (FreeBSD equivalent is jail) to full virtual machines. This is similar to what you wrote above so I think we are on the same page.

Exactly, that sums it up perfectly. I think this holds great potential for the acceptance and adoption of FreeBSD and its capabilities.