How to use a block device on host as a container's rootfs?

kata-containers / runtime

Kata Containers version 1.x runtime (for version 2.x see https://github.com/kata-containers/kata-containers).

https://katacontainers.io/

Apache License 2.0

2.1k stars 377 forks source link

How to use a block device on host as a container's rootfs? #3132

Closed mgjrr closed 3 years ago

mgjrr commented 3 years ago

I want to replace the 9pfs by a virtual block device(created by vfio-mdev), if there any way to achieve it? And can you tell me how to straight forward some hypervisor commands(qemu) in docker cli? For example, run docker run ... --straight_forward "-foo bar" will create a process qemu-system-x86_64 ... -foo bar

I would appreciate getting any pointers on this. Thanks in advance.

devimc commented 3 years ago

@mgjrr how did you create the device? (we need more details) try mounting the device and passing it using docker -v

jodh-intel commented 3 years ago

The Kata configuration file (configuration.toml) allows certain hypervisor options ("knobs") to be modified. These "knobs" should be sufficient for 99.99% of use cases. But if you really want to specify arbitrary hypervisor options you could do the following (but it's not recommended):

Create a shell script that modifies (by looking at $@) or sets the hypervisor options you need.
Make sure the script ends with something like: exec qemu-system-x86_64 $new_args.
Modify your configuration.toml so that hypervisor.qemu.path specifies the path to you script rather than the real hypervisor binary.

However, that would obviously be very fragile since your script would be hypervisor version specific, Kata version specific and also Kata configuration specific.

Also, have you looked at Kata 2.x which doesn't use 9p?

mgjrr commented 3 years ago

@devimc I create the device by a trival vfio-mdev routine Yes, by this command I can use the virtual device in container docker run -ti --runtime kata-qemu --cap-add=ALL --rm -v /dev:/dev --device /dev/vfio/73 centos/tools bash But this still use the host's memory as storage.

devimc commented 3 years ago

@kata-containers/redhat any thoughts?

fidencio commented 3 years ago

This issue is being automatically closed as Kata Containers 1.x has now reached EOL (End of Life). This means it is no longer being maintained.

Important:

All users should switch to the latest Kata Containers 2.x release to ensure they are using a maintained release that contains the latest security fixes, performance improvements and new features.

This decision was discussed by the @kata-containers/architecture-committee and has been announced via the Kata Containers mailing list:

If you believe this issue still applies to Kata Containers 2.x, please open an issue against the Kata Containers 2.x repository, pointing to this one, providing details to allow us to migrate it.