canonical / lxd

Powerful system container and virtual machine manager
https://canonical.com/lxd
GNU Affero General Public License v3.0
4.28k stars 914 forks source link

virtfs-proxy-helper for 9p filesystem shares deprecated in QEMU 8.1 #12828

Open tomponline opened 5 months ago

tomponline commented 5 months ago

The 9p proxy filesystem backend driver has been deprecated and will be removed (along with its proxy helper daemon) in a future version of QEMU. Please use -fsdev local or -virtfs local for using the 9p local filesystem backend, or alternatively consider deploying virtiofsd instead.

The 9p proxy backend was originally developed as an alternative to the 9p local backend. The idea was to enhance security by dispatching actual low level filesystem operations from 9p server (QEMU process) over to a separate process (the virtfs-proxy-helper binary). However this alternative never gained momentum. The proxy backend is much slower than the local backend, hasn’t seen any development in years, and showed to be less secure, especially due to the fact that its helper daemon must be run as root, whereas with the local backend QEMU is typically run as unprivileged user and allows to tighten behaviour by mapping permissions et al by using its ‘mapped’ security model option.

Nowadays it would make sense to reimplement the proxy backend by using QEMU’s vhost feature, which would eliminate the high latency costs under which the 9p proxy backend currently suffers. However as of to date nobody has indicated plans for such kind of reimplementation unfortunately.

https://www.qemu.org/docs/master/about/deprecated.html#fsdev-proxy-and-virtfs-proxy-since-8-1

LXD uses virtfs-proxy-helper to isolate 9p filesystem shares to a user namespace to allow custom UID/GID mapping. This is required for the LXD multi-user mode to function safely.

LXD already supports exporting filesystem volumes to VMs using virtiofs, and uses virtiofsd in a user namespace too. However 9p filesystem shares are the only type of filesystem share that can be live migrated (unlike virtiofs shares). Additionally some guest OSes do not support virtiofs and only support 9p.

We will need to explore whether we can use the 9p local backend and potentially have to start QEMU itself in a user namespace, although that will almost certainly cause other issues we will need to try and workaround.

Other possibilities include dropping 9p filesystem share support entirely and use only virtiofs, but this would then mean guest OSes without virtiofs support wouldn't be able to access the agent config drive to start the LXD agent - so this would need some alternative solution (and ISO drive export perhaps?), and it would also mean that all filesystem volume shares would be disabled for guests that could be live migratable.

tomponline commented 3 months ago

Seeing these warnings in LXD VMs now:

qemu-system-x86_64:/var/snap/lxd/common/lxd/logs/block-test2/qemu.conf:273: warning: '-fsdev proxy' and '-virtfs proxy' are deprecated, use 'local' instead of 'proxy, or consider deploying virtiofsd as alternative to 9p