search

rust-vmm / vm-memory

Virtual machine's guest memory crate
Apache License 2.0
299 stars 97 forks source link

Add replaceable-mmapped bitmap support #264

Closed germag closed 6 months ago

germag commented 9 months ago

Summary of the PR

The vhost user protocol supports live migration, and during live migration, the vhost-user frontend needs to track the modifications the vhost-user backend makes to the memory mapped regions, marking the dirty pages in a log (i.e., a bitmap).

If the backend has the VHOST_USER_PROTOCOL_F_LOG_SHMFD protocol feature it will receive the VHOST_USER_SET_LOG_BASE message with a file descriptor of the dirty-pages log memory region. This log covers all known guest addresses, and must be manipulated atomically.

For further info please see https://qemu-project.gitlab.io/qemu/interop/vhost-user.html#migration

This commit adds support for creating an atomic-memory-mapped bitmap, and being able to replace it in runtime. The vhost user protocol does not specify whether the previous bitmap is still active after replying to the VHOST_USER_SET_LOG_BASE message, so we use an RwLock to be sure that the in-flight requests are using the new bitmap after the message reply.

This version is currently being used in a version of virtiofsd that supports migration: https://gitlab.com/virtiofsd-live-migration

and is intended to be used as: https://github.com/rust-vmm/vhost/blob/05c4efa6b04a44a23092d9439fab8385b9148394/crates/vhost-user-backend/src/handler.rs#L616

Note to reviewers

This PR is intended to be as unobtrusive as possible, but in a future PR, I think we can make some improvements to eliminate code duplication: 1- Adding a parameter to NewBitmap::with_len() with the starting address of the memory region, Thus the region_start_addr: GuestAddress and region_len: GuestUsize parameters of BitmapMmap::from_file() could be removed.

2- move Bitmap::split_at() to its own trait because we cannot implement it for AtomicBitmapMmap (and I can get rid of BitmapReplace::InnerBitmap)

3- Make AtomicBitmap::new() take the bitmap's memory as parameter with a type bounded to Index (and a len() method). So, the code in AtomicBitmapMmap would be minimum.

(there is more duplicate code that could be unified, since all the mmap code here, and in mmap_unix and mmap_xen do similar things, but I think that is beyond the scope of this PR.)

germag commented 9 months ago

/cc @slp /cc @stefano-garzarella

germag commented 9 months ago

Rebase

germag commented 9 months ago

/cc @XanClic

germag commented 6 months ago

The bitmap support has been moved to the vhost-user-backend crate. So, I'm closing this PR in favor of https://github.com/rust-vmm/vhost/pull/206