search

nutanix / libvfio-user

framework for emulating devices in userspace
BSD 3-Clause "New" or "Revised" License
166 stars 51 forks source link

VFIO does not notify MUSER for DMA unmap events #50

Closed tmakatos closed 3 years ago

tmakatos commented 4 years ago

Becasue we no longer ping pages using vfio_pin_pages, VFIO doesn't sent MUSER DMA unmap events. This causes problems to libmuser because we get overlapping DMA regions we don't know how to properly handle, and also it's a resource leak. One way to deal with this is to hack VFIO blindly send the DMA unmap event (even if the driver hasn't pinned any page from that DMA region): https://www.redhat.com/archives/vfio-users/2020-February/msg00016.html.

Another way would be to not to use this hack and implicitly unmap the previous regions clobbered by the new region. The problem of this approach is that there can be leaks and handling of overlapping regions can be complicated. We need to check to see how VFIO handles overlapping regions.

tmakatos commented 4 years ago

I tried adding two overlapping DMA regions (0x000-0x2000 and 0x1000-0x3000) and this is failed with -EEXIST by vfio_dma_do_map in vfio_iommu_type1.c; the DMA map event doesn't reach MUSER at all. So the solution of implicitly unmapping existing regions could work, apart from the possibility of a leak. The possibility of a leak could be mitigated by unmapping the fd it hasn't been used for some time and then remapping it on demand.

tmakatos commented 3 years ago

We've switched to vfio-user and have dropped muser.ko.