search

nutanix / libvfio-user

framework for emulating devices in userspace
BSD 3-Clause "New" or "Revised" License
162 stars 51 forks source link

fix: server sample not marking dirty pages #748

Closed w-henderson closed 1 year ago

w-henderson commented 1 year ago

The server sample is supposed to demonstrate dirty page logging, but it was not marking dirty pages. This commit both adds client-side dirty page tracking for pages dirtied with vfu_sgl_write and server-side dirty page tracking for pages directly dirtied by the server using vfu_sgl_get/put.

tmakatos commented 1 year ago

Could you also add a check in the client that pages do get dirtied by the server?

w-henderson commented 1 year ago

Replaced by #753.

w-henderson commented 1 year ago

Could you also add a check in the client that pages do get dirtied by the server?

I'll do this separately.

w-henderson commented 1 year ago

Reopening this one to replace #753.

tmakatos commented 1 year ago

@w-henderson I'm not sure why you requested another review, you haven't addressed my comment here: https://github.com/nutanix/libvfio-user/pull/748#discussion_r1273634169

w-henderson commented 1 year ago

Sorry I missed that - I'm not sure I completely understand your comment, can you explain again what it is I need to do please?

tmakatos commented 1 year ago

Sorry I missed that - I'm not sure I completely understand your comment, can you explain again what it is I need to do please?

Sure I'll rephrase. In the server sample we try to demonstrate two things:

  1. Use of vfu_sgl_write.
  2. Tracking of dirty guest pages during live miration.

The server should not have to use vfu_sgl_mark_dirty when using vfu_sgl_write, it's the client's responsibility to track dirty pages in that case. Therefore, to demonstrate dirty page tracking in libvfio-user (point 2.), the server also needs to write to guest RAM directly by using vfu_sgl_get, write to memory, mark dirty (vfu_sgl_mark_dirty), etc. The client should track pages dirtied by:

  1. VFIO_USER_DMA_WRITE commands, and
  2. by explicitly requesting which pages the server dirtied by directly accessing guest RAM, via VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP.
jlevon commented 1 year ago

The server should not have to use vfu_sgl_mark_dirty when using vfu_sgl_write, it's the client's responsibility to track dirty pages in that case.

I'd like to add something to memory-mapping.md for that, to describe our expectations here. And a comment in the vfu_sgl_write() server code.

Perhaps even a comment in handle_dma_write() saying "here we'd mark these pages as dirty". Going further, actually keep track of dirty pages in client, both from shared memory regions and the handle_dma_write(), then assert it's what we expect later.

Therefore, to demonstrate dirty page tracking in libvfio-user (point 2.), the server also needs to write to guest RAM directly by using vfu_sgl_get, write to memory, mark dirty (vfu_sgl_mark_dirty), etc. The client should track pages dirtied by:

1. `VFIO_USER_DMA_WRITE` commands, and

2. by explicitly requesting which pages the server dirtied by directly accessing guest RAM, via `VFIO_IOMMU_DIRTY_PAGES_FLAG_GET_BITMAP`.