search

nutanix / libvfio-user

framework for emulating devices in userspace
BSD 3-Clause "New" or "Revised" License
164 stars 51 forks source link

Pass server->client command over a separate socket pair #762

Closed mnissler-rivos closed 1 year ago

mnissler-rivos commented 1 year ago

This pull request changes the VFIO-user protocol to optionally use a separate pair of sockets for server->client commands. Doing so simplifies command / reply processing for the case where both server and client send commands concurrently, which has proven problematic per issue #279.

Included are the functional changes, new pytests, as well as protocol specification updates.

mnissler-rivos commented 1 year ago

Rebased after https://github.com/nutanix/libvfio-user/pull/763 was merged.

mnissler-rivos commented 1 year ago

Now that I understand you're squashing, let me know whether I should break out any of the individual commits here into additional separate PRs - my intention was to make small self-contained commits in order to simplify reviews and keep the history clean for folks coming after us.

mnissler-rivos commented 1 year ago

Btw. I think it makes sense to break out https://github.com/nutanix/libvfio-user/pull/762/commits/4025f6e6acd6a32500d6dc4e3e9bfe5f8528dff0 into a separate MR so the mechanical s/sock/client/ changes don't clutter this MR up?

jlevon commented 1 year ago

Btw. I think it makes sense to break out 4025f6e into a separate MR so the mechanical s/sock/client/ changes don't clutter this MR up?

yep

mnissler-rivos commented 1 year ago

As discussed, I have now broken out various bits and pieces into separate merge requests:

https://github.com/nutanix/libvfio-user/pull/772 - Introduce client object in python tests https://github.com/nutanix/libvfio-user/pull/773 - Replace protocol header flags bit field with mask https://github.com/nutanix/libvfio-user/pull/774 - Prepare python test helpers for receiving commands https://github.com/nutanix/libvfio-user/pull/775 - Describe the twin-socket feature in the spec

And there's another one that we haven't discussed yet. This is preparation work for the server version capabilities to include the index of the socket file descriptor in the ancillary data array:

https://github.com/nutanix/libvfio-user/pull/771 - Construct server capabilities using json-c

I think it'll be best if we work through these preparatory MRs first, then I can rebase this one eventually. Or I can alternatively update this one to the final version now - let me know if you have a preference.

I will also go through the comment threads here and resolve what has been addressed already.