[RFE] Consider allowing wnbd-client to setup nbd connection via AF_UNIX unix sockets

[hgkamath]

Description:

Consider the possibility of also allowing AF_UNIX unix-socket connections to a nbd-server also (maybe also rbd-client, if the rbd-protocol also has the same ability)

Since Windows-10 1803 Windows allows for usable AF_UNIX unix-sockets.
AF_UNIX sockets can be used when the processes (storport-wnbd-client and nbd-server) are local.
An emulated socketpair() is also said to be do-able. (but some advanced things like abstract unix-sockets are not do-able) Advantages include:

• A performance benefit is to be expected as AF_UNIX sockets avoid the network packetization overhead that happens over TCP/IP layers. Peers communicate over unix-socket at the session-layer.
• avoids exposing the nbd-server tcp port over the network
• filesystem permissions safeguard the socket.
• TCP connections need layer-4 ACK-keep-alives going back and forth.

References

• Since 20171219, Insider Build 17063, Microsoft has added AF_UNIX to windows sockets. https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/
• Since 20220126, gnome/glib/gio-2.71.1 has AF_UNIX (current version 2.76.0 as of 20230312) https://gitlab.gnome.org/GNOME/glib/-/issues/2487
• Since 20210607, afunix.h may already be bundled with mingw-w64 https://www.mail-archive.com/mingw-w64-public@lists.sourceforge.net/msg18854.html
• some discussion on the feature : make AF_UNIX sockets on Windows work
  https://github.com/microsoft/WSL/issues/4240

Misc

• similar type bug: opengl over spice/qemu/virt-viewer can also benefit from AF_UNIX. (development, which has been inching along, precedes my filing of the issue) https://gitlab.freedesktop.org/spice/spice/-/issues/76

[petrutlucian94]

Hi,

Thanks for sending this feature request.

The NBD client functionality is currently implemented at the WNBD driver level. However, the AF_UNIX compatibility layer (built around named pipes as far as I remember) can't easily be used by Windows drivers.

If/when we move the nbd client to userspace (e.g. as part of libwnbd or a separate lib), then we can easily support AF_UNIX. I can't provide an ETA though.

As for RBD, it's no longer using NBD as an IO channel. We went with a different mechanism a while ago, using DeviceIOControl to retrieve IO requests and submit IO replies. It's much more efficient, doesn't require tcp sockets and it allows us to support SCSI features that are not covered by the NBD protocol (e.g. persistent reservations, read FUA, etc).

[hgkamath]

Just FYI, my understanding is that there used to be a userspace compatibility layer based on named pipes. But this AF_UNIX feature goes more native than that. This AF_UNIX feature is implemented by Microsoft as a windows kernel driver. Its a partial implementation that gets unix-sockets, but not socketpair or abstract-unix-sockets. But, if compatibility layers are still required, such as wanting socketpair, then as you say one might need to goto userspace, So I'm hoping compatibility layers would not be required.
The downside is the feature won't work on older Win10 versions, so one would need to check win10 version and turn-off feature if the version is older than 1803. Ref: misc mentions in other projects, some in favor, some not

• neovim https://github.com/neovim/neovim/issues/11363
• libuv https://github.com/libuv/libuv/issues/2537
• postgres https://www.postgresql.org/message-id/20180705001640.GA297630%40rfd.leadboat.com
• mozilla IPC https://bugzilla.mozilla.org/show_bug.cgi?id=1489860
• cygwin https://cygwin.com/pipermail/cygwin-developers/2021-April/012112.html

Like spice-opengl, storport-wnbd-nbd is a use-case where every bit of throughput gained matters.