The current socket transport code we have in tran_sock has a few issues:
it is sometimes non-blocking (#306) even when it shouldn't be, both for reads and for writes
client->server messages requiring a response don't react well if there's a server->client unrelated message in between (#279)
polling for any new requests on the socket (get_request_header()) path requires a system call
In general, in non-blocking mode, we should never get "stuck" in vfu_run_ctx(): if we can't do IO to the socket, we need to remember our state somehow, and return to the vfu_run_ctx() caller. Potentially we could do this by re-using the "pending message" code; this will require a bunch of refactoring so we only read/write from the socket in "known good" places we are able to return to.
As a first phase, we could implement an io_uring transport that's like tran sock, but doesn't require the system call. But it should be designed such that it's restartable to account for the future usage above.
As io_uring support is not in every kernel version, we should use it automatically, but fall back if we don't find the features we need.
One stumbling block is that SPDK's epoll_wait() loop doesn't look for writable events, so if we're blocked on a socket write, we would be stuck. We'll need to consider that.
vfu_dma_read/write(), as APIs, need some complete reworking. this is lower priority.
The current socket transport code we have in tran_sock has a few issues:
In general, in non-blocking mode, we should never get "stuck" in vfu_run_ctx(): if we can't do IO to the socket, we need to remember our state somehow, and return to the vfu_run_ctx() caller. Potentially we could do this by re-using the "pending message" code; this will require a bunch of refactoring so we only read/write from the socket in "known good" places we are able to return to.
As a first phase, we could implement an io_uring transport that's like tran sock, but doesn't require the system call. But it should be designed such that it's restartable to account for the future usage above. As io_uring support is not in every kernel version, we should use it automatically, but fall back if we don't find the features we need.
One stumbling block is that SPDK's epoll_wait() loop doesn't look for writable events, so if we're blocked on a socket write, we would be stuck. We'll need to consider that.
vfu_dma_read/write(), as APIs, need some complete reworking. this is lower priority.