Crate Addition Request: An epoll based IO event manager

[jiangliu]

Crate Name

vmm-epoll

Short Description

The vmm-epoll crate aims to implement an IO event manager based on Linux epoll. It's derived from the implementation from the firecracker project, but with several enhancements: 1) Provide a hash table based multi-thread safe manager, in addition to the vector based single-thread manager. 2) The 64-bit epoll_event.data field is split into two sub-fields (slot, user_data). 3) Dedicated IO event manager may be created for devices with heavy IO load.

Why is this crate relevant to the rust-vmm project?

The vmm-epoll crate be used to: 1) handle IO events for virtio devices/vhost devices. 2) handle OpenAPI requests. 3) handle IO events for legacy devices, such as serial. 4) optimize userspace vsock implementation.

[sameo]

@jiangliu Could this one live under the sys-util crate?

[jiangliu]

That's an option. Our current implementation still depends on the epoll crate instead of using the epoll wrapper from the vmm-sys-utils. On the other hand, it's not a simple wrapper over the linux epoll.

[sameo]

@jiangliu Yes, it's not going to be a simple wrapper. But otoh it's more of a utility than a VMM feature itself.

[acatangiu]

What is the design proposal for:

Dedicated IO event manager may be created for devices with heavy IO load.

Also do you plan to cascade epolls or use a single flat structure?

[jiangliu]

What is the design proposal for:

Dedicated IO event manager may be created for devices with heavy IO load.

Also do you plan to cascade epolls or use a single flat structure?

By design, it's a flat structure. We may create an epoll manager for each IO worker thread, and register different devices into different epoll manager. For example, we may use an epoll manager instance to replace the cascade epoll in vsock:)

[jiangliu]

Hi, @sameo @andreeaflorescu @sboeuf @acatangiu I have push the first implementation for proposal at my personal github repository at: https://github.com/jiangliu/vmm-epoll.git Please comment on the proposal:) We have unresolved discussion: whether to integrate this into the vmm-sys-utils crate or creating a dedicate crate. And the next step is to provide a multi-thread safe epoll manager implementation. Thanks!

[sandreim]

Hi,

I am also working on a epoll based IO event manager which is not based of any of the Firecracker code. We are trying to move from the EpollContext towards a more generic pattern of usage. The prototype implementation is based on a version of the Observer pattern that seems to work really well with Rust. The design is single threaded and supports cascaded epoll also.

The code is submitted in this PR: https://github.com/firecracker-microvm/firecracker/pull/1334 I also created a simple example app using it: https://github.com/sandreim/epolly/blob/master/src/main.rs

Thanks!

[sameo]

@sandreim @jiangliu @andreeaflorescu Since it seems there is a need for this and that it's actually bigger than just something that could live under vmm-sys-util, I think we should go ahead and create a crate for that one.

[andreeaflorescu]

Do we stick with the vmm-epoll name?

[sboeuf]

Sounds fine to me!

[sameo]

Do we stick with the vmm-epoll name?

Yes, sounds good to me.

[andreeaflorescu]

Done: https://github.com/rust-vmm/vmm-epoll

[andreeaflorescu]

Repository created. Closing this issue.