ERL-985: support new linux io_uring interface

erlang / otp

Erlang/OTP

http://erlang.org

Apache License 2.0

11.38k stars 2.96k forks source link

ERL-985: support new linux io_uring interface #4052

Open OTP-Maintainer opened 5 years ago

OTP-Maintainer commented 5 years ago

Original reporter: oshanz Affected version: Not Specified Component: Not Specified Migrated from: https://bugs.erlang.org/browse/ERL-985

Would Erlang be able to benefit from leveraging the new kernel polling interface that was just released with Linux kernel version 5.1? 

https://lwn.net/Articles/776703/
http://kernel.dk/io_uring.pdf

I wanted to open a discussion on whether we could (and should) add transparent support for this on supported linux kernels.

OTP-Maintainer commented 5 years ago

lukas said:

I've looked at it before and it could be a useful interface, but it would require a lot of work as it does not fit into the current polling model that much of the I/O system within erts uses today.

Possibly the best way to use this would be to handle the io ring directly in the nif/driver that needs the functionality instead of involving a central I/O handler for all types of events.

OTP-Maintainer commented 4 years ago

JIRAUSER15503 said:

I'm C# programmer by profession and I've been using Elixir for my own projects and I came across this Twitter post about io_uring [23% more throughput and 74% less latency with ASP.NET on Linux using io_uring? Yes, please!|https://twitter.com/tkp1n/status/1270010546205659139]. Then I tried to find out if OTP already supported io_uring and found this issue. It seems there is lot benefit for using io_uring with Linux kernel 5.7+. Here is [https://gist.github.com/sebastienros/82f5dd4ef1560b793574f3c7bd8dc656] Microsoft testing it on their machine.

oshanz commented 3 years ago

just saw this news and looked for the erlang support. I don't know how useful this feature will be to the erlang/elixir eco-system.

Linux 5.15 I/O Can Achieve Up To ~3.5M IOPS Per-Core https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.15-IO-3.5M-IOPS

update: 25 October 2021 Newest Linux Optimizations Can Achieve 10M IOPS Per-Core With IO_uring https://www.phoronix.com/scan.php?page=news_item&px=Linux-IO_uring-10M-IOPS

heri16 commented 2 years ago

Not sure why this is low priority considering that Whatsapp have been complaining about epoll vs kqueue performance since 3 years back: https://youtu.be/FJQyv26tFZ8

vans163 commented 2 years ago

This would be useful not only for TCP/Net/UDP but for filesystem reads/writes.

Also it seems TCP support for IO_uring is well wired now. With modern NICS it looks like a big win.