tokio-rs / io-uring

The `io_uring` library for Rust
Apache License 2.0
1.19k stars 131 forks source link

Question about the implementation of tcp_echo #22

Closed CarterLi closed 4 years ago

CarterLi commented 4 years ago

I did some benchmark about your echo server example, and got very good results. Well done man! The performance of rio is a disaster.

I'm curious about the implementation, but I'm not familiar with rust. Do you use poll before accept? What method are you using? recv/send or read/write? Do you use fixed buffers and files? IOSQE_IO_LINK?

quininer commented 4 years ago

Thanks, I didn't use any io_uring specific tricks, I created this exmaple to compare with ritsu.

I don't use poll before accept, but use poll before readv, in order to better share buffer; I use readv and writev because they work in stable kernel; I did not use fixed opcode and IOSQE_IO_LINK.

CarterLi commented 4 years ago

Thanks, I didn't use any io_uring specific tricks, I created this exmaple to compare with ritsu .

I don't use poll before accept, but use poll before readv, in order to better share buffer; I use readv and writev because they work in stable kernel; I did not use fixed opcode and IOSQE_IO_LINK.

So nothing special

in order to better share buffer

Well I don't agree with the idea you specified in the comment. Although you have polled before readv, multiple readv operations can still be run in parallel, only the readv becomes very short, short doesn't mean that it doesn't need time to run.

我不同意那个帖的观点。就算你在readv之前先poll了一把,多个readv仍然可以并发执行。先poll一把只意味着readv不会阻塞,但非阻塞操作执行仍然需要时间。

你想,磁盘I/O是不需要poll的,但是对于io_uring来说磁盘I/O仍然是并发执行的,设计就是如此。

Even so, the grouped two operations readv and writev are not atomic, you can't expect writev is run before other readv is run for the same buffer.

退一步说,就算在poll之后readv是原子性的,但是readvwritev这两个操作加起来肯定不是原子性的。你不能假设在前一个readv执行之后就不会有别的readv操作插进来。所以这个缓冲区还是有可能被其他readv覆盖掉

quininer commented 4 years ago

You misunderstood what he means. Actually we have a buffer pool, and we only take the buffer from pool after fd is ready.

Looking at ritsu's high-level code may be clearer.

see https://github.com/quininer/ritsu/blob/master/examples/tcp_echo.rs#L30

CarterLi commented 4 years ago

所以你是在poll之后选择一个空闲的buffer用是吧

quininer commented 4 years ago

Yes.

CarterLi commented 4 years ago

我的代码,甚至用了 fixed_files & fixed_buffers,有时候运行起来也没你的代码快 😭

https://github.com/CarterLi/io_uring-echo-server/blob/switch/io_uring_echo_server.c#L122

我怀疑是不是 liburing 本身有性能瓶颈。。。

quininer commented 4 years ago

我們在 matrix 私聊吧。

CarterLi commented 4 years ago

ok