search

cifsd-team / ksmbd

ksmbd kernel server(SMB/CIFS server)
152 stars 23 forks source link

ksmbd and io_uring ? #435

Closed Andy2244 closed 2 years ago

Andy2244 commented 4 years ago

Just wondering whats your opinion on io_uring and liburing, regarding ksmbd?

namjaejeon commented 4 years ago

Ah, I have known io_uring was recently applied to samba. I don't know how AIO is useful in smb server yet. Maybe there's a reason to ask this.

Andy2244 commented 4 years ago

here is what i checked, last link has benchmarks: https://wiki.samba.org/index.php/Samba_4.12_Features_added/changed#.27io_uring.27_vfs_module https://lwn.net/Articles/810414/ https://kernel.dk/io_uring.pdf https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.6-IO-uring-Tests

namjaejeon commented 4 years ago

Yes, ksmbd doesn't even support AIO, and don't know how there is an usage for this. to work AIO, client do also support it together ?

namjaejeon commented 4 years ago

ksmbd I/O use workqueue, i.e. I/O is processed in parallel asynchronously. So I guess io_uring is not needed for ksmbd.

Andy2244 commented 4 years ago

ksmbd I/O use workqueue, i.e. I/O is processed in parallel asynchronously. So I guess io_uring is not needed for ksmbd.

Thats what samba and many other projects also did, yet if i understand this correctly the overhead to setup/manage the threads/async i/o is what makes io_uring preferable over older kernel AIO or handcrafted userspace thread solution's. As i understand it the more overlapping async i/o operations you have the better io_uring performs, compared to other approaches. From my tests with the new samba4 vfs_uring module on openwrt+decent router+ssd the gains are minimal for normal sequential operations or light 4 AIO OP's on a single file. Yet as Phoronix has shown the gains get way better if you try something like accessing a file backed database with many I/O requests or have them spread out on multiple files. Similar if you have a heavy multi-user environment, that need to simultaneously access some shared data.

So i think for us (openwrt) users its not really important and just a nice extra knob to test, while for corporate use cases this may solve some specific workloads or even outperform NFSv4 setups.

PS: Just wanted to ask for your opinion, since i see liburing support added rapidly to many projects.

neheb commented 3 years ago

@Andy2244 is ksmbd slower than samba?

namjaejeon commented 3 years ago

ksmbd's performance is better than samba(4.7.6), It is even more different in the latest version of Samba.

See:

  1. https://www.snia.org/sites/default/files/SDC/2019/presentations/SMB/Jeon_Namjae_Lee_Hyunchul_Introduce_New_In-kernel_SMB3_Server_Called_CIFSD.pdf
  2. https://lists.samba.org/archive/samba-technical/2019-September/134200.html

I hope someone other than me to measure and compare samba and cifsd performance. Including io_uring applied version...

Neustradamus commented 3 years ago

@Andy2244: What do you think about performance?

Andy2244 commented 3 years ago

@Andy2244: What do you think about performance?

Its a bit tricky, but io_uring is winning over the linux world and shows decent performance increases in every application it is deployed. The main improvements are in thread heavy, deep io-queue and multi-segment file workloads, which is more a enterprise/heavy NAS use case side of things. I cant say much since ksmbd is already in kernel space and i'm not a linux i/o coder, so cant judge ksmbd+io_uring vs samba+io_uring.

Yet, there is a stable uringlib for easy integration, so i think its worth a try, since it should also have coding/maintenance advantages over the old threaded i/o calls/framework.