search

gluster / glusterfs

Gluster Filesystem : Build your distributed storage in minutes
https://www.gluster.org
GNU General Public License v2.0
4.73k stars 1.08k forks source link

io_uring support in glusterfs main branch #1398

Closed perrynzhou closed 4 years ago

perrynzhou commented 4 years ago

@itisravi @mykaul @mohit84 @amarts @hari which version for glusterfs support io_uring. we want to try testing glusterfs with io_uring? at same time,i found a io_uring path for glusterfs, details show as https://review.gluster.org/#/c/glusterfs/+/24350/ but that not exists in any branch og glusterfs?

itisravi commented 4 years ago

@perrynzhou I have been running tests internally with that patch (with some modifications). So far I am getting similar numbers when I run iozone, fio etc. on my setup. Are you interested in testing it by applying the patch and doing a source install? If yes, I can refresh my patch to reflect my modifications.

itisravi commented 4 years ago

@perrynzhou You can use version https://review.gluster.org/#/c/glusterfs/+/24350/4 of the patch to compile and install on master. Steps recommended for testing:

  1. Create a distribute-replicate volume
  2. gluster vol set $volname storage.linux-io_uring off
  3. Start the volume and fuse mount it
  4. Run your performance tests from fuse mount and note results.
  5. Stop the volume and umount it.
  6. gluster v set $volname storage.linux-io_uring on
  7. start the volume and remount.
  8. Run your performance tests from fuse mount and note results.

Compare results from 4 and 8. Note stopping and unmounting the volume between runs ensures there is no caching.

perrynzhou commented 4 years ago

Note

@itisravi yes ,i already download this path ,at same time merged io_uring path to local master source ,next we will compile that,and test performance

itisravi commented 4 years ago

@perrynzhou Thanks! Please do make sure you downloaded the latest (v4) of the patch.

itisravi commented 4 years ago

@perrynzhou Did you get a chance to test the patch?

gluster-ant commented 4 years ago

A patch https://review.gluster.org/24350 has been posted that references this issue.

posix: add io_uring support

This patch adds support for reads, writes and fsyncs using io-uring in posix xlator. A volume option 'storage.linux-io_uring' is introduced to turn it on or off.

When I ran tests with fio and iozone on physical machines on a 2x3 volume + single client, the numbers were not any worse than regular pwrite/pread syscalls which posix does with default volume configurations. But given that io_uring itself undergoing rapid improvement and bug fixes,it is good to have an intial working implementation in gluster and iteratively develop on top of it.

Updates: #1398 Change-Id: Ia47456ebb4c16a3b66ad9beb6a9043cc090fed2b Signed-off-by: Ravishankar N ravishankar@redhat.com

itisravi commented 4 years ago

Hi @amarts, For https://review.gluster.org/#/c/glusterfs/+/24350/ to pass smoke, I was wondering if we need SpecApproved/DocApproved or if I should convert this issue into Type:Bug.

amarts commented 4 years ago

This is a feature... But as its completely internal, and it's as good as mapping 1 - 1 with uring APIs, should be good to get the label given. Will provide label.

itisravi commented 4 years ago

Closing the issue since the patch referenced above is merged. Will create separate issues for improvements.