search

westerndigitalcorporation / zenfs

ZenFS is a storage backend for RocksDB that enables support for ZNS SSDs and SMR HDDs.
GNU General Public License v2.0
238 stars 87 forks source link

Can't switch ZDB scheduler to mq-deadline to create ZenFS #220

Closed ryderhsu0129 closed 2 years ago

ryderhsu0129 commented 2 years ago

I have established the relevant environment to create ZenFS, using the mkfs command got some error : $./zenfs mkfs --zbd=nvme0n1 --aux_path=/usr/src/zenfs_temp Failed to open zoned block device: nvme0n1, error: Invalid argument: Current ZBD scheduler is not mq-deadline, set it to mq-deadline.

but I can only set scheduler to mq-deadline on nvme0c0n1, nvme0n1 doesn't have the option : $cat /sys/block/nvme0n1/queue/scheduler none $cat /sys/block/nvme0c0n1/queue/scheduler [mq-deadline] none

then I specify --zbd argument to nvme0c0n1, and got another error : $./zenfs mkfs --zbd=nvme0c0n1 --aux_path=/usr/src/zenfs_temp Failed to open zoned block device: nvme0c0n1, error: Invalid argument: Failed to open zoned block device for read: No such file or directory

looks like the command doesn't recognize the ZBD device, how should i solve it?thanks!

yhr commented 2 years ago

@ryderhsu0129 : Thanks for reporting, we're digging into the cause of this

yhr commented 2 years ago

What kernel version are you using?

ryderhsu0129 commented 2 years ago

@yhr my kernel version is 5.10.32.

If I disable multipath by adding nvme-core.multipath=NO to grub file, then I have the scheduler mq-deadline option in the sysfs node : $cat /sys/block/nvme0n1/queue/scheduler [mq-deadline] none I can create ZenFS success after doing the above action.

Is it not possible to create a ZenFS with the multipath function turned on?

yhr commented 2 years ago

@ryderhsu0129 : It looks like it might be a restriction/issue in the kernel, continuing to look into it. You could disable the check in the code as a temporary workaround for testing purposes (https://github.com/westerndigitalcorporation/zenfs/blob/d226fb48824dd7a6a479495c8cb11801c06fe7bc/fs/zbd_zenfs.cc#L202)

ZenFS works 99.9% out of the time without the deadline scheduler. Its only in rare cases where the kernel decides to split and reorder writes errors can occur.

ryderhsu0129 commented 2 years ago

@yhr ok i got it, thanks for your reply.