koverstreet / bcachefs

Other
686 stars 71 forks source link

Question about write io path for multiple devices #584

Open Dikay900 opened 1 year ago

Dikay900 commented 1 year ago

First of all big thanks for your work on this fs.

In the principles of operation you state:

bcachefs is a multi-device filesystem. Devices need not be the same size: by
default, the allocator will stripe across all available devices but biasing in favor
of the devices with more free space, so that all devices in the filesystem fill up at
the same rate. Devices need not have the same performance characteristics: we
track device IO latency and direct reads to the device that is currently fastest

by default sounds like you can change this behavior, so is this already possible to setup or rather something for the future?

I would like to have the maximum of write speed for multiple devices which have different size and performance characteristics. So the fastest devices will fill fastest and then the data gets balanced, which would make manually setting up a promote_target unnecessary or rather optional.

My user scenario is having mixes of ssds and nvmes which are all having slightly different performance characteristics which will slow down the system to the slowest device for the promote_target. This is obviously the same as the allocator does this in the same way for all devices when using no promote_target.

mrnerdhair commented 1 year ago

You can do this by using foreground_target to direct writes to your fast devices, background_target to balance things to your slow devices, and then use promote_target to cache data from your slow devices in your fast ones' free space.