search

koverstreet / bcachefs

Other
686 stars 71 forks source link

QUESTION: create array in degraded state? #599

Open TurkeyMan opened 1 year ago

TurkeyMan commented 1 year ago

Hi, with apparently imminent inclusion in linux, I'm considering migrating a ZFS array to bcachefs. As if often the case, the motivator is the need to expand my array.

There are a various common tricks to migrate arrays, ie RAID6(/Z2) without doubling the number of drives by degrading the source array and using some of those parity drives in the new array during migration. Using a technique like this, the destination array also needs to be created in a degraded state, and then the parity drives rebuilt after migration is complete.

For example, in my hypothetical migration, I have a 2+2 RAIDZ2, and I want to expand to a 4+2 bcachefs. I could dismount the 2 parity drives from the source array, and then with those and the 2 new drives, allocate the 4 data drives in a degraded 4+2 erasure coding set, copy the data, and then move the remaining decommissioned drives over to the new array and rebuild the parity.

A less edgy way to do this migration might be to remove one parity drive from the 2+2 RAIDZ2, leaving one parity drive for security during migration, and create a bcachefs using the 3 available drives with 2+1 erasure coding set for destination data security while migrating, and then afterwards, add the 3 remaining ZFS drives to the pool and expand from 2+1 to 4+2? It would need to rewrite all the data with the new stripe width though in that case, whereas the prior approach would just need to rebuild parity.

How to best achieve strategies like this with bcachefs in the version that is likely to be made available? Are there better options?

YellowOnion commented 10 months ago

I'm not exactly sure what "2+2" means here, EC is currently broken and I wouldn't recommend using it, you can add drives and format with replicas = 1 and metadata_replicas = 1, and then when you add more drives increase replica counts, bcachefs just tries to make sure x copies exist somewhere, once EC is stable you can then set that and bcachefs will convert everything in the background with enough time. I happily compressed 4TB with lz4 after format over a week or so.