search

szatmary / PlotFS

a fuse filesystem for efficiently storage of Chia plot files
MIT License
50 stars 9 forks source link

Transfer Speeds #7

Closed johnathon-b closed 2 years ago

johnathon-b commented 2 years ago

I have large farms and migrating one to PlotFS. It's taking a long time to transfer the plots. I'm looking for efficiencies as I plan migrating some other environments.

Are there specific limitations? I've tried to start multiple jobs to transfer from multiple drives but it made everything slower.

Anyways to parallel transfers?

szatmary commented 2 years ago

Plots will always be written to the disk with the most free space. So if one disk has significantly more free space, all plots will be written to that disk. Once the disks are balanced it should parallelize across spindles.

johnathon-b commented 2 years ago

Does the parallelize across spindles affect the blast radius of a drive dying?

szatmary commented 2 years ago

Sorry, I explained that poorly. Each plot will write to 1 spindle. To spread across spindles, you need to write multiple plots at the same time.

johnathon-b commented 2 years ago

Can confirm, this drastically sped up the transfers into PlotFS.

I use this one-liner to begin transfers based on how many disks I added to PlotFS.

# run 5 jobs parallel
find /bigpool/plot-storage/ -name "*.plot" | shuf | parallel --jobs 5 -I% --max-args 1 ' plotfs --add_plot % --remove_source'

In iotop I can see decent speeds considering I'm transferring from a ZFS pool.