Can -d parameter support dir list?

[ljboy1979]

Can -d parameter support dir list?

[Dartellum]

No.

I created a script to select a destination based on file size. You can find it here: https://github.com/Dartellum/chia-madmax-plot_loop

If you just want the disk selection logic, that is the disk_space function. I still use my loop but now use mergerfs to handle my destination.

[trhr]

mergerfs

[wallentx]

I've found nothing better, and will be including logic to replot soon. https://gist.github.com/wallentx/a462521b0c8ea0ebd64d19f0c1ed82f9

[trhr]

why in the world would you do that in code instead of using native linux/posix methods

/etc/fstab

/mnt/sd* /mnt/chia fuse.mergerfs allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs,moveonenospc=true,threads=2 0 0

i think the default option is to fill emptiest drive first, but there are others.

[wallentx]

why in the world would you do that in code instead of using native linux/posix methods

/etc/fstab

/mnt/sd* /mnt/chia fuse.mergerfs allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs,moveonenospc=true,threads=2 0 0

i think the default option is to fill emptiest drive first, but there are others.

Hm, I've never looked at mergerfs before. A quick glance seems that there is a hit on throughput and CPU usage. Does that increase per mount added? I currently have 93 drives.

[trhr]

No idea. Our compute servers is where we'd really notice, but they only have 6x16TBs on array. Once we swap them over into the cold storage, it's 60HDDs being managed by a Xeon Silver that doesn't do much else, so who cares?

I guess my final answer is that we're in the middle of plotting ~100TB/day onto 3 petabytes and the convenience of it outweighs any theoretical hit to throughput. Mounts "just work" - no complaining about destinations that don't exist anymore, no trying to write to directories that are root's now because there's nothing mounted on them - since both the farmer and the plotter are always looking at the same directory, we can hotswap with zero configuration.

[trhr]

tbh we went with mergerfs because of one thing: i wanted a filesystem that acted, at the posix level, like ceph/glusterfs, but i did -not- want any of the bells and whistles of those systems, like redundancy or striping. i wanted flat files on an xfs filesystem just to keep things simple. there's a good chance these hard drives outlive active development on ceph.

so i asked myself, "why do ceph/gluster behave like single drives to posix apps?" because they're FUSE filesystems. then the problem becomes simple: what other fuse filesystems support union but don't require redundancy? and mergerfs is the one.

in the real world, i guess we're more used to taking performance hits on our workloads than in the hyper-optimization of cryptocurrency. if the devs came to me at work and said "your systems can't handle our throughput" i wouldn't say "let me hyperoptimize my systems for the workload you have right now," i'd say "i'll get the boss to buy a better server." :)