Plot transfer to NFS is really slow

madMAx43v3r / chia-plotter

Apache License 2.0

2.27k stars 664 forks source link

Plot transfer to NFS is really slow #631

Open ebeng opened 3 years ago

ebeng commented 3 years ago

Hi Everyone,

from a network perspective and interfaces perspective on all my servers, nothings has been changed. I have moved only from Official-Chia plotter to MadMax, what I see is that copying/moving the plots are really slow.

as you can see, the biggest spike yesterday is when it was still on the official plotter. (1gpbs) the others are now around the 50-200Mbps, which makes it really slow.

Any idea what could be the issue here?

madMAx43v3r commented 3 years ago

which commit are you at? I increased the block size for copy recently, in hopes to increase NTFS performance, see 67da6828801dcc56c57e36af78bf8da867aaed0a

carlfarrington commented 3 years ago

On windows, I was using NFS, and I found that if I re-niced chia_plot to 'below normal', my tx speed increased form 300mbps to 700mbps.

ebeng commented 3 years ago

which commit are you at? I increased the block size for copy recently, in hopes to increase NTFS performance, see 67da682

commit a2e06b01ae71b2a5811fa853d57abfd14c78a003 (HEAD -> master, origin/master, origin/HEAD) Author: Max max.wittal@mwittal.de Date: Mon Jun 21 15:43:34 2021 +0200

Edit: I see that your commit is in the build files... hmmm... let me recheck some of the servers, even reboot them. Probably as all the servers were transformed into your plotter, the NFS servers can't maybe handle all the load (?)... I will update accordingly

madMAx43v3r commented 3 years ago

ah you mean NFS, I've been using that too without any issues...

ebeng commented 3 years ago

ah you mean NFS, I've been using that too without any issues...

Haha yes! 2 NFS servers are pointed from 10 virtuals on different hardware. Somehow/something was impacted but Im not sure what exactly. Half of the servers are now on the official and half with yours. I will check later on today what the outcome is and try to tweak a bit more. Hopefully it was just an error somewhere...