[Feature] Limit Transfer Rate to the Final Drives

[MarFre22]

Hi, thank you very much for your good work!

I have been using your program, when plots are moved to the final drive I have longer responses to challenges, sometimes even errors (more than 30seconds). I think it's a "must have" feature to avoid these bad situations. If you decide to implement this, for example, you can make a text box where we can put the maximum MB/s for each end unit in the "New Job" window. I think with Robocopy we can apply that.

Thank you very much for your hard work :)

[Jacek-ghub]

I think that you may be experiencing harvester code degradation that started with v1.2.0 (and still continues). Check their Github page, as there are several recent posts there about that. I would also suggest that you run PSCP heat maps against your plotter/harvester box(es), as this way you can also see the history, and you may catch some problems faster. I do run them, and my full node/harvester that doesn't do any plotting started to have plenty of issues starting with v1.2.0.

Going back to your request. either 5 sec to pass the filter or 30 sec to provide full response are rather huge timeouts, where your box operates at GHz rates. Also, PSCP only writes to one disk, and that is not affecting harvester checking other drives. That also means, that initially there is zilch on that destination drive, as such there should be no issues about 50% of time. Saying that, I also often wish that the moving process should run at a lower priority, but that type of thinking is coming out of frustration of seeing and not understanding those problems.

[MarFre22]

@Jacek-ghub , thank you very much for your explanation, I didn't know that. Before I was having longer response time (like 4/5s), without these errors, when a proof was found on the same disk where the plot was being moved. Due to the code degradation you said this situation is getting worse. I think we can try to minimize this situation by limiting the transfer rate to the final drive.

[Jacek-ghub]

@MarFre22

By the way, how do you know that moving those plots is causation not just correlation? Did you try to run those heat maps, and pause your plotter for some time, and check whether you are getting less or no timeouts anymore? Actually, as it was discussed in another thread, you can specify your final destination as your temp1 drive (if you are using MM, potentially temp2 if that is the official Chia plotter), this way it will take 0 seconds to move those plots. If you have some extra room on your temp1 drive, you don't need to stop your plotter, just point it to that folder. Once you check those heat maps, you will have to manually move those new plots. However, this way you can test: 1. whether just CPU usage from the plotter may influence those timeouts, 2. the final move process is contributing to that. Again, I see those timeouts mostly on my full node / main harvester that doesn't do any plotting, but sits on most of my plots. On the other hand, my plotter doesn't have much, if any timeouts (it sits on two/three disks at most).

Again, the main Chia code responsible for those timeouts has badly deteriorated since 1.2.0, and Chia team so far is refusing to accept that (you cannot fix, what you don't understand). So, putting some band-aid code in PSCP may really not be the best thing to do. But again, as PSCP will soon have a mover component, I would think that just for the sake of it, it could be run with BelowNormal priority, what would not change anything, but let few people sleep better - mainly psychological value, that's it.