search

MrPig91 / PSChiaPlotter

A repo for powershell module that helps Chia Plotting
MIT License
182 stars 47 forks source link

Tmp Space calculations #128

Open Jacek-ghub opened 3 years ago

Jacek-ghub commented 3 years ago

Hi,

I understand that based on Chia info, tmp space needed per one plot is 256.6 GB or so. However, that is the peak, and potentially on pre v.1.1.7. That means, it only affects those that run one plot per one temp devices (SAS guys).

I have just checked my PSCP, and have the following: 162 GB, 155, 161, 211, 197, 165, 152, 136, and 64. As I run that job on three tmp folders, their usage at that time was: 520, 455, and 395 (3 plots being worked on in parallel on one NVMe).

Therefore, I think that all the tmp space calculations, when there is more than one plot going on on that volume could be relaxed a bit, and PSCP should display 4 as the max number of plots on those 1TB volumes.

In my case, I have 10 cores on my CPU, so for now, I decided to run just 9 plots in parallel on those three folders. However, if someone has number of cores divisible by 4, I think that it is no problem at all to run 4 plots per 1TB of tmp space. It may be a bit slower, but that may be negligible. At the same time, running that extra one/two plots may cover that slowdown (e.g., recent post about 8 core CPU running just 6 plots in parallel). It also doesn't save anything as far as the cost of those NVMes, so from price point of view, it is a wash.

I understand that there is a checkbox to ignore MaxParallel calculations, but that checkmark is just pushing the burden of knowing those calculations on the end user and that is not what we really want.

Thank you, Jacek

MrPig91 commented 3 years ago

I completely agree with your assessment on the max temp parallel plots restrictions should be relaxed a bit. When first writing PSCP I tried to stay on the cautious side to allow for any user to run the plot manager without worrying about running into issues (even if it was under optimized). I think I was a little bit too cautious in this particular case, I did add that Ignore MaxParllel checkbox as a band-aide solution to allow people to further optimized with being restricted. I will see what I can do to ease these restrictions to allow for more plots in parallel.

Jacek-ghub commented 3 years ago

Two things that can be done:

Using a table instead of calculations, e.g., 250GB - no-go 500GB - 2x 1TB - 4x 2TB - 9x (maybe 10) Caveat is that if NVMes are used, 3.3 GB/s is max (regardless of the size), so potentially they will flatline at 5, maybe 6 parallel plots. Still, being able to run 10 instead may be a preferred choice for some people.

The second thing is to add to the lower/left corner MaxTempSpace usage per day. This way, the user will roughly know what is the borderline, and it will be easy to ask people what they see for their queues, as such fine tune that table.

Actually, maybe that function could return two values - one for max, and one for preferred? That would be good for those that have 2TB NVMes.