MrPig91 / PSChiaPlotter

A repo for powershell module that helps Chia Plotting
MIT License
181 stars 47 forks source link

Optimal Stagger per hardware #95

Closed GeoWeb-Pro closed 3 years ago

GeoWeb-Pro commented 3 years ago

Again this is a great project and my sincerely tanks for the time you put in this.

I literally know nothing about all this and started all in a rush but for newbies like me is there a way to present a auto stagger mode with the script checking the existing hardware, could this be added on enhancement list, if to much no trouble at all.

If not possible could you advise for a not computer tech guy ho would i stagger the best for my set up

AMD Ryzen 3960x - 48 Threads 96 gb ram 3 x 2tb NVME (1 Sabrent Q and 2 Samsung 970 EVO Plus) not in raid 15 x 8 or 10 tb portable hdds all connected and sleep mode disabled

What is the way as i cant get this time under 10h i was running 12 plots in parallel in different way (3 in pase 1 then another r 3 after 32% and another 3 after 78% and Another 3 at 85% or i run the 1 at interval of 60 minute whatever i tried nothing diminished that time.

Any advice would be much appreciated.

setup

sk00t3r commented 3 years ago

I'm sure you know but your phase 4 is going to be limited to the write speed or interface speed of your portable drives. With that said I have not done much testing on NVME raid vs single. However one of my servers is getting 20 plots per day with 8 600gb 15k SAS drives in a raid 0. I have not done much testing to max or perfect the timing and just run 8 plots in parallel with 120 minute delay, 6 cores each, and 6144MB RAM.

I do not have any NVME drives that I am willing to sacrifice (not even my enterprise grade ones) on this coin. I didn't do the exact math but I wouldn't expect more than ~160 days of usage out of your 3 NVME drives before they die. Someone can correct me if I am way off on that number.

You could try putting your 3 NVME drives in a storage pool and setting them up in a raid 0. See if your speed increases ( I would assume it would). I will link a great video on how to optimize your plots by looking at logs and using PowerShell.

Optimizing The Delay Between Parallel Plots Using Log Files and Powershell - Chia Plotting Part 3

GeoWeb-Pro commented 3 years ago

@sk00t3r thanks a million for reply, in terms of the NVMe ware I'm aware and is fine as long as i can get my 200tb full quickly, in terms of the copy phase it takes rafely 9-10 minutes they are connected all through usb 3.2 gen ports and the hdd's are usb 3

I had done one plot only and used 12 threads and 6750 memory and it took 6h 35min as as soon i start to add more the time increase drastically.

Now i added to que: 5 plots per Nvme with 160 minute stager, the second Nvme kicks in 20 minutes later with the same settings, and the 3rd Nvme kicks in 45 later leading to a total of 15 plots each assigned 6gb ram and 12 threads

Are the above optimal or is unproductive.

Stat Plot

This is how the result look like with the above stager

stat 2

sk00t3r commented 3 years ago

Someone please correct me if I am wrong here but I think only phase 1 uses multithreading and there are diminishing returns after using 4 threads. Your RAM is on the high side many people say that with bitfield enabled (default settings) the max is ~3400, and after reading up on your setup, you should be aiming for 7 plots on each 2tb nvme (1862GiB/239GiB=7.79). A good starting might be trying to stagger them 40 minutes apart and start all three nvme's at the same time. From there you can look at logs or trial and error and see if its any faster to stager each nvme's starting time or not.

MrPig91 commented 3 years ago

I think sk00t3r provided a great starting point a good optimization of settings. I think from there it will just come down tweaking it a bit.