search

madMAx43v3r / chia-plotter

Apache License 2.0
2.27k stars 663 forks source link

Slow parallel plotting #228

Open Mr-Hoffmann opened 3 years ago

Mr-Hoffmann commented 3 years ago

Hello everyone. My system is x5950 4.7ghz; x570 asus tuf; 128gb 3200mhz; 4tb nvme 4.0 sabrent; I ran 30 threads on got 30 minutes for a single plot, insane speeds. However, when i try to have more then 1 paralel plots it starts to get jammed. The issue is, although i have 32 threads to use, if i try to divide them in several parallel plots i get really bad results. Like trying to run 8 plots with 4 threads each (8x4=32). Does this mean that this plotter is designed to run only 1 plot at a time with max usage of CPU? Ignoring NVME capacity?

EverCurse commented 3 years ago

how about the speed of 8 plots with 4 threads ?

SebMoore commented 3 years ago

Yes. That's correct. If you try to run two plots at once, the time taken per plot will pretty much double, therefore you'll average the same amount of plots per hour. This plotter doesn't parallelise well because it already uses all of your CPU or storage device bandwidth.

Mr-Hoffmann commented 3 years ago

how about the speed of 8 plots with 4 threads ?

[P1] Table 1 took 156.836 sec [P1] Table 2 took 942.596 sec, found 4294943876 matches

And i stopped because more then half an hour has passed and it didnt add anything new.

Mr-Hoffmann commented 3 years ago

Yes. That's correct. If you try to run two plots at once, the time taken per plot will pretty much double, therefore you'll average the same amount of plots per hour. This plotter doesn't parallelise well because it already uses all of your CPU or storage device bandwidth.

I think that is even worse then that, it gets even slower for some reason When you say it uses all of CPU are you referring to CPU speed or the core itself? Shouldnt the THREADS option prevent this somehow?

SebMoore commented 3 years ago

Yes. That's correct. If you try to run two plots at once, the time taken per plot will pretty much double, therefore you'll average the same amount of plots per hour. This plotter doesn't parallelise well because it already uses all of your CPU or storage device bandwidth.

I think that is even worse then that, it gets even slower for some reason When you say it uses all of CPU are you referring to CPU speed or the core itself? Shouldnt the THREADS option prevent this somehow?

Well, yes: you first ran it with your whole CPU (32 threads) and now when you run it with 8 plots, each plot is probably about 8x slower. Probably even worse though possibly because of a drive bottleneck possibly? Also keep in mind that the threads option is not necessarily a hard limit - see the readme - it's more of a multiplier or guide. If you are in Linux, you can use the taskset command to enforce a CPU limit on a process.