madMAx43v3r / chia-plotter

Apache License 2.0
2.27k stars 664 forks source link

With the fastest CPU in the world, one plot takes 2 hours to finish #842

Open psun888 opened 3 years ago

psun888 commented 3 years ago

I'm using AMD EPYC 7H12 64-Core 2.60Ghz, Here are some results with different thread settings for a single plot to finish:

System is Windows Server 2019, Ram is 1TB, first hard drives is latest Intel NVME enterprise 3.84TB, second drive is set with 120GB RAM disk. (four exact same systems)

-r 4, takes 3 hours, CPU usage is about 2% -r 6, takes 2 hours, CPU usage is about 3% -r 8, takes 1.5 hours, CPU usage is about 3% -r 10, takes 1.2 hours, CPU usage is about 4% -r 12, takes 1.5 hours, CPU usage is about 5% -r 32, takes 6-7 hours, CPU usage is about 30%, system becomes very slow on everything, include access any RAM or NVME drives even without any read or write. -r 64, takes more than 12 hours, CPU usage is about 40%, system becomes very slow on everything, include access any RAM or NVME drives even without any read or write.

with -r 10, I run another instance of the plotter, the system becomes immediately very slow, even the CPU usage is less than 6%.

It seems the plotter is not compatible with such high end CPU, or the plotter has some serious flaws, so sadly surprising. With official Chia plotter, I can mange to plot around 64 plots per day with 6 NVME drives with this CPU. Anybody have this kind of experience? any help would be greatly appreciated.

ataa commented 3 years ago

Someone ran MadMax on a 192 cores/384 threads with success and posted the results on Youtube (~1 minute per plot). So I don't think the issue here is related to the plotter. There are some reports on slow plotting on Windows Server (#378, ... ). My 6 core, 10 years old cpu with pcie 2.0 nvme and 16GB ram can make a plot in 55 minutes using Mad Max on Ubuntu 21.04 (7 Hours using official plotter). So try Ubuntu if you can. You can install it on USB flash drive without touching your Windows OS disk.

SebMoore commented 3 years ago

I've run the plotter on a 32C/64T Xeon system. The issue I had was if I ran it with -r 32 or -r 64, the CPU utilisation was only about 50-60%. This was because it was a dual socket system: to improve times, I ran one plotter on one CPU and one on the other CPU. I achieved this by using numactl, but that's a linux thing. It's possible you have the same problem because some Epyc chips use CCXs, which bundle 4 cores together in a compute unit. I don't have any experience with that particular CPU, but I can tell you that the plotter does scale with more cores - it should not take that long.

jackgump commented 3 years ago

Someone ran MadMax on a 192 cores/384 threads with success and posted the results on Youtube (~1 minute per plot). So I don't think the issue here is related to the plotter. There are some reports on slow plotting on Windows Server (#378, ... ). hi, my computer hardware is quite close to yours, i use Ubutun 20.04, however i can only get 2hours for one plot. can you please show your setup for plotting? including the ubutun disk and nvme formating setup. thx!

ataa commented 3 years ago

hi, my computer hardware is quite close to yours, i use Ubutun 20.04, however i can only get 2hours for one plot. can you please show your setup for plotting? including the ubutun disk and nvme formating setup. thx!

You can find it here: https://github.com/madMAx43v3r/chia-plotter/discussions/452#discussioncomment-912683 OS disk FS is EXT4, Temp1/2 and D XFS with CRC off.

jackgump commented 3 years ago

hi, my computer hardware is quite close to yours, i use Ubutun 20.04, however i can only get 2hours for one plot. can you please show your setup for plotting? including the ubutun disk and nvme formating setup. thx!

You can find it here: #452 (comment) OS disk FS is EXT4, Temp1/2 and D XFS with CRC off.

thx, how to turn off CRC In Ubuntu? I'm new to Linux.

mcsham commented 3 years ago

mkfs.xfs -f -m crc=0 /dev/nvme0n1p1

ataa commented 3 years ago

thx, how to turn off CRC In Ubuntu? I'm new to Linux.

@jackgump As far as I know, you can not just turn it off, you have to reformat the disk/nvme, the command provided by mcsham will do that for you. Be aware that it will format your drive without any confirmation. back up important data, then use that command.

gryan315 commented 3 years ago

In my testing on dual xeon system, crc had zero impact on performance, so no reason to disable it.

mo-shangren commented 3 years ago

有人在 192 核/384 线程上成功运行了 MadMax,并将结果发布在 Youtube 上(每个图约 1 分钟)。所以我认为这里的问题与绘图仪无关。有一些关于 Windows Server 上缓慢绘图的报告(#378,...)。 我的 6 核、10 年老 cpu 和 pcie 2.0 nvme 和 16GB ram 可以在 55 分钟内使用 Ubuntu 21.04 上的 Mad Max(使用官方绘图仪 7 小时)绘制一个绘图。因此,如果可以,请尝试使用 Ubuntu。您可以将其安装在 USB 闪存驱动器上,而无需接触 Windows 操作系统磁盘。

please give me the link,thanks

SebMoore commented 3 years ago

有人在 192 核/384 线程上成功运行了 MadMax,并将结果发布在 Youtube 上(每个图约 1 分钟)。所以我认为这里的问题与绘图仪无关。有一些关于 Windows Server 上缓慢绘图的报告(#378,...)。 我的 6 核、10 年老 cpu 和 pcie 2.0 nvme 和 16GB ram 可以在 55 分钟内使用 Ubuntu 21.04 上的 Mad Max(使用官方绘图仪 7 小时)绘制一个绘图。因此,如果可以,请尝试使用 Ubuntu。您可以将其安装在 USB 闪存驱动器上,而无需接触 Windows 操作系统磁盘。

please give me the link,thanks

https://www.youtube.com/watch?v=dQRBmKCDV7E