madMAx43v3r / chia-plotter

Apache License 2.0
2.27k stars 662 forks source link

Dual Xeon on Windows version doesn't work (kills process) #126

Closed Hedonay closed 3 years ago

Hedonay commented 3 years ago

So I'm using dual Xeon (8 cores, 16 threads each) server with 64G of ram and running Windows 2019 server. I start it with the following command (alpha version 0.2):

.\chia_plot.exe -p "key" -f "key" -n 1 -r 24 -u 7 -t E:\directory\ -d D:\directory\

What I get as a result is:

Final Directory: D:\directory\
Number of Plots: 1
Process ID: 7128
Number of Threads: 24
Number of Buckets: 2^7 (128)

And after that process is killed and does not continue.

I tried with lower number of threads (went even down to 4).

Was looking for help and got same configuration from a friend with same end result. Process killed and he's also not able to find a way to make it work.

Any ideas?

phucha1986 commented 3 years ago

try with no avx https://github.com/stotiks/chia-plotter/releases/download/v0.0.4/chia_plot_noavx.zip

Version=1 EventType=BEX64 EventTime=132679716519702967 ReportType=2 Consent=1 UploadTime=132679716522068463 ReportStatus=268435456 ReportIdentifier=cb4b73cd-c907-4c7b-b310-1557a749bada IntegratorReportIdentifier=72ad3d0c-09f2-45d1-b925-8ea74084c573 Wow64Host=34404 NsAppName=chia_plot.exe AppSessionGuid=00002d30-0001-0006-1054-94707f5fd701 TargetAppId=W:000688868ca76bff6b7d930701438b3622d10000ffff!0000b3c17933a6bc1426c764d17474ad37740ab3b366!chia_plot.exe TargetAppVer=2021//06//12:11:17:56!0!chia_plot.exe BootId=4294967295 TargetAsId=1497 IsFatal=1 EtwNonCollectReason=1 Response.BucketId=c894e2d44db350ba5988248d97ed226f Response.BucketTable=5 Response.LegacyBucketId=1839760638338736751 Response.type=4 Sig[0].Name=Application Name Sig[0].Value=chia_plot.exe Sig[1].Name=Application Version Sig[1].Value=0.0.0.0 Sig[2].Name=Application Timestamp Sig[2].Value=60c497e4 Sig[3].Name=Fault Module Name Sig[3].Value=chia_plot.exe Sig[4].Name=Fault Module Version Sig[4].Value=0.0.0.0 Sig[5].Name=Fault Module Timestamp Sig[5].Value=60c497e4 Sig[6].Name=Exception Offset Sig[6].Value=00000000000f1651 Sig[7].Name=Exception Code Sig[7].Value=c0000409 Sig[8].Name=Exception Data Sig[8].Value=0000000000000007 DynamicSig[1].Name=OS Version DynamicSig[1].Value=10.0.19043.2.0.0.768.101 DynamicSig[2].Name=Locale ID DynamicSig[2].Value=1033 DynamicSig[22].Name=Additional Information 1 DynamicSig[22].Value=76a3 DynamicSig[23].Name=Additional Information 2 DynamicSig[23].Value=76a373cd86f1762b5955d337a726c094 DynamicSig[24].Name=Additional Information 3 DynamicSig[24].Value=9aa1 DynamicSig[25].Name=Additional Information 4 DynamicSig[25].Value=9aa108504b96a170a98f45e075793f09 UI[2]=C:\Users\hahan\Downloads\madMAx43v3r_chia-plotter_win_build_v0.0.4\chia_plot.exe LoadedModule[0]=C:\Users\hahan\Downloads\madMAx43v3r_chia-plotter_win_build_v0.0.4\chia_plot.exe LoadedModule[1]=C:\Windows\SYSTEM32\ntdll.dll LoadedModule[2]=C:\Windows\System32\KERNEL32.DLL LoadedModule[3]=C:\Windows\System32\KERNELBASE.dll LoadedModule[4]=C:\Windows\System32\ADVAPI32.dll LoadedModule[5]=C:\Windows\System32\msvcrt.dll LoadedModule[6]=C:\Windows\System32\sechost.dll LoadedModule[7]=C:\Windows\System32\RPCRT4.dll LoadedModule[8]=C:\Windows\SYSTEM32\CRYPTBASE.DLL LoadedModule[9]=C:\Windows\SYSTEM32\CRYPTSP.dll LoadedModule[10]=C:\Windows\system32\rsaenh.dll LoadedModule[11]=C:\Windows\System32\bcrypt.dll LoadedModule[12]=C:\Windows\System32\bcryptPrimitives.dll LoadedModule[13]=C:\Windows\SYSTEM32\kernel.appcore.dll State[0].Key=Transport.DoneStage1 State[0].Value=1 OsInfo[0].Key=vermaj OsInfo[0].Value=10 OsInfo[1].Key=vermin OsInfo[1].Value=0 OsInfo[2].Key=verbld OsInfo[2].Value=19043 OsInfo[3].Key=ubr OsInfo[3].Value=1052 OsInfo[4].Key=versp OsInfo[4].Value=0 OsInfo[5].Key=arch OsInfo[5].Value=9 OsInfo[6].Key=lcid OsInfo[6].Value=2057 OsInfo[7].Key=geoid OsInfo[7].Value=251 OsInfo[8].Key=sku OsInfo[8].Value=101 OsInfo[9].Key=domain OsInfo[9].Value=0 OsInfo[10].Key=prodsuite OsInfo[10].Value=768 OsInfo[11].Key=ntprodtype OsInfo[11].Value=1 OsInfo[12].Key=platid OsInfo[12].Value=10 OsInfo[13].Key=sr OsInfo[13].Value=0 OsInfo[14].Key=tmsi OsInfo[14].Value=221130155 OsInfo[15].Key=osinsty OsInfo[15].Value=2 OsInfo[16].Key=iever OsInfo[16].Value=11.789.19041.0-11.0.1000 OsInfo[17].Key=portos OsInfo[17].Value=0 OsInfo[18].Key=ram OsInfo[18].Value=65428 OsInfo[19].Key=svolsz OsInfo[19].Value=111 OsInfo[20].Key=wimbt OsInfo[20].Value=0 OsInfo[21].Key=blddt OsInfo[21].Value=191206 OsInfo[22].Key=bldtm OsInfo[22].Value=1406 OsInfo[23].Key=bldbrch OsInfo[23].Value=vb_release OsInfo[24].Key=bldchk OsInfo[24].Value=0 OsInfo[25].Key=wpvermaj OsInfo[25].Value=0 OsInfo[26].Key=wpvermin OsInfo[26].Value=0 OsInfo[27].Key=wpbuildmaj OsInfo[27].Value=0 OsInfo[28].Key=wpbuildmin OsInfo[28].Value=0 OsInfo[29].Key=osver OsInfo[29].Value=10.0.19041.1052.amd64fre.vb_release.191206-1406 OsInfo[30].Key=buildflightid OsInfo[31].Key=edition OsInfo[31].Value=Core OsInfo[32].Key=ring OsInfo[32].Value=Retail OsInfo[33].Key=expid OsInfo[34].Key=fconid OsInfo[35].Key=containerid OsInfo[36].Key=containertype OsInfo[37].Key=edu OsInfo[37].Value=0 FriendlyEventName=Stopped working ConsentKey=BEX64 AppName=chia_plot.exe AppPath=C:\Users\hahan\Downloads\madMAx43v3r_chia-plotter_win_build_v0.0.4\chia_plot.exe NsPartner=windows NsGroup=windows8 ApplicationIdentity 007C3C96EF8D447F803743E472CF2C3 MetadataHash=125575459

celalanak commented 3 years ago

@stotiks why slow my system ? Dell R720 2x E5-2670 32 Core 64GB RAM 3x300 raid0 Samsung Evo 1Tb Table 1took 80 sec, same system win10 Table took 30 sec. can you help me for this. Thank you.

stotiks commented 3 years ago

I think crash on phase1 is hardware problem, please made memtest and cpu stresstest

Some users reported downclocking cpu or memory is helped

tydeno commented 3 years ago

FYI: Was able to finish a plot with catchmeifyoucan's version on 4x Xeon, Windows Server 2016:

Final Directory: F:\ Number of Plots: 1 Process ID: 7600 Number of Threads: 64 Number of Buckets: 2^8 (256) Pool Public Key: MyKey Farmer Public Key: MyKey Working Directory: B:\ Working Directory 2: B:\ Plot Name: plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7 [P1] Table 1 took 216.814 sec [P1] Table 2 took 1016.06 sec, found 4294900520 matches [P1] Table 3 took 1395.56 sec, found 4294913933 matches [P1] Table 4 took 1694.66 sec, found 4294960724 matches [P1] Table 5 took 1683.73 sec, found 4294892549 matches [P1] Table 6 took 1618.51 sec, found 4294912001 matches [P1] Table 7 took 1036.07 sec, found 4294857450 matches Phase 1 took 8662.24 sec [P2] max_table_size = 4294967296 [P2] Table 7 scan took 41.5382 sec [P2] Table 7 rewrite took 132.856 sec, dropped 0 entries (0 %) [P2] Table 6 scan took 52.1252 sec [P2] Table 6 rewrite took 181.177 sec, dropped 581297235 entries (13.5346 %) [P2] Table 5 scan took 65.4004 sec [P2] Table 5 rewrite took 172.544 sec, dropped 761939563 entries (17.7406 %) [P2] Table 4 scan took 122.309 sec [P2] Table 4 rewrite took 175.116 sec, dropped 828818947 entries (19.2975 %) [P2] Table 3 scan took 124.939 sec [P2] Table 3 rewrite took 173.492 sec, dropped 855039177 entries (19.9082 %) [P2] Table 2 scan took 124.127 sec [P2] Table 2 rewrite took 171.112 sec, dropped 865567521 entries (20.1534 %) Phase 2 took 1594.5 sec Wrote plot header with 268 bytes [P3-1] Table 2 took 173.618 sec, wrote 3429332999 right entries [P3-2] Table 2 took 157.627 sec, wrote 3429332999 left entries, 3429332999 final [P3-1] Table 3 took 180.526 sec, wrote 3439874756 right entries [P3-2] Table 3 took 158.528 sec, wrote 3439874756 left entries, 3439874756 final [P3-1] Table 4 took 181.764 sec, wrote 3466141777 right entries [P3-2] Table 4 took 157.603 sec, wrote 3466141777 left entries, 3466141777 final [P3-1] Table 5 took 184.408 sec, wrote 3532952986 right entries [P3-2] Table 5 took 162.157 sec, wrote 3532952986 left entries, 3532952986 final [P3-1] Table 6 took 203.517 sec, wrote 3713614766 right entries [P3-2] Table 6 took 165.707 sec, wrote 3713614766 left entries, 3713614766 final [P3-1] Table 7 took 221.168 sec, wrote 4294857450 right entries [P3-2] Table 7 took 200.895 sec, wrote 4294857450 left entries, 4294857450 final Phase 3 took 2169.08 sec, wrote 21876774734 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 362.115 sec, final plot size is 108833505979 bytes Total plot creation time was 12788.3 sec Started copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot Copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot finished, took 207.259 sec, 0 MB/s avg.

Trying noavx now.

NT3327 commented 3 years ago

FYI: Was able to finish a plot with catchmeifyoucan's version on 4x Xeon, Windows Server 2016:

Final Directory: F: Number of Plots: 1 Process ID: 7600 Number of Threads: 64 Number of Buckets: 2^8 (256) Pool Public Key: MyKey Farmer Public Key: MyKey Working Directory: B: Working Directory 2: B: Plot Name: plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7 [P1] Table 1 took 216.814 sec [P1] Table 2 took 1016.06 sec, found 4294900520 matches [P1] Table 3 took 1395.56 sec, found 4294913933 matches [P1] Table 4 took 1694.66 sec, found 4294960724 matches [P1] Table 5 took 1683.73 sec, found 4294892549 matches [P1] Table 6 took 1618.51 sec, found 4294912001 matches [P1] Table 7 took 1036.07 sec, found 4294857450 matches Phase 1 took 8662.24 sec [P2] max_table_size = 4294967296 [P2] Table 7 scan took 41.5382 sec [P2] Table 7 rewrite took 132.856 sec, dropped 0 entries (0 %) [P2] Table 6 scan took 52.1252 sec [P2] Table 6 rewrite took 181.177 sec, dropped 581297235 entries (13.5346 %) [P2] Table 5 scan took 65.4004 sec [P2] Table 5 rewrite took 172.544 sec, dropped 761939563 entries (17.7406 %) [P2] Table 4 scan took 122.309 sec [P2] Table 4 rewrite took 175.116 sec, dropped 828818947 entries (19.2975 %) [P2] Table 3 scan took 124.939 sec [P2] Table 3 rewrite took 173.492 sec, dropped 855039177 entries (19.9082 %) [P2] Table 2 scan took 124.127 sec [P2] Table 2 rewrite took 171.112 sec, dropped 865567521 entries (20.1534 %) Phase 2 took 1594.5 sec Wrote plot header with 268 bytes [P3-1] Table 2 took 173.618 sec, wrote 3429332999 right entries [P3-2] Table 2 took 157.627 sec, wrote 3429332999 left entries, 3429332999 final [P3-1] Table 3 took 180.526 sec, wrote 3439874756 right entries [P3-2] Table 3 took 158.528 sec, wrote 3439874756 left entries, 3439874756 final [P3-1] Table 4 took 181.764 sec, wrote 3466141777 right entries [P3-2] Table 4 took 157.603 sec, wrote 3466141777 left entries, 3466141777 final [P3-1] Table 5 took 184.408 sec, wrote 3532952986 right entries [P3-2] Table 5 took 162.157 sec, wrote 3532952986 left entries, 3532952986 final [P3-1] Table 6 took 203.517 sec, wrote 3713614766 right entries [P3-2] Table 6 took 165.707 sec, wrote 3713614766 left entries, 3713614766 final [P3-1] Table 7 took 221.168 sec, wrote 4294857450 right entries [P3-2] Table 7 took 200.895 sec, wrote 4294857450 left entries, 4294857450 final Phase 3 took 2169.08 sec, wrote 21876774734 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 362.115 sec, final plot size is 108833505979 bytes Total plot creation time was 12788.3 sec Started copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot Copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot finished, took 207.259 sec, 0 MB/s avg.

Trying noavx now.

Curious how you make out with the noavx version also. I've got a Dell R910 w/ 4X E7-4870's and with the standard madmax v.0.0.4 if I deviate from the default amount of threads it seems to have a negative impact on the times and overall performance of the machine. (even when not at 100% CPU, lookup times for some external drives I have go through the roof - I know I should just plot on this rig but, no room for the externals anywhere else just yet for farming)

With default threads going to a PCIe Fusion-IO card my best time so far was 3.4 hours which is amazing considering prior plotting time was 3-4x that to SAS drives and even my first couple runs to the PCIe card with the Official chia plotter were long, man the phase one could take more then 5+ hours!

BTW for this run, did you go to a ramdisk?

Got a plot going with the noavx version now with just slightly higher threads and it seems to not be impacting the machine as poorly, so we'll see.

Thanks!

tydeno commented 3 years ago

FYI: Was able to finish a plot with catchmeifyoucan's version on 4x Xeon, Windows Server 2016:

Final Directory: F: Number of Plots: 1 Process ID: 7600 Number of Threads: 64 Number of Buckets: 2^8 (256) Pool Public Key: MyKey Farmer Public Key: MyKey Working Directory: B: Working Directory 2: B: Plot Name: plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7 [P1] Table 1 took 216.814 sec [P1] Table 2 took 1016.06 sec, found 4294900520 matches [P1] Table 3 took 1395.56 sec, found 4294913933 matches [P1] Table 4 took 1694.66 sec, found 4294960724 matches [P1] Table 5 took 1683.73 sec, found 4294892549 matches [P1] Table 6 took 1618.51 sec, found 4294912001 matches [P1] Table 7 took 1036.07 sec, found 4294857450 matches Phase 1 took 8662.24 sec [P2] max_table_size = 4294967296 [P2] Table 7 scan took 41.5382 sec [P2] Table 7 rewrite took 132.856 sec, dropped 0 entries (0 %) [P2] Table 6 scan took 52.1252 sec [P2] Table 6 rewrite took 181.177 sec, dropped 581297235 entries (13.5346 %) [P2] Table 5 scan took 65.4004 sec [P2] Table 5 rewrite took 172.544 sec, dropped 761939563 entries (17.7406 %) [P2] Table 4 scan took 122.309 sec [P2] Table 4 rewrite took 175.116 sec, dropped 828818947 entries (19.2975 %) [P2] Table 3 scan took 124.939 sec [P2] Table 3 rewrite took 173.492 sec, dropped 855039177 entries (19.9082 %) [P2] Table 2 scan took 124.127 sec [P2] Table 2 rewrite took 171.112 sec, dropped 865567521 entries (20.1534 %) Phase 2 took 1594.5 sec Wrote plot header with 268 bytes [P3-1] Table 2 took 173.618 sec, wrote 3429332999 right entries [P3-2] Table 2 took 157.627 sec, wrote 3429332999 left entries, 3429332999 final [P3-1] Table 3 took 180.526 sec, wrote 3439874756 right entries [P3-2] Table 3 took 158.528 sec, wrote 3439874756 left entries, 3439874756 final [P3-1] Table 4 took 181.764 sec, wrote 3466141777 right entries [P3-2] Table 4 took 157.603 sec, wrote 3466141777 left entries, 3466141777 final [P3-1] Table 5 took 184.408 sec, wrote 3532952986 right entries [P3-2] Table 5 took 162.157 sec, wrote 3532952986 left entries, 3532952986 final [P3-1] Table 6 took 203.517 sec, wrote 3713614766 right entries [P3-2] Table 6 took 165.707 sec, wrote 3713614766 left entries, 3713614766 final [P3-1] Table 7 took 221.168 sec, wrote 4294857450 right entries [P3-2] Table 7 took 200.895 sec, wrote 4294857450 left entries, 4294857450 final Phase 3 took 2169.08 sec, wrote 21876774734 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 362.115 sec, final plot size is 108833505979 bytes Total plot creation time was 12788.3 sec Started copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot Copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot finished, took 207.259 sec, 0 MB/s avg.

Trying noavx now.

Curious how you make out with the noavx version also. I've got a Dell R910 w/ 4X E7-4870's and with the standard madmax v.0.0.4 if I deviate from the default amount of threads it seems to have a negative impact on the times and overall performance of the machine. (even when not at 100% CPU, lookup times for some external drives I have go through the roof - I know I should just plot on this rig but, no room for the externals anywhere else just yet for farming)

With default threads going to a PCIe Fusion-IO card my best time so far was 3.4 hours which is amazing considering prior plotting time was 3-4x that to SAS drives and even my first couple runs to the PCIe card with the Official chia plotter were long, man the phase one could take more then 5+ hours!

BTW for this run, did you go to a ramdisk?

Got a plot going with the noavx version now with just slightly higher threads and it seems to not be impacting the machine as poorly, so we'll see.

Thanks!

It was using an NVMe as TempDir. When I use a RamDisk it sometimes crashes in P1. Will do a memtest eventually. Stotiks NoAVX version also got me crashing with any drives (NVME/SSD/HDD/RamDisk).

The interesting thing is, that I only see high CPU utilization for the first few minutes (goes up 60-80%) and then plunges down to 30% and eventually 0%. There still read activity on the disks going on. When it plunges to 30% I see that it only uses 2x Physical CPUs instead of 4x available.

On my i9-9900k system, I see almost 100% CPU utliization all the time (using stotitks v.0.0.4).

celalanak commented 3 years ago

FYI: Was able to finish a plot with catchmeifyoucan's version on 4x Xeon, Windows Server 2016:

Final Directory: F: Number of Plots: 1 Process ID: 7600 Number of Threads: 64 Number of Buckets: 2^8 (256) Pool Public Key: MyKey Farmer Public Key: MyKey Working Directory: B: Working Directory 2: B: Plot Name: plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7 [P1] Table 1 took 216.814 sec [P1] Table 2 took 1016.06 sec, found 4294900520 matches [P1] Table 3 took 1395.56 sec, found 4294913933 matches [P1] Table 4 took 1694.66 sec, found 4294960724 matches [P1] Table 5 took 1683.73 sec, found 4294892549 matches [P1] Table 6 took 1618.51 sec, found 4294912001 matches [P1] Table 7 took 1036.07 sec, found 4294857450 matches Phase 1 took 8662.24 sec [P2] max_table_size = 4294967296 [P2] Table 7 scan took 41.5382 sec [P2] Table 7 rewrite took 132.856 sec, dropped 0 entries (0 %) [P2] Table 6 scan took 52.1252 sec [P2] Table 6 rewrite took 181.177 sec, dropped 581297235 entries (13.5346 %) [P2] Table 5 scan took 65.4004 sec [P2] Table 5 rewrite took 172.544 sec, dropped 761939563 entries (17.7406 %) [P2] Table 4 scan took 122.309 sec [P2] Table 4 rewrite took 175.116 sec, dropped 828818947 entries (19.2975 %) [P2] Table 3 scan took 124.939 sec [P2] Table 3 rewrite took 173.492 sec, dropped 855039177 entries (19.9082 %) [P2] Table 2 scan took 124.127 sec [P2] Table 2 rewrite took 171.112 sec, dropped 865567521 entries (20.1534 %) Phase 2 took 1594.5 sec Wrote plot header with 268 bytes [P3-1] Table 2 took 173.618 sec, wrote 3429332999 right entries [P3-2] Table 2 took 157.627 sec, wrote 3429332999 left entries, 3429332999 final [P3-1] Table 3 took 180.526 sec, wrote 3439874756 right entries [P3-2] Table 3 took 158.528 sec, wrote 3439874756 left entries, 3439874756 final [P3-1] Table 4 took 181.764 sec, wrote 3466141777 right entries [P3-2] Table 4 took 157.603 sec, wrote 3466141777 left entries, 3466141777 final [P3-1] Table 5 took 184.408 sec, wrote 3532952986 right entries [P3-2] Table 5 took 162.157 sec, wrote 3532952986 left entries, 3532952986 final [P3-1] Table 6 took 203.517 sec, wrote 3713614766 right entries [P3-2] Table 6 took 165.707 sec, wrote 3713614766 left entries, 3713614766 final [P3-1] Table 7 took 221.168 sec, wrote 4294857450 right entries [P3-2] Table 7 took 200.895 sec, wrote 4294857450 left entries, 4294857450 final Phase 3 took 2169.08 sec, wrote 21876774734 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 362.115 sec, final plot size is 108833505979 bytes Total plot creation time was 12788.3 sec Started copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot Copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot finished, took 207.259 sec, 0 MB/s avg.

Trying noavx now.

Curious how you make out with the noavx version also. I've got a Dell R910 w/ 4X E7-4870's and with the standard madmax v.0.0.4 if I deviate from the default amount of threads it seems to have a negative impact on the times and overall performance of the machine. (even when not at 100% CPU, lookup times for some external drives I have go through the roof - I know I should just plot on this rig but, no room for the externals anywhere else just yet for farming) With default threads going to a PCIe Fusion-IO card my best time so far was 3.4 hours which is amazing considering prior plotting time was 3-4x that to SAS drives and even my first couple runs to the PCIe card with the Official chia plotter were long, man the phase one could take more then 5+ hours! BTW for this run, did you go to a ramdisk? Got a plot going with the noavx version now with just slightly higher threads and it seems to not be impacting the machine as poorly, so we'll see. Thanks!

It was using an NVMe as TempDir. When I use a RamDisk it sometimes crashes in P1. Will do a memtest eventually. Stotiks NoAVX version also got me crashing with any drives (NVME/SSD/HDD/RamDisk).

The interesting thing is, that I only see high CPU utilization for the first few minutes (goes up 60-80%) and then plunges down to 30% and eventually 0%. There still read activity on the disks going on. When it plunges to 30% I see that it only uses 2x Physical CPUs instead of 4x available.

On my i9-9900k system, I see almost 100% CPU utliization all the time (using stotitks v.0.0.4).

Bro If possible, use ubuntu. I getting [P1] Table 1 took 300, when change win to ubuntu getting 35-40 sec. ;) try and then format nvme to ext4, and sorry for my bad english. :)

tydeno commented 3 years ago

FYI: Was able to finish a plot with catchmeifyoucan's version on 4x Xeon, Windows Server 2016:

Final Directory: F: Number of Plots: 1 Process ID: 7600 Number of Threads: 64 Number of Buckets: 2^8 (256) Pool Public Key: MyKey Farmer Public Key: MyKey Working Directory: B: Working Directory 2: B: Plot Name: plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7 [P1] Table 1 took 216.814 sec [P1] Table 2 took 1016.06 sec, found 4294900520 matches [P1] Table 3 took 1395.56 sec, found 4294913933 matches [P1] Table 4 took 1694.66 sec, found 4294960724 matches [P1] Table 5 took 1683.73 sec, found 4294892549 matches [P1] Table 6 took 1618.51 sec, found 4294912001 matches [P1] Table 7 took 1036.07 sec, found 4294857450 matches Phase 1 took 8662.24 sec [P2] max_table_size = 4294967296 [P2] Table 7 scan took 41.5382 sec [P2] Table 7 rewrite took 132.856 sec, dropped 0 entries (0 %) [P2] Table 6 scan took 52.1252 sec [P2] Table 6 rewrite took 181.177 sec, dropped 581297235 entries (13.5346 %) [P2] Table 5 scan took 65.4004 sec [P2] Table 5 rewrite took 172.544 sec, dropped 761939563 entries (17.7406 %) [P2] Table 4 scan took 122.309 sec [P2] Table 4 rewrite took 175.116 sec, dropped 828818947 entries (19.2975 %) [P2] Table 3 scan took 124.939 sec [P2] Table 3 rewrite took 173.492 sec, dropped 855039177 entries (19.9082 %) [P2] Table 2 scan took 124.127 sec [P2] Table 2 rewrite took 171.112 sec, dropped 865567521 entries (20.1534 %) Phase 2 took 1594.5 sec Wrote plot header with 268 bytes [P3-1] Table 2 took 173.618 sec, wrote 3429332999 right entries [P3-2] Table 2 took 157.627 sec, wrote 3429332999 left entries, 3429332999 final [P3-1] Table 3 took 180.526 sec, wrote 3439874756 right entries [P3-2] Table 3 took 158.528 sec, wrote 3439874756 left entries, 3439874756 final [P3-1] Table 4 took 181.764 sec, wrote 3466141777 right entries [P3-2] Table 4 took 157.603 sec, wrote 3466141777 left entries, 3466141777 final [P3-1] Table 5 took 184.408 sec, wrote 3532952986 right entries [P3-2] Table 5 took 162.157 sec, wrote 3532952986 left entries, 3532952986 final [P3-1] Table 6 took 203.517 sec, wrote 3713614766 right entries [P3-2] Table 6 took 165.707 sec, wrote 3713614766 left entries, 3713614766 final [P3-1] Table 7 took 221.168 sec, wrote 4294857450 right entries [P3-2] Table 7 took 200.895 sec, wrote 4294857450 left entries, 4294857450 final Phase 3 took 2169.08 sec, wrote 21876774734 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 362.115 sec, final plot size is 108833505979 bytes Total plot creation time was 12788.3 sec Started copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot Copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot finished, took 207.259 sec, 0 MB/s avg.

Trying noavx now.

Curious how you make out with the noavx version also. I've got a Dell R910 w/ 4X E7-4870's and with the standard madmax v.0.0.4 if I deviate from the default amount of threads it seems to have a negative impact on the times and overall performance of the machine. (even when not at 100% CPU, lookup times for some external drives I have go through the roof - I know I should just plot on this rig but, no room for the externals anywhere else just yet for farming) With default threads going to a PCIe Fusion-IO card my best time so far was 3.4 hours which is amazing considering prior plotting time was 3-4x that to SAS drives and even my first couple runs to the PCIe card with the Official chia plotter were long, man the phase one could take more then 5+ hours! BTW for this run, did you go to a ramdisk? Got a plot going with the noavx version now with just slightly higher threads and it seems to not be impacting the machine as poorly, so we'll see. Thanks!

It was using an NVMe as TempDir. When I use a RamDisk it sometimes crashes in P1. Will do a memtest eventually. Stotiks NoAVX version also got me crashing with any drives (NVME/SSD/HDD/RamDisk). The interesting thing is, that I only see high CPU utilization for the first few minutes (goes up 60-80%) and then plunges down to 30% and eventually 0%. There still read activity on the disks going on. When it plunges to 30% I see that it only uses 2x Physical CPUs instead of 4x available. On my i9-9900k system, I see almost 100% CPU utliization all the time (using stotitks v.0.0.4).

Bro If possible, use ubuntu. I getting [P1] Table 1 took 300, when change win to ubuntu getting 35-40 sec. ;) try and then format nvme to ext4, and sorry for my bad english. :)

I went for 4 parallel stotiks jobs, CPU is now used 90%+ all the time. Will try Ubuntu eventually.

celalanak commented 3 years ago

FYI: Was able to finish a plot with catchmeifyoucan's version on 4x Xeon, Windows Server 2016:

Final Directory: F: Number of Plots: 1 Process ID: 7600 Number of Threads: 64 Number of Buckets: 2^8 (256) Pool Public Key: MyKey Farmer Public Key: MyKey Working Directory: B: Working Directory 2: B: Plot Name: plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7 [P1] Table 1 took 216.814 sec [P1] Table 2 took 1016.06 sec, found 4294900520 matches [P1] Table 3 took 1395.56 sec, found 4294913933 matches [P1] Table 4 took 1694.66 sec, found 4294960724 matches [P1] Table 5 took 1683.73 sec, found 4294892549 matches [P1] Table 6 took 1618.51 sec, found 4294912001 matches [P1] Table 7 took 1036.07 sec, found 4294857450 matches Phase 1 took 8662.24 sec [P2] max_table_size = 4294967296 [P2] Table 7 scan took 41.5382 sec [P2] Table 7 rewrite took 132.856 sec, dropped 0 entries (0 %) [P2] Table 6 scan took 52.1252 sec [P2] Table 6 rewrite took 181.177 sec, dropped 581297235 entries (13.5346 %) [P2] Table 5 scan took 65.4004 sec [P2] Table 5 rewrite took 172.544 sec, dropped 761939563 entries (17.7406 %) [P2] Table 4 scan took 122.309 sec [P2] Table 4 rewrite took 175.116 sec, dropped 828818947 entries (19.2975 %) [P2] Table 3 scan took 124.939 sec [P2] Table 3 rewrite took 173.492 sec, dropped 855039177 entries (19.9082 %) [P2] Table 2 scan took 124.127 sec [P2] Table 2 rewrite took 171.112 sec, dropped 865567521 entries (20.1534 %) Phase 2 took 1594.5 sec Wrote plot header with 268 bytes [P3-1] Table 2 took 173.618 sec, wrote 3429332999 right entries [P3-2] Table 2 took 157.627 sec, wrote 3429332999 left entries, 3429332999 final [P3-1] Table 3 took 180.526 sec, wrote 3439874756 right entries [P3-2] Table 3 took 158.528 sec, wrote 3439874756 left entries, 3439874756 final [P3-1] Table 4 took 181.764 sec, wrote 3466141777 right entries [P3-2] Table 4 took 157.603 sec, wrote 3466141777 left entries, 3466141777 final [P3-1] Table 5 took 184.408 sec, wrote 3532952986 right entries [P3-2] Table 5 took 162.157 sec, wrote 3532952986 left entries, 3532952986 final [P3-1] Table 6 took 203.517 sec, wrote 3713614766 right entries [P3-2] Table 6 took 165.707 sec, wrote 3713614766 left entries, 3713614766 final [P3-1] Table 7 took 221.168 sec, wrote 4294857450 right entries [P3-2] Table 7 took 200.895 sec, wrote 4294857450 left entries, 4294857450 final Phase 3 took 2169.08 sec, wrote 21876774734 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 362.115 sec, final plot size is 108833505979 bytes Total plot creation time was 12788.3 sec Started copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot Copy to F:\plot-k32-2021-06-12-12-52-0bdc4aa012622c80501d8b040482d45736a3f4739bf7ecddb9e506a5d32546b7.plot finished, took 207.259 sec, 0 MB/s avg.

Trying noavx now.

Curious how you make out with the noavx version also. I've got a Dell R910 w/ 4X E7-4870's and with the standard madmax v.0.0.4 if I deviate from the default amount of threads it seems to have a negative impact on the times and overall performance of the machine. (even when not at 100% CPU, lookup times for some external drives I have go through the roof - I know I should just plot on this rig but, no room for the externals anywhere else just yet for farming) With default threads going to a PCIe Fusion-IO card my best time so far was 3.4 hours which is amazing considering prior plotting time was 3-4x that to SAS drives and even my first couple runs to the PCIe card with the Official chia plotter were long, man the phase one could take more then 5+ hours! BTW for this run, did you go to a ramdisk? Got a plot going with the noavx version now with just slightly higher threads and it seems to not be impacting the machine as poorly, so we'll see. Thanks!

It was using an NVMe as TempDir. When I use a RamDisk it sometimes crashes in P1. Will do a memtest eventually. Stotiks NoAVX version also got me crashing with any drives (NVME/SSD/HDD/RamDisk). The interesting thing is, that I only see high CPU utilization for the first few minutes (goes up 60-80%) and then plunges down to 30% and eventually 0%. There still read activity on the disks going on. When it plunges to 30% I see that it only uses 2x Physical CPUs instead of 4x available. On my i9-9900k system, I see almost 100% CPU utliization all the time (using stotitks v.0.0.4).

Bro If possible, use ubuntu. I getting [P1] Table 1 took 300, when change win to ubuntu getting 35-40 sec. ;) try and then format nvme to ext4, and sorry for my bad english. :)

I went for 4 parallel stotiks jobs, CPU is now used 90%+ all the time. Will try Ubuntu eventually.

I think for ubuntu don't think, don't wait. ;)