search

Chia-Network / bladebit

A high-performance k32-only, Chia (XCH) plotter supporting in-RAM and disk-based plotting
Apache License 2.0
340 stars 107 forks source link

STDERR: CUDA error cudaErrorIllegalAddress : an illegal memory access was encountered. #433

Closed shaneo257 closed 8 months ago

shaneo257 commented 8 months ago

Whilst plotting with the 128gb Hybrid plotter in 2.1.0 GUI i get the error STDERR: CUDA error cudaErrorIllegalAddress : an illegal memory access was encountered.

OS: Windows 10

Below is the log of the plotting process

Bladebit Chia Plotter Version : 3.1.0 Git Commit : e9836f8bd963321457bc86eb5d61344bfb76dcf0 Compiled With: msvc 19.29.30152

[Global Plotting Config] Will create 1 plots. Thread count : 24 Warm start enabled : false NUMA disabled : false CPU affinity disabled : false Farmer public key : Pool contract address : Compression Level : 4 Benchmark mode : disabled

[Bladebit CUDA Plotter] Host RAM : 127 GiB Plot checks : disabled

Selected cuda device 0 : NVIDIA GeForce GTX 1660 CUDA Compute Capability : 7.5 SM count : 22 Max blocks per SM : 16 Max threads per SM : 1024 Async Engine Count : 6 L2 cache size : 1.50 MB L2 persist cache max size : 0.00 MB Stack Size : 1.00 KB Memory: Total : 6.00 GB Free : 5.02 GB

Allocating buffers (this may take a few seconds)... Kernel RAM required : 92405843664 bytes ( 88125.08 MiB or 86.06 GiB ) Intermediate RAM required : 4378927104 bytes ( 4176.07 MiB or 4.08 GiB ) Host RAM required : 28420603904 bytes ( 27104.00 MiB or 26.47 GiB ) Total Host RAM required : 120826447568 bytes ( 115229.08 MiB or 112.53 GiB ) GPU RAM required : 6161465344 bytes ( 5876.03 MiB or 5.74 GiB ) Allocating buffers... Done.

Generating plot 1 / 1: cf6284ab402697c431d902ebde09514c8574756a07ae340eeb31dc292bb9aa1f Plot temporary file: F:\plot-k32-c04-2023-10-12-14-53-cf6284ab402697c431d902ebde09514c8574756a07ae340eeb31dc292bb9aa1f.plot.tmp

Generating F1 Progress update: 0.01 Finished F1 in 8.08 seconds. Progress update: 0.1 Table 2 completed in 112.01 seconds with 4294929037 entries. Progress update: 0.2 Table 3 completed in 307.73 seconds with 4294770529 entries. Progress update: 0.3 Table 4 completed in 274.57 seconds with 4294461195 entries. Progress update: 0.4 Table 5 completed in 259.13 seconds with 4293751630 entries. Progress update: 0.5 Table 6 completed in 254.86 seconds with 4292505069 entries. Progress update: 0.6 Table 7 completed in 123.13 seconds with 4290096020 entries. Progress update: 0.7 Finalizing Table 7 Finalized Table 7 in 22.11 seconds. Completed Phase 1 in 1362.26 seconds Progress update: 0.8 Marked Table 6 in 11.91 seconds. Marked Table 5 in 10.92 seconds. Marked Table 4 in 10.63 seconds. Marked Table 3 in 10.60 seconds. Completed Phase 2 in 44.07 seconds Progress update: 0.9 Compressing Table 2 and 3... STDERR: CUDA error: 700 (0x2bc) cudaErrorIllegalAddress : an illegal memory access was encountered

STDERR:

STDERR: Panic!!! Fatal Error:

STDERR: CUDA error cudaErrorIllegalAddress : an illegal memory access was encountered.

0x00007FF72E6293E2 @ :: 0x00007FF72E72ED79 @ :: 0x00007FF72E74CE0D @ :: 0x00007FF72E7722CA @ :: 0x00007FF72E771F38 @ :: 0x00007FF72E75892B @ :: 0x00007FF72E759CEB @ :: 0x00007FF72E750152 @ :: 0x00007FF72E7506BF @ :: 0x00007FF72E60F0A8 @ :: 0x00007FF72E7BAFEC @ :: 0x00007FFD31FB7344 @ ::BaseThreadInitThunk() 0x00007FFD33E626B1 @ ::RtlUserThreadStart()

sobertram commented 8 months ago

Selected cuda device 0 : NVIDIA GeForce GTX 1660 CUDA Compute Capability : 7.5 SM count : 22 Max blocks per SM : 16 Max threads per SM : 1024 Async Engine Count : 6 L2 cache size : 1.50 MB L2 persist cache max size : 0.00 MB Stack Size : 1.00 KB Memory: Total : 6.00 GB Free : 5.02 GB

GPU RAM required : 6161465344 bytes ( 5876.03 MiB or 5.74 GiB ) 5.74 GiB = 6.16327807 GB

harold-b commented 8 months ago

Good observation @sobertram. It seems the CUDA api allows the allocation to go through. @shaneo257 you might try setting the buckets to 256. Although this isn't technically officially supported, it should work and it lowers memory requirements. Other users with 6G GPUs have successfully plotted this way).

You'll have to change this line to say 256u instead of 128u. Then re-compile.

shaneo257 commented 8 months ago

thanks guys, i did think of that earlier today that i didn't have enough GPU RAM when i was plotting, framing and had nicehash all good, ive already committed to GH now