search

sp-hash / TeamBlackMiner

Fast and stable mining software for Ethereum POW, Ethereum Classic, Zilliqa, Ravencoin (KAWPOW), Firo (FIROPOW), MEOWPOW, EVRPROGPOW , Vertcoin and Hypra.
92 stars 28 forks source link

[BUG] Cannot allocate memory!: Not enough space #9

Closed RainbowMiner closed 3 years ago

RainbowMiner commented 3 years ago

The miner v1.0 crashes regularly on a 6xGTX1070 / Windows 10 / CUDA 11.3 (Nvidia v466.74) mining rig while mining Ethash to Nicehash. DRAM is 8GB, pagefile/virtual memory is set to 120GB (120000). Sometimes it crashes without the cuda runtime error, but always with twice Critical error. Cannot allocate memory!: Not enough space

     3m [2021-09-10 15:22:50.224] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
     3m [2021-09-10 15:22:50.224] Shutting down threads
     3m [2021-09-10 15:22:50.224] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
     3m [2021-09-10 15:22:50.224] Shutting down threads
     3m [2021-09-10 15:22:50.240] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
Critical error. Cannot allocate memory!: Not enough space
Critical error. Cannot allocate memory!: Not enough space
RainbowMiner commented 3 years ago

Same happens on 6xGTX1070 / Windows 10 / CUDA 11.4 (Nvidia v471.41), mining Ethash to Ethermine. DRAM is 8GB, pagefile/virtual memory is set to 120GB (120000). 

     3m [2021-09-10 16:22:14.308] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: unspecified launch failure
     3m [2021-09-10 16:22:14.308] Shutting down threads
     3m [2021-09-10 16:22:14.323] Stratum thread exited
     3m [2021-09-10 16:22:14.323] Control thread exited
     3m [2021-09-10 16:22:14.339] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: unspecified launch failure
     3m [2021-09-10 16:22:14.339] Shutting down threads
     3m [2021-09-10 16:22:14.339] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: unspecified launch failure
     3m [2021-09-10 16:22:14.339] Shutting down threads
     3m [2021-09-10 16:22:14.339] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: unspecified launch failure
     3m [2021-09-10 16:22:14.339] Shutting down threads
Critical error. Cannot allocate memory!: Not enough space
Critical error. Cannot allocate memory!: Not enough space
     3m [2021-09-10 16:22:14.339] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cu
sp-hash commented 3 years ago

Add more virtual memory 32gb

RainbowMiner commented 3 years ago

I have set the virtual memory to 160gb (up from 120gb), now. But it still happens. This time etchash on Ethermine, after 2-3 minutes runtime

     2m [2021-09-10 21:08:35.811] ID BOARD   TEMP    FAN CORE    MEM WATT    kW/h    COST/h
     2m [2021-09-10 21:08:35.811] GPU0  1070    63  43  1956    4404    135 0.14    inf
     2m [2021-09-10 21:08:35.811] GPU1  1070    67  62  1939    4404    117 0.12    inf
     2m [2021-09-10 21:08:35.811] GPU2  1070    68  66  1911    4404    135 0.14    inf
     2m [2021-09-10 21:08:35.811] GPU3  1070    68  66  1863    4404    134 0.13    inf
     2m [2021-09-10 21:08:35.811] GPU4  1070    63  50  1936    4404    132 0.13    inf
     2m [2021-09-10 21:08:35.811] GPU5  1070    65  54  1927    4404    135 0.14    inf
     2m [2021-09-10 21:08:35.811]                         788 0.79    inf
     2m [2021-09-10 21:08:35.811] 
     2m [2021-09-10 21:08:35.811] ID BOARD   TYPE    HASHRATE/W  HASHRATE    AVERAGE     SHARES
     2m [2021-09-10 21:08:35.811] GPU0  1070    Cuda    231.78 kH/s 31.29 MH/s  31.29 MH/s  0/0 (100.00)
Critical error. Cannot allocate memory!: Not enough space
Critical error. Cannot allocate memory!: Not enough space
sp-hash commented 3 years ago

Did you try v1.01? Which driver and exe are you using?

RainbowMiner commented 3 years ago

Yes, v1.01 - Nvidia v471.41 thus CUDA11.4 It is happening during Etchash and Ethash mining.

sp-hash commented 3 years ago

Use gpu-z and check if you have enough memory. Perhaps a process is not shut down correctly or you didn't wait long enough when killing the process. What's the launch string of the miner`?

Btw, with no colors use the option:

-I --no-ansi

RainbowMiner commented 3 years ago

I have checked with gpu-z - there is more than enough memory all the time. So it's something different. Could that crash be related to overclocking or damaged DAGs? This morning, the miner (on ethash) ran long time without crash, but it seem to have lost one GPU after 5 minutes GPU 3: has timed out. From that time on, it reported 0 H/s for that specific GPU. Also, it seems, that the etchash does crash the miner more often.

Thanks for the --no-ansi hint :) makes the logs much more readable.

This is the last crash-error while mining etchash:

     3m [2021-09-11 14:05:17.026] Job 27d501
     3m [2021-09-11 14:05:40.710] Job 1d7daa
     3m [2021-09-11 14:06:02.057] GPU0 Solution found in 21.90s (Submit every 30.01 seconds)
     3m [2021-09-11 14:06:02.099] Accepted (8/8) Reply: 41ms
     3m [2021-09-11 14:06:02.947] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
     3m [2021-09-11 14:06:02.947] Shutting down threads
     3m [2021-09-11 14:06:02.953] Control thread exited
     3m [2021-09-11 14:06:02.997] Stratum thread exited
     3m [2021-09-11 14:06:02.997] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
     3m [2021-09-11 14:06:02.997] Shutting down threads
     3m [2021-09-11 14:06:02.997] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
     3m [2021-09-11 14:06:02.997] Shutting down threads
     3m [2021-09-11 14:06:02.997] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
     3m [2021-09-11 14:06:02.997] Shutting down threads
     3m [2021-09-11 14:06:02.997] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
     3m [2021-09-11 14:06:02.997] Shutting down threads
     3m [2021-09-11 14:06:02.997] C:\gc\TBMiner\source\sp_cuda\cudaminer.cpp:cuda_search():349: A cuda runtime error occured: an illegal memory access was encountered
     3m [2021-09-11 14:06:02.997] Shutting down threads
Critical error. Cannot allocate memory!: Not enough space
Critical error. Cannot allocate memory!: Not enough space
Critical error. Cannot allocate memory!: Not enough space
Critical error. Cannot allocate memory!: Not enough space
sp-hash commented 3 years ago

Thanks for the report. The error is because the miner doesn't shut down properly. If the gpu crash, the hashrate will go to 0. This is probobly because the clock is to high, or the rig is using too much power. Your log indicate a crash in one of the gpus that cause the whole program to exit.

To make sure the miner exits properly run it in a loop with a timeout.

:loop
    TBMiner --algo ethash --hostname s1.solopool.org --port 8002 --wallet %ETH_WALLET% --worker_name %computername%
    timeout 10
goto :loop
sp-hash commented 3 years ago

v1.02 will have a small code change that might help here.

sp-hash commented 3 years ago

v1.02 beta has been released with a possible fix

RainbowMiner commented 3 years ago

Thank you! I have updated the RainbowMiner library with the v1.02. Now I have to wait some hours, until one of my rigs isn't rented any more. I'll report back as soon as the new version gets benchmarked.

sp-hash commented 3 years ago

fixed in v1.02