psychocrypt
commented
6 years ago reduce the number of threads.
Closed NdranC closed 6 years ago
psychocrypt
commented
6 years ago reduce the number of threads.
NdranC
commented
6 years ago Why would the number of threads matter if each card is able to handle that load alone? Do I have to calculate threads and blocks differently if both cards work at the same time?
psychocrypt
commented
6 years ago You running out ouf memory. Therefore you need to reduce the thread number. Or disable the x server.
NdranC
commented
6 years ago I might be missing something. The error doesn't seem to be an out of memory one. Each card is able to mine with those threads just fine as long as I don't run them together. Unless you are talking about some other memory instead of vram.
Do you mind elaborating why 2 cards that work individually at those threads give an "what(): [CUDA] Error: all CUDA-capable devices are busy or unavailable" when I try to launch them together? Even without xorg running.
psychocrypt
commented
6 years ago Oh yes I looked only to the source code line and this is the allocation of the scratchpad memory. Could you please try to compile the dev branch. I fixed something in the dev for cuda but currently can not remembet what and searching whith my smartphone is not fun.
NdranC
commented
6 years ago I just compiled the dev branch and I still get the same error. Out of curiosity I reduced the threads to under 10 on both cards and that didn't have any effect on the outcome either. Both cards are still able to work with those settings as long as I have one of them commented out.
I also tried to run two separate instances of xmr-stak with different configs (one for each card) but I always get
WARNING: NVIDIA Unable to query number of CUDA devices! WARNING: NVIDIA no device found [2017-12-02 03:44:25] : WARNING: backend NVIDIA disabled. [2017-12-02 03:44:25] : WARNING: UNKNOWN_ERROR when calling clGetPlatformIDs for number of platforms.
when I try to launch the second instance. I assume this is normal but I wanted to try anyway.
psychocrypt
commented
6 years ago please remove nvidia.txt and start the miner again. Please post the full output.
NdranC
commented
6 years ago [2017-12-02 03:50:31] : MEMORY ALLOC FAILED: mmap failed [2017-12-02 03:50:31] : MEMORY ALLOC FAILED: mmap failed
xmr-stak 2.0.0 2920e9a
Brought to you by fireice_uk and psychocrypt under GPLv3. Based on CPU mining code by wolf9466 (heavily optimized by fireice_uk). Based on NVIDIA mining code by KlausT and psychocrypt. Based on OpenCL mining code by wolf9466.
Configurable dev donation level is set to 2.0%
You can use following keys to display reports: 'h' - hashrate 'r' - results 'c' - connection
[2017-12-02 03:50:31] : Start mining: MONERO [2017-12-02 03:50:31] : NVIDIA: GPU configuration stored in file 'nvidia.txt' [2017-12-02 03:50:31] : Starting NVIDIA GPU thread 0, no affinity. [2017-12-02 03:50:31] : Starting NVIDIA GPU thread 1, no affinity. [2017-12-02 03:50:31] : MEMORY ALLOC FAILED: mmap failed [2017-12-02 03:50:31] : MEMORY ALLOC FAILED: mmap failed [2017-12-02 03:50:31] : WARNING: No AMD OpenCL platform found. Possible driver issues or wrong vendor driver. [2017-12-02 03:50:31] : WARNING: backend AMD disabled. [2017-12-02 03:50:31] : Starting single thread, affinity: 0. [CUDA] Error gpu 0: </home/ndranc/xmr-stak-dev/xmrstak/backend/nvidia/nvcc_code/cuda_extra.cu>:206 terminate called after throwing an instance of 'std::runtime_error' what(): [CUDA] Error: all CUDA-capable devices are busy or unavailable [1] 6695 abort (core dumped) ./xmr-stak-dev/build/bin/xmr-stak
This is the nvidia.txt generated (without the comments)
"gpu_threads_conf" : [ // gpu: GeForce GTX 760 architecture: 30 // memory: 1529/1991 MiB // smx: 6 { "index" : 0, "threads" : 38, "blocks" : 18, "bfactor" : 0, "bsleep" : 0, "affine_to_cpu" : false, }, // gpu: GeForce GTX 460 architecture: 21 // memory: 886/964 MiB // smx: 7 { "index" : 1, "threads" : 26, "blocks" : 14, "bfactor" : 0, "bsleep" : 0, "affine_to_cpu" : false, },
],
NdranC
commented
6 years ago Is there anything else I can do to provide more info on this issue? Any clues if it's a problem on my end or the miner?
Sorry to bother, not being able to mine with one of the cards feels awful.
psychocrypt
commented
6 years ago you could try to start the miner with root rights sudo ./xmr-stak if this
works than your user is not allowed to acces the gpus.
NdranC
commented
6 years ago I have been doing this already. Somewhere else on the documentation I was reading about large pages and even though I didn't read carefully (at the time I was figuring how to compile with cuda8 and gcc5) I did read that launching the script with sudo would avoid some of those issues. At least until I configure that properly.
Is there a log file I could access that would be a little more verbose about the error?
I doubt there will be anything here but this are the last entries of my dmesg. Should I normally be able to launch 2 separate instances of the script with different configs that is targeted to each card? When I try to do that I get a different error but I'm not sure if that is even normally possible.
[ 78.527837] nvidia: module license 'NVIDIA' taints kernel. [ 78.527838] Disabling lock debugging due to kernel taint [ 78.533456] nvidia-nvlink: Nvlink Core is being initialized, major device number 242 [ 78.533636] nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem [ 78.533699] nvidia 0000:03:00.0: enabling device (0000 -> 0003) [ 78.533746] nvidia 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none [ 78.533782] NVRM: loading NVIDIA UNIX x86_64 Kernel Module 387.34 Tue Nov 21 03:09:00 PST 2017 (using threaded interrupts) [ 78.533969] fuse init (API version 7.26) [ 78.555198] ppdev: user-space parallel port driver [ 78.567931] intel_rapl: Found RAPL domain package [ 78.567932] intel_rapl: Found RAPL domain core [ 78.567933] intel_rapl: Found RAPL domain dram [ 78.569541] nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms 387.34 Tue Nov 21 02:09:45 PST 2017 [ 78.571238] [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver [ 78.571238] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 0 [ 78.571269] [drm] [nvidia-drm] [GPU ID 0x00000300] Loading driver [ 78.571270] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:03:00.0 on minor 1 [ 78.621457] iTCO_vendor_support: vendor-support=0 [ 78.622333] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11 [ 78.622442] r8169 0000:05:00.0 enp5s0: renamed from eth0 [ 78.622452] iTCO_wdt: Found a Intel PCH TCO device (Version=4, TCOBASE=0x0400) [ 78.622523] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0) [ 78.648337] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: data=ordered [ 78.726878] dw-apb-uart.2: ttyS2 at MMIO 0xde22b000 (irq = 20, base_baud = 115200) is a 16550A [ 79.267601] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input21 [ 79.267623] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input22 [ 79.267642] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input23 [ 79.267660] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:01.0/0000:01:00.1/sound/card1/input24 [ 79.473605] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:1c.4/0000:03:00.1/sound/card2/input25 [ 79.473628] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:1c.4/0000:03:00.1/sound/card2/input26 [ 79.473648] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:1c.4/0000:03:00.1/sound/card2/input27 [ 79.473666] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:1c.4/0000:03:00.1/sound/card2/input28 [ 79.720331] input: ckb1: Corsair STRAFE RGB Gaming Keyboard as /devices/virtual/input/input29 [ 79.720562] input: ckb1: Corsair STRAFE RGB Gaming Keyboard as /devices/virtual/input/input30 [ 79.843105] IPv6: ADDRCONF(NETDEV_UP): enp5s0: link is not ready [ 79.859503] r8169 0000:05:00.0 enp5s0: link down [ 79.859506] r8169 0000:05:00.0 enp5s0: link down [ 79.859582] IPv6: ADDRCONF(NETDEV_UP): enp5s0: link is not ready [ 83.394694] r8169 0000:05:00.0 enp5s0: link up [ 83.394698] IPv6: ADDRCONF(NETDEV_CHANGE): enp5s0: link becomes ready [ 90.655664] NVRM: Your system is not currently configured to drive a VGA console [ 90.656083] nvidia-modeset: Allocated GPU:0 (GPU-9db0b4b2-1237-4d91-77fa-81fca932d7ea) @ PCI:0000:01:00.0 [ 90.768676] ACPI Warning: _SB.PCI0.RP05.PXSX._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20170531/nsarguments-95) [ 90.768714] ACPI Warning: _SB.PCI0.RP05.PXSX._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20170531/nsarguments-95) [ 90.768735] ACPI Warning: _SB.PCI0.RP05.PXSX._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20170531/nsarguments-95) [ 90.768755] ACPI Warning: _SB.PCI0.RP05.PXSX._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20170531/nsarguments-95) [ 90.768774] ACPI Warning: _SB.PCI0.RP05.PXSX._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20170531/nsarguments-95) [ 90.768793] ACPI Warning: _SB.PCI0.RP05.PXSX._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20170531/nsarguments-95) [ 90.768813] ACPI Warning: _SB.PCI0.RP05.PXSX._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20170531/nsarguments-95) [ 91.537194] nvidia-modeset: Allocated GPU:1 (GPU-4dce6e35-c7c3-131b-f44b-57c414e49522) @ PCI:0000:03:00.0 [ 96.420158] input: Microsoft X-Box 360 pad as /devices/virtual/input/input31 [ 181.457340] nvidia-uvm: Loaded the UVM driver in 8 mode, major device number 241 [ 2281.287272] perf: interrupt took too long (2501 > 2500), lowering kernel.perf_event_max_samplerate to 79800 [20525.408474] Chrome~dThread[8739]: segfault at 0 ip 00007fd46c0f7073 sp 00007fd46a83eb10 error 6 in libxul.so[7fd46be52000+438a000] [20525.408527] Chrome~dThread[1189]: segfault at 0 ip 00007f52c3a02d4d sp 00007f52c214db10 error 6 in libxul.so (deleted)[7f52c3760000+437b000] [20525.408779] Chrome~dThread[1505]: segfault at 0 ip 00007f34b2a02d4d sp 00007f34b114db10 error 6 in libxul.so (deleted)[7f34b2760000+437b000]
psychocrypt
commented
6 years ago large page support is something else let us focus first on the gpu issue. You wrote some errors are gone if you run it as root. Is the gpu not availavle error gone?
NdranC
commented
6 years ago No the error is still there with or without root. This error only happens when I try to run the miner with both cards on the "nvidia.txt" config. If I comment out one card or the other one the miner runs perfectly.
I get a different error if I try to run 2 separate instances of the miner (trying something different). The first one configured to only use the CPU and the GTX760 runs fine, but when I tried to run a second separate instance with only the GTX460 I get this error instead
WARNING: NVIDIA Unable to query number of CUDA devices! WARNING: NVIDIA no device found [2017-12-03 07:00:35] : WARNING: backend NVIDIA disabled. [2017-12-03 07:00:35] : WARNING: UNKNOWN_ERROR when calling clGetPlatformIDs for number of platforms. [2017-12-03 07:00:35] : WARNING: No OpenCL platform found. [2017-12-03 07:00:35] : WARNING: No AMD OpenCL platform found. Possible driver issues or wrong vendor driver. [2017-12-03 07:00:35] : WARNING: backend AMD disabled. [2017-12-03 07:00:35] : ERROR: No miner backend enabled.
psychocrypt
commented
6 years ago pease post your commands and output you used to compile the miner.
NdranC
commented
6 years ago git clone https://github.com/fireice-uk/xmr-stak.git -b dev mv xmr-stak xmr-stak-dev mkdir xmr-stak-dev/build cd xmr-stak-dev/build export CXX=/usr/bin/g++-5 export CC=/usr/bin/gcc-5 cmake ..
-- The C compiler identification is GNU 5.5.0 -- The CXX compiler identification is GNU 5.5.0 -- Check for working C compiler: /usr/bin/gcc-5 -- Check for working C compiler: /usr/bin/gcc-5 -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/g++-5 -- Check for working CXX compiler: /usr/bin/g++-5 -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Set miner currency to 'monero' and 'aeon' -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
-- Found CUDA: /opt/cuda (found suitable version "8.0", minimum required is "7.5") -- Looking for CL_VERSION_2_2 -- Looking for CL_VERSION_2_2 - found -- Found OpenCL: /usr/lib/libOpenCL.so (found version "2.2") -- Found OpenSSL: /usr/lib/libcrypto.so (found version "1.1.0g") -- Configuring done -- Generating done -- Build files have been written to: /home/ndranc/xmr-stak-dev/build
make install
Scanning dependencies of target xmr-stak-c [ 2%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_blake256.c.o [ 5%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_groestl.c.o [ 8%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_jh.c.o [ 11%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_keccak.c.o [ 14%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_skein.c.o [ 17%] Linking C static library bin/libxmr-stak-c.a [ 17%] Built target xmr-stak-c Scanning dependencies of target xmr-stak-backend [ 20%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/jconf.cpp.o [ 23%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/version.cpp.o [ 26%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/hwlocMemory.cpp.o [ 29%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/jconf.cpp.o [ 32%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/minethd.cpp.o [ 35%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/backendConnector.cpp.o [ 38%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/globalStates.cpp.o [ 41%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/crypto/cryptonight_common.cpp.o [ 44%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/http/httpd.cpp.o [ 47%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/http/webdesign.cpp.o [ 50%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/console.cpp.o [ 52%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/executor.cpp.o [ 55%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/telemetry.cpp.o [ 58%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/utility.cpp.o [ 61%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/net/jpsock.cpp.o [ 64%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/net/socket.cpp.o [ 67%] Linking CXX static library bin/libxmr-stak-backend.a [ 67%] Built target xmr-stak-backend [ 70%] Building NVCC (Device) object CMakeFiles/xmrstak_cuda_backend.dir/xmrstak/backend/nvidia/nvcc_code/xmrstak_cuda_backend_generated_cuda_extra.cu.o nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning). nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning). [ 73%] Building NVCC (Device) object CMakeFiles/xmrstak_cuda_backend.dir/xmrstak/backend/nvidia/nvcc_code/xmrstak_cuda_backend_generated_cuda_core.cu.o nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning). nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning). Scanning dependencies of target xmrstak_cuda_backend [ 76%] Building CXX object CMakeFiles/xmrstak_cuda_backend.dir/xmrstak/backend/nvidia/jconf.cpp.o [ 79%] Building CXX object CMakeFiles/xmrstak_cuda_backend.dir/xmrstak/backend/nvidia/minethd.cpp.o [ 82%] Linking CXX shared library bin/libxmrstak_cuda_backend.so [ 82%] Built target xmrstak_cuda_backend Scanning dependencies of target xmrstak_opencl_backend [ 85%] Building CXX object CMakeFiles/xmrstak_opencl_backend.dir/xmrstak/backend/amd/amd_gpu/gpu.cpp.o [ 88%] Building CXX object CMakeFiles/xmrstak_opencl_backend.dir/xmrstak/backend/amd/jconf.cpp.o [ 91%] Building CXX object CMakeFiles/xmrstak_opencl_backend.dir/xmrstak/backend/amd/minethd.cpp.o [ 94%] Linking CXX shared library bin/libxmrstak_opencl_backend.so [ 94%] Built target xmrstak_opencl_backend Scanning dependencies of target xmr-stak [ 97%] Building CXX object CMakeFiles/xmr-stak.dir/xmrstak/cli/cli-miner.cpp.o [100%] Linking CXX executable bin/xmr-stak [100%] Built target xmr-stak Install the project... -- Install configuration: "Release" xmr-stak installed to folder 'bin'
cd bin sudo ./xmr-stak
[2017-12-03 07:21:31] : MEMORY ALLOC FAILED: mmap failed [2017-12-03 07:21:31] : MEMORY ALLOC FAILED: mmap failed
xmr-stak 2.0.0 2920e9a
Brought to you by fireice_uk and psychocrypt under GPLv3. Based on CPU mining code by wolf9466 (heavily optimized by fireice_uk). Based on NVIDIA mining code by KlausT and psychocrypt. Based on OpenCL mining code by wolf9466.
Configurable dev donation level is set to 2.0%
You can use following keys to display reports: 'h' - hashrate 'r' - results 'c' - connection
[2017-12-03 07:21:31] : Start mining: MONERO [2017-12-03 07:21:31] : Starting NVIDIA GPU thread 0, no affinity. [2017-12-03 07:21:31] : Starting NVIDIA GPU thread 1, no affinity. [2017-12-03 07:21:31] : MEMORY ALLOC FAILED: mmap failed [2017-12-03 07:21:31] : MEMORY ALLOC FAILED: mmap failed [2017-12-03 07:21:31] : WARNING: No AMD OpenCL platform found. Possible driver issues or wrong vendor driver. [2017-12-03 07:21:31] : WARNING: backend AMD disabled. [CUDA] Error gpu 0: </home/ndranc/xmr-stak-dev/xmrstak/backend/nvidia/nvcc_code/cuda_extra.cu>:206 terminate called after throwing an instance of 'std::runtime_error' what(): [CUDA] Error: all CUDA-capable devices are busy or unavailable [1] 2423 abort sudo ./xmr-stak
vim nvidia.txt (I comment the entry for gtx760 at this point) sudo ./xmr-stak
[2017-12-03 07:23:32] : MEMORY ALLOC FAILED: mmap failed [2017-12-03 07:23:32] : MEMORY ALLOC FAILED: mmap failed
xmr-stak 2.0.0 2920e9a
Brought to you by fireice_uk and psychocrypt under GPLv3. Based on CPU mining code by wolf9466 (heavily optimized by fireice_uk). Based on NVIDIA mining code by KlausT and psychocrypt. Based on OpenCL mining code by wolf9466.
Configurable dev donation level is set to 2.0%
You can use following keys to display reports: 'h' - hashrate 'r' - results 'c' - connection
[2017-12-03 07:23:32] : Start mining: MONERO [2017-12-03 07:23:32] : Starting NVIDIA GPU thread 0, no affinity. [2017-12-03 07:23:32] : MEMORY ALLOC FAILED: mmap failed [2017-12-03 07:23:32] : WARNING: No AMD OpenCL platform found. Possible driver issues or wrong vendor driver. [2017-12-03 07:23:32] : WARNING: backend AMD disabled. [2017-12-03 07:23:32] : CPU configuration stored in file 'cpu.txt' [2017-12-03 07:23:32] : Starting single thread, affinity: 0. [2017-12-03 07:23:32] : hwloc: memory pinned [2017-12-03 07:23:32] : Starting single thread, affinity: 1. [2017-12-03 07:23:32] : MEMORY ALLOC FAILED: mmap failed [2017-12-03 07:23:32] : hwloc: memory pinned [2017-12-03 07:23:32] : Starting single thread, affinity: 2. [2017-12-03 07:23:32] : MEMORY ALLOC FAILED: mmap failed [2017-12-03 07:23:32] : hwloc: memory pinned [2017-12-03 07:23:32] : MEMORY ALLOC FAILED: mmap failed [2017-12-03 07:23:32] : Fast-connecting to stratum+tcp://xmr.mixpools.org:8080 pool ... [2017-12-03 07:23:32] : Pool stratum+tcp://xmr.mixpools.org:8080 connected. Logging in... [2017-12-03 07:23:33] : Difficulty changed. Now: 20000. [2017-12-03 07:23:33] : Pool logged in.
It doesn't matter which card I remove from the nvidia.txt config. They both work fine as long as it just one of them alone plus the CPU.
psychocrypt
commented
6 years ago Thx for the summery they give me avlot of information. I need to think about the issue.
surfinoma
commented
6 years ago Hi,
just wanted to inform you, that I have EXACTLY the same issue here.
My combination: Tesla C2075/Quadro K620.
Regards
surfinoma
commented
6 years ago What I did (CentOS 7.4)...
# git clone https://github.com/fireice-uk/xmr-stak.git -b dev xmr-stak-dev.git
Cloning into 'xmr-stak-dev.git'...
remote: Counting objects: 2772, done.
remote: Total 2772 (delta 0), reused 0 (delta 0), pack-reused 2772
Receiving objects: 100% (2772/2772), 844.70 KiB | 242.00 KiB/s, done.
Resolving deltas: 100% (1930/1930), done.
# cd xmr-stak-dev.git && mkdir build && cd build
# scl enable devtoolset-4 bash
# cmake3 -DOpenCL_ENABLE=OFF -DXMR-STAK_LARGEGRID=OFF -DCMAKE_LINK_STATIC=ON -DXMR-STAK_COMPILE=generic ..
-- The C compiler identification is GNU 5.3.1
-- The CXX compiler identification is GNU 5.3.1
-- Check for working C compiler: /opt/rh/devtoolset-4/root/usr/bin/cc
-- Check for working C compiler: /opt/rh/devtoolset-4/root/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/rh/devtoolset-4/root/usr/bin/c++
-- Check for working CXX compiler: /opt/rh/devtoolset-4/root/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Set miner currency to 'monero' and 'aeon'
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found suitable version "8.0", minimum required is "7.5")
-- Found OpenSSL: /usr/lib64/libssl.so;/usr/lib64/libcrypto.so (found version "1.0.2k")
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/src/xmr-stak-dev.git/build
# make
Scanning dependencies of target xmr-stak-c
[ 3%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_blake256.c.o
[ 6%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_groestl.c.o
[ 10%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_jh.c.o
[ 13%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_keccak.c.o
[ 16%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_skein.c.o
[ 20%] Linking C static library bin/libxmr-stak-c.a
[ 20%] Built target xmr-stak-c
Scanning dependencies of target xmr-stak-backend
[ 23%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/jconf.cpp.o
[ 26%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/version.cpp.o
[ 30%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/hwlocMemory.cpp.o
[ 33%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/jconf.cpp.o
[ 36%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/minethd.cpp.o
[ 40%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/backendConnector.cpp.o
[ 43%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/globalStates.cpp.o
[ 46%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/crypto/cryptonight_common.cpp.o
[ 50%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/http/httpd.cpp.o
[ 53%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/http/webdesign.cpp.o
[ 56%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/console.cpp.o
[ 60%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/executor.cpp.o
[ 63%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/telemetry.cpp.o
[ 66%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/utility.cpp.o
[ 70%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/net/jpsock.cpp.o
[ 73%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/net/socket.cpp.o
[ 76%] Linking CXX static library bin/libxmr-stak-backend.a
[ 76%] Built target xmr-stak-backend
Scanning dependencies of target xmr-stak
[ 80%] Building CXX object CMakeFiles/xmr-stak.dir/xmrstak/cli/cli-miner.cpp.o
[ 83%] Linking CXX executable bin/xmr-stak
[ 83%] Built target xmr-stak
[ 86%] Building NVCC (Device) object CMakeFiles/xmrstak_cuda_backend.dir/xmrstak/backend/nvidia/nvcc_code/xmrstak_cuda_backend_generated_cuda_extra.cu.o
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
[ 90%] Building NVCC (Device) object CMakeFiles/xmrstak_cuda_backend.dir/xmrstak/backend/nvidia/nvcc_code/xmrstak_cuda_backend_generated_cuda_core.cu.o
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
Scanning dependencies of target xmrstak_cuda_backend
[ 93%] Building CXX object CMakeFiles/xmrstak_cuda_backend.dir/xmrstak/backend/nvidia/jconf.cpp.o
[ 96%] Building CXX object CMakeFiles/xmrstak_cuda_backend.dir/xmrstak/backend/nvidia/minethd.cpp.o
[100%] Linking CXX shared library bin/libxmrstak_cuda_backend.so
[100%] Built target xmrstak_cuda_backend
# make install
# cd /opt/xmr-stak-dev
# ./xmr-stak --noCPU
-------------------------------------------------------------------
xmr-stak 2.0.0 2920e9a
Brought to you by fireice_uk and psychocrypt under GPLv3.
Based on CPU mining code by wolf9466 (heavily optimized by fireice_uk).
Based on NVIDIA mining code by KlausT and psychocrypt.
Configurable dev donation level is set to 2.0%
You can use following keys to display reports:
'h' - hashrate
'r' - results
'c' - connection
-------------------------------------------------------------------
[2017-12-03 18:29:03] : Start mining: MONERO
[2017-12-03 18:29:03] : Starting NVIDIA GPU thread 0, no affinity.
[2017-12-03 18:29:03] : Starting NVIDIA GPU thread 1, no affinity.
[2017-12-03 18:29:03] : Fast-connecting to cryptonight.eu.nicehash.com:3355 pool ...
[2017-12-03 18:29:03] : Fast-connecting to cryptonight.usa.nicehash.com:3355 pool ...
[2017-12-03 18:29:03] : Pool cryptonight.eu.nicehash.com:3355 connected. Logging in...
[2017-12-03 18:29:03] : Pool cryptonight.usa.nicehash.com:3355 connected. Logging in...
[CUDA] Error gpu 1: </usr/local/src/xmr-stak-dev.git/xmrstak/backend/nvidia/nvcc_code/cuda_extra.cu>:206
terminate called after throwing an instance of 'std::runtime_error'
what(): [CUDA] Error: all CUDA-capable devices are busy or unavailable
# cat nvidia.txt
"gpu_threads_conf" :
[
// gpu: Tesla C2075 architecture: 20
// memory: 5205/5301 MiB
// smx: 14
{ "index" : 0,
"threads" : 24, "blocks" : 42,
"bfactor" : 0, "bsleep" : 0,
"affine_to_cpu" : false,
},
// gpu: Quadro K620 architecture: 50
// memory: 1969/2000 MiB
// smx: 3
{ "index" : 1,
"threads" : 32, "blocks" : 9,
"bfactor" : 0, "bsleep" : 0,
"affine_to_cpu" : false,
},
],
Splitting nvidia.txt into two separate files & running two instances of xmr-stak concurrently ...
# ./xmr-stak --noCPU --nvidia nvidia0.txt
-------------------------------------------------------------------
xmr-stak 2.0.0 2920e9a
Brought to you by fireice_uk and psychocrypt under GPLv3.
Based on CPU mining code by wolf9466 (heavily optimized by fireice_uk).
Based on NVIDIA mining code by KlausT and psychocrypt.
Configurable dev donation level is set to 2.0%
You can use following keys to display reports:
'h' - hashrate
'r' - results
'c' - connection
-------------------------------------------------------------------
[2017-12-03 18:24:12] : Start mining: MONERO
[2017-12-03 18:24:12] : Starting NVIDIA GPU thread 0, no affinity.
[2017-12-03 18:24:12] : Fast-connecting to cryptonight.eu.nicehash.com:3355 pool ...
[2017-12-03 18:24:12] : Fast-connecting to cryptonight.usa.nicehash.com:3355 pool ...
[2017-12-03 18:24:12] : Pool cryptonight.eu.nicehash.com:3355 connected. Logging in...
[2017-12-03 18:24:12] : Pool cryptonight.usa.nicehash.com:3355 connected. Logging in...
[2017-12-03 18:24:12] : Difficulty changed. Now: 100001.
[2017-12-03 18:24:12] : Pool logged in.
[2017-12-03 18:24:38] : Difficulty changed. Now: 100001.
[2017-12-03 18:24:38] : New block detected.
[2017-12-03 18:24:52] : New block detected.
.
.
.
# ./xmr-stak --noCPU --nvidia nvidia1.txt
-------------------------------------------------------------------
xmr-stak 2.0.0 2920e9a
Brought to you by fireice_uk and psychocrypt under GPLv3.
Based on CPU mining code by wolf9466 (heavily optimized by fireice_uk).
Based on NVIDIA mining code by KlausT and psychocrypt.
Configurable dev donation level is set to 2.0%
You can use following keys to display reports:
'h' - hashrate
'r' - results
'c' - connection
-------------------------------------------------------------------
[2017-12-03 18:27:56] : Start mining: MONERO
[2017-12-03 18:27:56] : Starting NVIDIA GPU thread 0, no affinity.
[2017-12-03 18:27:56] : Fast-connecting to cryptonight.eu.nicehash.com:3355 pool ...
[2017-12-03 18:27:56] : Fast-connecting to cryptonight.usa.nicehash.com:3355 pool ...
[2017-12-03 18:27:56] : Pool cryptonight.eu.nicehash.com:3355 connected. Logging in...
[2017-12-03 18:27:56] : Pool cryptonight.usa.nicehash.com:3355 connected. Logging in...
[2017-12-03 18:27:56] : Difficulty changed. Now: 100001.
[2017-12-03 18:27:56] : Pool logged in.
[2017-12-03 18:27:57] : Difficulty changed. Now: 100001.
[2017-12-03 18:27:57] : New block detected.
[2017-12-03 18:28:01] : New block detected.
.
.
.
# nvidia-smi (2 instances concurrently, 1 for each GPU, working fine)
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.98 Driver Version: 384.98 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla C2075 Off | 00000000:01:00.0 Off | 0 |
| 45% 76C P0 135W / N/A | 2074MiB / 5301MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 1 Quadro K620 Off | 00000000:02:00.0 Off | N/A |
| 55% 65C P0 13W / 30W | 575MiB / 2000MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 9403 C ./xmr-stak 2067MiB |
| 1 9620 C ./xmr-stak 562MiB |
+-----------------------------------------------------------------------------+
it looks like it has somethink to do that a fermi and a new gpu is mixed but I do not know why this error is triggered
NdranC
commented
6 years ago Thanks to @surfinoma post I decided to try harder and succeeded to make both cards work under separate instances at the same time. The second instance seems to fail if I launch the first instance with CPU mining. If I just instead mine with one of the cards without CPU I can then launch the second card+CPU and everything seems to work fine.
It's definitely cumbersome but at least possible.
surfinoma
commented
6 years ago @psychocrypt
Indeed there seems to be a problem mixing cards from different architectures/generations.
I also had an idea that this could be the problem ...
To clarify this for a quick test I swapped the Quadro K620 against a Quadro 4000.
So the new combo was Tesla C2075/Quadro 4000 (both Fermi cards, compute 20):
# cat nvidia.txt
"gpu_threads_conf" :
[
// gpu: Tesla C2075 architecture: 20
// memory: 5205/5301 MiB
// smx: 14
{ "index" : 0,
"threads" : 24, "blocks" : 42,
"bfactor" : 8, "bsleep" : 25,
"affine_to_cpu" : false,
},
// gpu: Quadro 4000 architecture: 20
// memory: 1912/1984 MiB
// smx: 8
{ "index" : 1,
"threads" : 32, "blocks" : 16,
"bfactor" : 8, "bsleep" : 25,
"affine_to_cpu" : false,
},
],
# ./xmr-stak --noCPU
-------------------------------------------------------------------
xmr-stak 2.0.0 2920e9a
Brought to you by fireice_uk and psychocrypt under GPLv3.
Based on CPU mining code by wolf9466 (heavily optimized by fireice_uk).
Based on NVIDIA mining code by KlausT and psychocrypt.
Configurable dev donation level is set to 2.0%
You can use following keys to display reports:
'h' - hashrate
'r' - results
'c' - connection
-------------------------------------------------------------------
[2017-12-03 21:44:19] : Start mining: MONERO
[2017-12-03 21:44:19] : Starting NVIDIA GPU thread 0, no affinity.
[2017-12-03 21:44:19] : Starting NVIDIA GPU thread 1, no affinity.
[2017-12-03 21:44:19] : Fast-connecting to cryptonight.eu.nicehash.com:3355 pool ...
[2017-12-03 21:44:19] : Fast-connecting to cryptonight.usa.nicehash.com:3355 pool ...
[2017-12-03 21:44:19] : Pool cryptonight.eu.nicehash.com:3355 connected. Logging in...
[2017-12-03 21:44:19] : Pool cryptonight.usa.nicehash.com:3355 connected. Logging in...
[2017-12-03 21:44:20] : Difficulty changed. Now: 100001.
[2017-12-03 21:44:20] : Pool logged in.
HASHRATE REPORT - NVIDIA
| ID | 10s | 60s | 15m | ID | 10s | 60s | 15m |
| 0 | 311.6 | (na) | (na) | 1 | 180.2 | (na) | (na) |
-----------------------------------------------------
Totals: 491.8 (na) (na) H/s
Highest: 491.8 H/s
.
.
^C
Sun Dec 3 21:55:14 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.98 Driver Version: 384.98 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla C2075 Off | 00000000:01:00.0 Off | 0 |
| 50% 52C P0 122W / N/A | 2113MiB / 5301MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 1 Quadro 4000 Off | 00000000:02:00.0 Off | N/A |
| 36% 57C P0 N/A / N/A | 1099MiB / 1984MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 3511 C ./xmr-stak 2067MiB |
| 1 3511 C ./xmr-stak 1057MiB |
+-----------------------------------------------------------------------------+So this is working as expected.
If i remember correctly, I tested the first card combination (Fermi/Kepler) once at work using xmrig, which worked also (not sure) ,,,
psychocrypt
commented
6 years ago Different architectures are no problem I tested it. But it looks like fermi
surfinoma
commented
6 years ago Seems NVIDIA wants to do us a favour in getting rid of all that old crap, forcing us to buy their nifty new toys ;-)
By the way: Thanks for all your efforts!
P.S. I'll have another look at the Fermi/Kepler combo at work using "the other miner" and look if this works or not. I'll report on this, stay tuned ...
surfinoma
commented
6 years ago I've checked this issue at work using xmrig. My assumption that this worked in the past was wrong.
It behaves exactly the same: A combination of Fermi/Non-Fermi leads to the well known error.
This seems to be a CUDA inherent issue, that has nothing to do with xmr-stak, so this won't be fixed :-(
Would be useful to mention this behaviour in the official xmr-stak documentation.
I think this can be closed ..
Regards
fireice-uk
commented
6 years ago @surfinoma Fermi is being dropped from cuda 9 - we went back to cuda 8 temporarily with 2.1
ColonelRyzen
commented
6 years ago Try running two separate xmr-stak process with those gpus. I am using a Quadro 2000 (2.1) and GT 730 (3.5) and together they produce the same CUDA error. I made a copy of the compiled xmr-stak directory. I run the quadro out of the original and the gt730 out of the copy. So far it's going pretty well.
Spudz76
commented
6 years ago Same thing here, annoying silly CUDA 8. Dual processes work OK but certainly not ideal (how is that any different, they are still both running?) I now have two rigs running two xmr-stak's I guess I could move the cards around so its not mixed... I think I might look into this issue. If xmr-stak adjusted to fire a separate entire CUDA session per card (per host thread) then it might work same as running two totally separate ones. It has to have something to do with trying to run mixed cards via one CUDA handle. And everyone on the planet has the same problem when mixing 20's with anything newer.
rpbigpapa
commented
6 years ago i mix a gtx 580 and a gtx 750 in windows 10 x64 cuda 8 using the pre-compiled xmr-stacks and it works fine but in ubuntu 18.04 i compile with cmake -DCUDA_HOST_COMPILER=/usr/bin/gcc-5 .. then make install and no errors but when i try to run i get [2018-05-21 08:55:33] : Mining coin: turtlecoin [2018-05-21 08:55:33] : NVIDIA: GPU configuration stored in file 'nvidia.txt' [2018-05-21 08:55:33] : Starting NVIDIA GPU thread 0, no affinity. [2018-05-21 08:55:33] : Starting NVIDIA GPU thread 1, no affinity. [CUDA] Error gpu 1: </home/big/trtl-stak/xmrstak/backend/nvidia/nvcc_code/cuda_extra.cu>:280 terminate called after throwing an instance of 'std::runtime_error' what(): [CUDA] Error: all CUDA-capable devices are busy or unavailable Aborted and i know its trtl -stak but get the same results with a fresh git clone of xmr-stak downloaded and compiled 5-21-18 trtl-stak same date so on windows 10 the pre-compiled of both xmr-stak and trtl-stak work fine with mixed fermi gtx 580 and the gtx 750 and been getting great hashes on windows i also have 1 comp that has a gtx 480 fermi running on ubuntu 16 with a phenom II x4 955 i know the 955 dont give great hashes but im getting 1040 H/s on cryptonight light v7 with 2 cores of the 955 and the gtx 480 so fireice-uk you rock for me you guys smoke the other miner software in speed and ease of use but hoping you can help me because my windows h/s with gtx 580+gtx750+5 cores of fx6300@3.6Mgz i get 2230 H/s in CNL V7 but my other comp with the gtx 480 i was only getting 960h/s on win 10 x64 but get 1030H/s on same computer with ubuntu so was trying to switch my other gtx 580+gtx750 comp to linux to see if i get a boost like the other comp did but have run into this problem with the error above if you need any logs or any other info please let me know thanks you guys are great and i ive my part in dev fee in all your miners YOU ROCK GUYS
psychocrypt
commented
6 years ago The cuda driver has problems to mix fermi gpus(sm_20 and sm_21) with othwr newer gpus. In that case you must run two processes.
2 GPUs (gtx760 and gtx460) Arch Linux
I compiled xmr-stak with CUDA 8 and gcc-5 since I needed the ability to use compute20/21 for my gtx 460. I just exported CC and CXX, make .., make install.
I configured my nvidia.txt this way
when I run it I get this problem
I know this settings work because I can comment out one card or the other and they work just fine alone but they fail when I run them together.
This is the output of nvidia-smi
Any ideas of what could be causing this? Thanks