search

rapidsai / cudf

cuDF - GPU DataFrame Library
https://docs.rapids.ai/api/cudf/stable/
Apache License 2.0
8.3k stars 886 forks source link

[BUG]CUDA_ERROR_NO_DEVICE #14343

Closed Zakk-Yang closed 8 months ago

Zakk-Yang commented 11 months ago

Setup:

Installed RAPIDS in a WSL2 environment. Error when importing cudf:

CudaSupportError: Error at driver init: 
Call to cuInit results in CUDA_ERROR_NO_DEVICE (100):

Full error code:

---------------------------------------------------------------------------
CudaAPIError                              Traceback (most recent call last)
File [~/miniconda3/envs/rapids-23.12/lib/python3.10/site-packages/numba/cuda/cudadrv/driver.py:258](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/learn-rapids/~/miniconda3/envs/rapids-23.12/lib/python3.10/site-packages/numba/cuda/cudadrv/driver.py:258), in Driver.ensure_initialized(self)
    257     _logger.info('init')
--> 258     self.cuInit(0)
    259 except CudaAPIError as e:

File [~/miniconda3/envs/rapids-23.12/lib/python3.10/site-packages/numba/cuda/cudadrv/driver.py:331](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/learn-rapids/~/miniconda3/envs/rapids-23.12/lib/python3.10/site-packages/numba/cuda/cudadrv/driver.py:331), in Driver._ctypes_wrap_fn.<locals>.safe_cuda_api_call(*args)
    330 retcode = libfn(*args)
--> 331 self._check_ctypes_error(fname, retcode)

File [~/miniconda3/envs/rapids-23.12/lib/python3.10/site-packages/numba/cuda/cudadrv/driver.py:399](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/mnt/d/learn-rapids/~/miniconda3/envs/rapids-23.12/lib/python3.10/site-packages/numba/cuda/cudadrv/driver.py:399), in Driver._check_ctypes_error(self, fname, retcode)
    398     self._detect_fork()
--> 399 raise CudaAPIError(retcode, msg)

CudaAPIError: [100] Call to cuInit results in CUDA_ERROR_NO_DEVICE

During handling of the above exception, another exception occurred:

CudaSupportError                          Traceback (most recent call last)
/mnt/d/learn-rapids/Untitled.ipynb Cell 1 line 2
     [22](vscode-notebook-cell://wsl%2Bubuntu/mnt/d/learn-rapids/Untitled.ipynb#W0sdnNjb2RlLXJlbW90ZQ%3D%3D?line=21) from numba import cuda
     [24](vscode-notebook-cell://wsl%2Bubuntu/mnt/d/learn-rapids/Untitled.ipynb#W0sdnNjb2RlLXJlbW90ZQ%3D%3D?line=23) print("Allocating array")
---> [26](vscode-notebook-cell://wsl%2Bubuntu/mnt/d/learn-rapids/Untitled.ipynb#W0sdnNjb2RlLXJlbW90ZQ%3D%3D?line=25) cuda.device_array(1)
     [28](vscode-notebook-cell://wsl%2Bubuntu/mnt/d/learn-rapids/Untitled.ipynb#W0sdnNjb2RlLXJlbW90ZQ%3D%3D?line=27) print("Finished")
...
--> 262     raise CudaSupportError(f"Error at driver init: {description}")
    263 else:
    264     self.pid = _getpid()

CudaSupportError: Error at driver init: Call to cuInit results in CUDA_ERROR_NO_DEVICE (100)
Output is truncated. View as a [scrollable element](command:cellOutput.enableScrolling?4ab5c997-fe7c-4103-adb6-5a271daa80f1) or open in a [text editor](command:workbench.action.openLargeOutput?4ab5c997-fe7c-4103-adb6-5a271daa80f1). Adjust cell output [settings](command:workbench.action.openSettings?%5B%22%40tag%3AnotebookOutputLayout%22%5D)...

nvidia-smi has the following info in wsl:

(rapids-23.12) zy-wsl@yjl-dl:/mnt/c/Users/zakky$ nvidia-smi
Sat Oct 28 17:36:40 2023       nt/c/Users/zakky$
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.05              Driver Version: 545.84       CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA RTX A6000               On  | 00000000:01:00.0  On |                  Off |
| 30%   48C    P5              40W / 300W |   2003MiB / 49140MiB |     40%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A        23      G   /Xwayland                                 N/A      |
|    0   N/A  N/A      7498      C   /python3.10                               N/A      |
+---------------------------------------------------------------------------------------+

Installation command used:

conda create --solver=libmamba -n rapids-23.12 -c rapidsai-nightly -c conda-forge -c nvidia  \
    cudf=23.12 cuml=23.12 python=3.10 cuda-version=12.0 \
    jupyterlab

Command numba -s in WSL environment with the following info:

(rapids-23.12) zy-wsl@yjl-dl:/mnt/c/Users/zakky$ numba -s
System info:
/home/zy-wsl/miniconda3/envs/rapids-23.12/lib/python3.10/site-packages/numba/np/ufunc/parallel.py:371: NumbaWarning: The TBB threading layer requires TBB version 2021 update 6 or later i.e., TBB_INTERFACE_VERSION >= 12060. Found TBB_INTERFACE_VERSION = 12050. The TBB threading layer is disabled.
  warnings.warn(problem)
--------------------------------------------------------------------------------
__Time Stamp__
Report started (local time)                   : 2023-10-28 17:21:28.731152
UTC start time                                : 2023-10-28 16:21:28.731155
Running time (s)                              : 1.117497

__Hardware Information__
Machine                                       : x86_64
CPU Name                                      : alderlake
CPU Count                                     : 24
Number of accessible CPUs                     : 24
List of accessible CPUs cores                 : 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
CFS Restrictions (CPUs worth of runtime)      : None

CPU Features                                  : 64bit adx aes avx avx2 avxvnni bmi
                                                bmi2 clflushopt clwb cmov crc32
                                                cx16 cx8 f16c fma fsgsbase fxsr
                                                gfni invpcid lzcnt mmx movbe
                                                movdir64b movdiri pclmul popcnt
                                                prfchw rdpid rdrnd rdseed sahf
                                                serialize sha shstk sse sse2 sse3
                                                sse4.1 sse4.2 ssse3 vaes
                                                vpclmulqdq waitpkg xsave xsavec
                                                xsaveopt xsaves

Memory Total (MB)                             : 31943
Memory Available (MB)                         : 29762

__OS Information__
Platform Name                                 : Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Platform Release                              : 5.15.90.1-microsoft-standard-WSL2
OS Name                                       : Linux
OS Version                                    : #1 SMP Fri Jan 27 02:56:13 UTC 2023
OS Specific Version                           : ?
Libc Version                                  : glibc 2.35

__Python Information__
Python Compiler                               : GCC 12.3.0
Python Implementation                         : CPython
Python Version                                : 3.10.13
Python Locale                                 : en_US.UTF-8

__Numba Toolchain Versions__
Numba Version                                 : 0.57.1
llvmlite Version                              : 0.40.1

__LLVM Information__
LLVM Version                                  : 14.0.6

__CUDA Information__
CUDA Device Initialized                       : False
CUDA Driver Version                           : ?
CUDA Runtime Version                          : ?
CUDA NVIDIA Bindings Available                : ?
CUDA NVIDIA Bindings In Use                   : ?
CUDA Minor Version Compatibility Available    : ?
CUDA Minor Version Compatibility Needed       : ?
CUDA Minor Version Compatibility In Use       : ?
CUDA Detect Output:
None
CUDA Libraries Test Output:
None

__NumPy Information__
NumPy Version                                 : 1.24.4
NumPy Supported SIMD features                 : ('MMX', 'SSE', 'SSE2', 'SSE3', 'SSSE3', 'SSE41', 'POPCNT', 'SSE42', 'AVX', 'F16C', 'FMA3', 'AVX2')
NumPy Supported SIMD dispatch                 : ('SSSE3', 'SSE41', 'POPCNT', 'SSE42', 'AVX', 'F16C', 'FMA3', 'AVX2', 'AVX512F', 'AVX512CD', 'AVX512_SKX', 'AVX512_CLX', 'AVX512_CNL', 'AVX512_ICL')
NumPy Supported SIMD baseline                 : ('SSE', 'SSE2', 'SSE3')
NumPy AVX512_SKX support detected             : False

__SVML Information__
SVML State, config.USING_SVML                 : False
SVML Library Loaded                           : False
llvmlite Using SVML Patched LLVM              : False
SVML Operational                              : False

__Threading Layer Information__
TBB Threading Layer Available                 : False
+--> Disabled due to Unknown import problem.
OpenMP Threading Layer Available              : True
+-->Vendor: GNU
Workqueue Threading Layer Available           : True
+-->Workqueue imported successfully.

__Numba Environment Variable Information__
None found.

__Conda Information__
Conda Build                                   : not installed
Conda Env                                     : 23.5.2
Conda Platform                                : linux-64
Conda Python Version                          : 3.11.4.final.0
Conda Root Writable                           : True

__Installed Packages__
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
anyio                     4.0.0              pyhd8ed1ab_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py310h2372a71_4    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_0    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
aws-c-auth                0.7.4                h1a24852_6    conda-forge
aws-c-cal                 0.6.7                h6e18cf3_0    conda-forge
aws-c-common              0.9.4                hd590300_0    conda-forge
aws-c-compression         0.2.17               h037bafe_4    conda-forge
aws-c-event-stream        0.3.2                he4fbe49_4    conda-forge
aws-c-http                0.7.13               hbbfb9a7_7    conda-forge
aws-c-io                  0.13.35              hd1885a1_4    conda-forge
aws-c-mqtt                0.9.8                h31a96f8_0    conda-forge
aws-c-s3                  0.3.19               hb128593_1    conda-forge
aws-c-sdkutils            0.1.12               h037bafe_3    conda-forge
aws-checksums             0.1.17               h037bafe_3    conda-forge
aws-crt-cpp               0.24.4               h53d10bb_0    conda-forge
aws-sdk-cpp               1.11.182             hb97d603_2    conda-forge
babel                     2.13.1             pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
bokeh                     3.3.0              pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py310hc6cd4ac_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.20.1               hd590300_1    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py310h2fee648_0    conda-forge
charset-normalizer        3.3.1              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.1           py310hd41b1e2_1    conda-forge
cuda-cccl_linux-64        12.0.90              ha770c72_1    conda-forge
cuda-cudart               12.0.107             h59595ed_6    conda-forge
cuda-cudart-dev           12.0.107             h59595ed_6    conda-forge
cuda-cudart-dev_linux-64  12.0.107             h59595ed_6    conda-forge
cuda-cudart-static        12.0.107             h59595ed_6    conda-forge
cuda-cudart-static_linux-64 12.0.107             h59595ed_6    conda-forge
cuda-cudart_linux-64      12.0.107             h59595ed_6    conda-forge
cuda-nvcc-dev_linux-64    12.0.76              ha770c72_1    conda-forge
cuda-nvcc-impl            12.0.76              h59595ed_1    conda-forge
cuda-nvcc-tools           12.0.76              h59595ed_1    conda-forge
cuda-nvrtc                12.0.76              h59595ed_1    conda-forge
cuda-nvtx                 12.0.76              hcb278e6_0    conda-forge
cuda-profiler-api         12.0.76              ha770c72_0    conda-forge
cuda-python               12.0.0          py310hcb41016_2    conda-forge
cuda-version              12.0                 hffde075_2    conda-forge
cudf                      23.12.00a       cuda12_py310_231028_g2a923dfff8_124    rapidsai-nightly
cuml                      23.12.00a       cuda12_py310_231028_gff635fc25_31    rapidsai-nightly
cupy                      12.2.0          py310he66c036_3    conda-forge
cytoolz                   0.12.2          py310h2372a71_1    conda-forge
dask                      2023.9.2           pyhd8ed1ab_0    conda-forge
dask-core                 2023.9.2           pyhd8ed1ab_0    conda-forge
dask-cuda                 23.12.00a24     py310_231028_gd9e1001_24    rapidsai-nightly
dask-cudf                 23.12.00a       cuda12_py310_231028_g2a923dfff8_124    rapidsai-nightly
debugpy                   1.8.0           py310hc6cd4ac_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
distributed               2023.9.2           pyhd8ed1ab_0    conda-forge
dlpack                    0.5                  h9c3ff4c_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
fastrlock                 0.8.2           py310hc6cd4ac_1    conda-forge
fmt                       9.1.0                h924138e_0    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fsspec                    2023.10.0          pyhca7485f_0    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
glog                      0.6.0                h6f12383_0    conda-forge
gmock                     1.14.0               ha770c72_1    conda-forge
gtest                     1.14.0               h00ab1b0_1    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
importlib_resources       6.1.0              pyhd8ed1ab_0    conda-forge
ipykernel                 6.26.0             pyhf8b6a83_0    conda-forge
ipython                   8.16.1             pyh0d859eb_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
json5                     0.9.14             pyhd8ed1ab_0    conda-forge
jsonpointer               2.4             py310hff52083_3    conda-forge
jsonschema                4.19.1             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.7.1           pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.19.1             pyhd8ed1ab_0    conda-forge
jupyter-lsp               2.2.0              pyhd8ed1ab_0    conda-forge
jupyter_client            8.5.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.4.0           py310hff52083_0    conda-forge
jupyter_events            0.8.0              pyhd8ed1ab_0    conda-forge
jupyter_server            2.9.1              pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
jupyterlab                4.0.7              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.25.0             pyhd8ed1ab_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 hb7c19ff_3    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20230802.1      cxx17_h59595ed_0    conda-forge
libarrow                  13.0.0          hecbb4c5_13_cpu    conda-forge
libblas                   3.9.0           19_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           19_linux64_openblas    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcublas                 12.0.1.189           hcb278e6_2    conda-forge
libcublas-dev             12.0.1.189           hcb278e6_2    conda-forge
libcudf                   23.12.00a       cuda12_231028_g2a923dfff8_124    rapidsai-nightly
libcufft                  11.0.0.21            hcb278e6_1    conda-forge
libcufile                 1.5.0.59             hcb278e6_0    conda-forge
libcufile-dev             1.5.0.59             hcb278e6_0    conda-forge
libcuml                   23.12.00a       cuda12_231028_gff635fc25_31    rapidsai-nightly
libcumlprims              23.12.00a       cuda12_231028_gc6cb52d_3    rapidsai-nightly
libcurand                 10.3.1.50            hcb278e6_0    conda-forge
libcurand-dev             10.3.1.50            hcb278e6_0    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libcusolver               11.4.2.57            hcb278e6_1    conda-forge
libcusolver-dev           11.4.2.57            hcb278e6_1    conda-forge
libcusparse               12.0.0.76            hcb278e6_1    conda-forge
libcusparse-dev           12.0.0.76            hcb278e6_1    conda-forge
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.12               hf998b51_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libgoogle-cloud           2.12.0               h19a6dae_3    conda-forge
libgrpc                   1.58.1               he06187c_2    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
libkvikio                 23.12.00a       cuda12_231028_gd56d88c_17    rapidsai-nightly
liblapack                 3.9.0           19_linux64_openblas    conda-forge
libllvm14                 14.0.6               hcd5def8_4    conda-forge
libnghttp2                1.55.1               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libnuma                   2.0.16               h0b41bf4_1    conda-forge
libnvjitlink              12.0.76              hcb278e6_1    conda-forge
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libprotobuf               4.24.3               hf27288f_1    conda-forge
libraft                   23.12.00a       cuda12_231028_g9ad76fae_68    rapidsai-nightly
libraft-headers           23.12.00a       cuda12_231028_g9ad76fae_68    rapidsai-nightly
libraft-headers-only      23.12.00a       cuda12_231028_g9ad76fae_68    rapidsai-nightly
libre2-11                 2023.06.02           h7a70373_0    conda-forge
librmm                    23.12.00a       cuda12_231028_g5bdcc056_17    rapidsai-nightly
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.43.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libthrift                 0.19.0               hb90f79a_1    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libutf8proc               2.8.0                h166bdaf_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
llvmlite                  0.40.1          py310h1b8f574_0    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
lz4                       4.3.2           py310h350c4a5_1    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
markupsafe                2.1.3           py310h2372a71_1    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mistune                   3.0.1              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.6           py310hd41b1e2_0    conda-forge
nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
nbconvert-core            7.9.2              pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
nccl                      2.19.3.1             h3a97aeb_0    conda-forge
ncurses                   6.4                  hcb278e6_0    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.3              pyhd8ed1ab_0    conda-forge
numba                     0.57.1          py310h0f6aa51_0    conda-forge
numpy                     1.24.4          py310ha4c1d20_0    conda-forge
nvcomp                    2.6.1                h10b603f_3    conda-forge
nvtx                      0.2.8           py310h2372a71_1    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
orc                       1.9.0                h208142c_3    conda-forge
overrides                 7.4.0              pyhd8ed1ab_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    1.5.3           py310h9b08913_1    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.4.1              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0          py310h01dd4db_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
prometheus_client         0.17.1             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
protobuf                  4.24.3          py310h620c231_1    conda-forge
psutil                    5.9.5           py310h2372a71_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   13.0.0          py310hf9e7431_13_cpu    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pylibraft                 23.12.00a       cuda12_py310_231028_g9ad76fae_68    rapidsai-nightly
pynvml                    11.4.1             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.10.13         hd12c33a_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.18.1             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python_abi                3.10                    4_cp310    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py310h2372a71_1    conda-forge
pyzmq                     25.1.1          py310h795f18f_2    conda-forge
raft-dask                 23.12.00a       cuda12_py310_231028_g9ad76fae_68    rapidsai-nightly
rdma-core                 28.9                 h59595ed_1    conda-forge
re2                       2023.06.02           h2873b5e_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.30.2             pyhd8ed1ab_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rmm                       23.12.00a       cuda12_py310_231028_g5bdcc056_17    rapidsai-nightly
rpds-py                   0.10.6          py310hcb5633a_0    conda-forge
s2n                       1.3.55               h06160fa_0    conda-forge
scipy                     1.11.3          py310hb13e2d6_1    conda-forge
send2trash                1.8.2              pyh41d4057_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
spdlog                    1.11.0               h9b3ece8_1    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tblib                     2.0.0              pyhd8ed1ab_0    conda-forge
terminado                 0.17.1             pyh41d4057_0    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h2797004_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py310h2372a71_1    conda-forge
traitlets                 5.12.0             pyhd8ed1ab_0    conda-forge
treelite                  3.9.1           py310h4a6579d_0    conda-forge
treelite-runtime          3.9.1                    pypi_0    pypi
types-python-dateutil     2.8.19.14          pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
ucx                       1.15.0               h195a15c_0    conda-forge
ucx-proc                  1.0.0                       gpu    rapidsai-nightly
ucx-py                    0.35.00a        py310_231028_gc34aad0_10    rapidsai-nightly
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
urllib3                   2.0.7              pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.8              pyhd8ed1ab_0    conda-forge
webcolors                 1.13               pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.6.4              pyhd8ed1ab_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xyzservices               2023.10.1          pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zict                      3.0.0              pyhd8ed1ab_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

No errors reported.

__Warning log__
Warning (cuda): CUDA device initialisation problem. Message:Error at driver init: Call to cuInit results in CUDA_ERROR_NO_DEVICE (100)
Exception class: <class 'numba.cuda.cudadrv.error.CudaSupportError'>
Warning (no file): /sys/fs/cgroup/cpuacct/cpu.cfs_quota_us
Warning (no file): /sys/fs/cgroup/cpuacct/cpu.cfs_period_us
--------------------------------------------------------------------------------
If requested, please copy and paste the information between
the dashed (----) lines, or from a given specific section as
appropriate.

=============================================================
IMPORTANT: Please ensure that you are happy with sharing the
contents of the information present, any information that you
wish to keep private you should remove before sharing.

Objective: Seeking help in resolving the CUDA initialization error when trying to import cudf in the WSL2 environment.

bdice commented 11 months ago

Thanks for the detail in your report above. Most of your environment setup seems right. Did you install any CUDA packages into the WSL environment, outside of conda? Your WSL instance should not have a CUDA driver installed, or else it will cause problems. Only the Windows host system should provide the driver. Maybe post the output of apt list?

NTNguyen13 commented 10 months ago

Hi, I have the same problem (trying to use cudf on WSL2), however my numba -s output is little bit different:

System info:
/home/nguyen/anaconda3/envs/rapids-23.10/lib/python3.10/site-packages/numba/np/ufunc/parallel.py:371: NumbaWarning: The TBB threading layer requires TBB version 2021 update 6 or later i.e., TBB_INTERFACE_VERSION >= 12060. Found TBB_INTERFACE_VERSION = 12050. The TBB threading layer is disabled.
  warnings.warn(problem)
--------------------------------------------------------------------------------
__Time Stamp__
Report started (local time)                   : 2023-11-15 00:51:07.117163
UTC start time                                : 2023-11-14 17:51:07.117166
Running time (s)                              : 4.804779

__Hardware Information__
Machine                                       : x86_64
CPU Name                                      : skylake
CPU Count                                     : 16
Number of accessible CPUs                     : 16
List of accessible CPUs cores                 : 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
CFS Restrictions (CPUs worth of runtime)      : None

CPU Features                                  : 64bit adx aes avx avx2 bmi bmi2
                                                clflushopt cmov crc32 cx16 cx8
                                                f16c fma fsgsbase fxsr invpcid
                                                lzcnt mmx movbe pclmul popcnt
                                                prfchw rdrnd rdseed sahf sse sse2
                                                sse3 sse4.1 sse4.2 ssse3 xsave
                                                xsavec xsaveopt xsaves

Memory Total (MB)                             : 32066
Memory Available (MB)                         : 26497

__OS Information__
Platform Name                                 : Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.35
Platform Release                              : 5.10.16.3-microsoft-standard-WSL2
OS Name                                       : Linux
OS Version                                    : #1 SMP Fri Apr 2 22:23:49 UTC 2021
OS Specific Version                           : ?
Libc Version                                  : glibc 2.35

__Python Information__
Python Compiler                               : GCC 12.3.0
Python Implementation                         : CPython
Python Version                                : 3.10.13
Python Locale                                 : en_US.UTF-8

__Numba Toolchain Versions__
Numba Version                                 : 0.57.1
llvmlite Version                              : 0.40.1

__LLVM Information__
LLVM Version                                  : 14.0.6

__CUDA Information__
CUDA Device Initialized                       : True
CUDA Driver Version                           : 12.2
CUDA Runtime Version                          : 11.8
CUDA NVIDIA Bindings Available                : True
CUDA NVIDIA Bindings In Use                   : False
CUDA Minor Version Compatibility Available    : True
CUDA Minor Version Compatibility Needed       : False
CUDA Minor Version Compatibility In Use       : False
CUDA Detect Output:
Found 1 CUDA devices
id 0    b'NVIDIA GeForce RTX 2080 Ti'                              [SUPPORTED]
                      Compute Capability: 7.5
                           PCI Device ID: 0
                              PCI Bus ID: 1
                                    UUID: GPU-01bc9f8e-d066-a5b1-bb43-007a711170ac
                                Watchdog: Enabled
             FP32/FP64 Performance Ratio: 32
Summary:
        1/1 devices are supported

CUDA Libraries Test Output:
Finding driver from candidates: /usr/lib/wsl/lib/libcuda.so.1...
Using loader <class 'ctypes.CDLL'>
        trying to load driver...        ok, loaded from /usr/lib/wsl/lib/libcuda.so.1
Finding nvvm from Conda environment
        named  libnvvm.so.4.0.0
        trying to open library...       ok
Finding cudart from Conda environment
        named  libcudart.so.11.8.89
        trying to open library...       ok
Finding cudadevrt from Conda environment
        named  libcudadevrt.a
Finding libdevice from Conda environment
        trying to open library...       ok

__NumPy Information__
NumPy Version                                 : 1.24.4
NumPy Supported SIMD features                 : ('MMX', 'SSE', 'SSE2', 'SSE3', 'SSSE3', 'SSE41', 'POPCNT', 'SSE42', 'AVX', 'F16C', 'FMA3', 'AVX2')
NumPy Supported SIMD dispatch                 : ('SSSE3', 'SSE41', 'POPCNT', 'SSE42', 'AVX', 'F16C', 'FMA3', 'AVX2', 'AVX512F', 'AVX512CD', 'AVX512_SKX', 'AVX512_CLX', 'AVX512_CNL', 'AVX512_ICL')
NumPy Supported SIMD baseline                 : ('SSE', 'SSE2', 'SSE3')
NumPy AVX512_SKX support detected             : False

__SVML Information__
SVML State, config.USING_SVML                 : False
SVML Library Loaded                           : False
llvmlite Using SVML Patched LLVM              : False
SVML Operational                              : False

__Threading Layer Information__
TBB Threading Layer Available                 : False
+--> Disabled due to Unknown import problem.
OpenMP Threading Layer Available              : True
+-->Vendor: GNU
Workqueue Threading Layer Available           : True
+-->Workqueue imported successfully.

__Numba Environment Variable Information__
NUMBA_CUDA_DRIVER                             : /usr/lib/wsl/lib/libcuda.so.1

__Conda Information__
Conda Build                                   : 3.27.0
Conda Env                                     : 23.10.0
Conda Platform                                : linux-64
Conda Python Version                          : 3.9.18.final.0
Conda Root Writable                           : True

__Installed Packages__
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
aiohttp                   3.8.6           py310h2372a71_1    conda-forge
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
alsa-lib                  1.2.10               hd590300_0    conda-forge
anyio                     4.0.0              pyhd8ed1ab_0    conda-forge
aom                       3.7.0                h59595ed_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py310h2372a71_4    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asttokens                 2.0.5              pyhd3eb1b0_0  
async-lru                 2.0.4           py310h06a4308_0  
async-timeout             4.0.3              pyhd8ed1ab_0    conda-forge
attr                      2.5.1                h166bdaf_1    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
aws-c-auth                0.7.4                hc8144f4_1    conda-forge
aws-c-cal                 0.6.2                h09139f6_2    conda-forge
aws-c-common              0.9.3                hd590300_0    conda-forge
aws-c-compression         0.2.17               h184a658_3    conda-forge
aws-c-event-stream        0.3.2                hd6ebb48_1    conda-forge
aws-c-http                0.7.13               hc690213_1    conda-forge
aws-c-io                  0.13.32              h161b759_6    conda-forge
aws-c-mqtt                0.9.6                h32970c0_2    conda-forge
aws-c-s3                  0.3.17               hb5e3142_3    conda-forge
aws-c-sdkutils            0.1.12               h184a658_2    conda-forge
aws-checksums             0.1.17               h184a658_2    conda-forge
aws-crt-cpp               0.23.1               h94c364a_5    conda-forge
aws-sdk-cpp               1.11.156             h6600424_3    conda-forge
babel                     2.11.0          py310h06a4308_0  
backcall                  0.2.0              pyhd3eb1b0_0  
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
blosc                     1.21.5               h0f2a231_0    conda-forge
bokeh                     3.3.1              pyhd8ed1ab_0    conda-forge
branca                    0.7.0              pyhd8ed1ab_1    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
brotli-python             1.1.0           py310hc6cd4ac_1    conda-forge
brunsli                   0.1                  h9c3ff4c_0    conda-forge
bzip2                     1.0.8                hd590300_5    conda-forge
c-ares                    1.21.0               hd590300_0    conda-forge
c-blosc2                  2.11.2               hb4ffafa_0    conda-forge
ca-certificates           2023.08.22           h06a4308_0  
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
cairo                     1.18.0               h3faef2a_0    conda-forge
certifi                   2023.7.22       py310h06a4308_0  
cffi                      1.16.0          py310h2fee648_0    conda-forge
cfitsio                   4.3.0                hbdc6101_0    conda-forge
charls                    2.4.2                h59595ed_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
colorcet                  3.0.1              pyhd8ed1ab_0    conda-forge
comm                      0.1.2           py310h06a4308_0  
contourpy                 1.2.0           py310hd41b1e2_0    conda-forge
cubinlinker               0.3.0           py310hfdf336d_0    rapidsai
cucim                     23.10.00        cuda11_py310_231011_g8dd1727_0    rapidsai
cuda-profiler-api         11.8.86                       0    nvidia
cuda-python               11.8.3          py310h70a93da_0    conda-forge
cuda-version              11.8                 h70ddcb2_2    conda-forge
cudatoolkit               11.8.0              h4ba93d1_12    conda-forge
cudf                      23.10.01        cuda11_py310_231107_g1a0c07670f_0    rapidsai
cudf_kafka                23.10.01        cuda11_py310_231107_g1a0c07670f_0    rapidsai
cugraph                   23.10.00        cuda11_py310_231011_gc3dc860d_0    rapidsai
cuml                      23.10.00        cuda11_py310_231011_g623996072_0    rapidsai
cuproj                    23.10.00        cuda11_py310_231011_g29cddcdc_0    rapidsai
cupy                      12.2.0          py310h7b03888_3    conda-forge
cuspatial                 23.10.00        cuda11_py310_231011_g29cddcdc_0    rapidsai
custreamz                 23.10.01        cuda11_py310_231107_g1a0c07670f_0    rapidsai
cuxfilter                 23.10.00        cuda11_py310_231011_g6600aff_0    rapidsai
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
cyrus-sasl                2.1.27               h54b06d7_7    conda-forge
cytoolz                   0.12.2          py310h2372a71_1    conda-forge
dask                      2023.9.2           pyhd8ed1ab_0    conda-forge
dask-core                 2023.9.2           pyhd8ed1ab_0    conda-forge
dask-cuda                 23.10.00        py310_231011_gdc811d3_0    rapidsai
dask-cudf                 23.10.01        cuda11_py310_231107_g1a0c07670f_0    rapidsai
datashader                0.16.0             pyhd8ed1ab_0    conda-forge
dav1d                     1.2.1                hd590300_0    conda-forge
dbus                      1.13.18              hb2f20db_0  
debugpy                   1.6.7           py310h6a678d5_0  
decorator                 5.1.1              pyhd3eb1b0_0  
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
distributed               2023.9.2           pyhd8ed1ab_0    conda-forge
dlpack                    0.5                  h9c3ff4c_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 0.8.3              pyhd3eb1b0_0  
expat                     2.5.0                hcb278e6_1    conda-forge
fastrlock                 0.8.2           py310hc6cd4ac_1    conda-forge
fiona                     1.9.5           py310h7631d76_0    conda-forge
fmt                       9.1.0                h924138e_0    conda-forge
folium                    0.15.0             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.44.0          py310h2372a71_0    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
freexl                    2.0.0                h743c826_0    conda-forge
frozenlist                1.4.0           py310h2372a71_1    conda-forge
fsspec                    2023.10.0          pyhca7485f_0    conda-forge
gdal                      3.7.3           py310h5c4b078_3    conda-forge
gdk-pixbuf                2.42.10              h829c605_4    conda-forge
geopandas                 0.14.1             pyhd8ed1ab_0    conda-forge
geopandas-base            0.14.1             pyha770c72_0    conda-forge
geos                      3.12.0               h59595ed_0    conda-forge
geotiff                   1.7.1               hf074850_14    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
glib                      2.78.1               hfc55251_0    conda-forge
glib-tools                2.78.1               hfc55251_0    conda-forge
glog                      0.6.0                h6f12383_0    conda-forge
gmock                     1.14.0               ha770c72_1    conda-forge
gmp                       6.2.1                h295c915_3  
graphite2                 1.3.14               h295c915_1  
gst-plugins-base          1.22.7               h8e1006c_0    conda-forge
gstreamer                 1.22.7               h98fc4e7_0    conda-forge
gtest                     1.14.0               h00ab1b0_1    conda-forge
harfbuzz                  8.3.0                h3d44ed6_0    conda-forge
hdf4                      4.2.15               h2a13503_7    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
holoviews                 1.18.1             pyhd8ed1ab_0    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
imagecodecs               2023.9.18       py310h496a806_2    conda-forge
imageio                   2.31.5             pyh8c1a49c_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
importlib_resources       6.1.1              pyhd8ed1ab_0    conda-forge
ipykernel                 6.25.0          py310h2f386ee_0  
ipython                   8.15.0          py310h06a4308_0  
ipython_genutils          0.2.0              pyhd3eb1b0_1  
ipywidgets                8.0.4           py310h06a4308_0  
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
jbig                      2.1               h7f98852_2003    conda-forge
jedi                      0.18.1          py310h06a4308_1  
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
json-c                    0.17                 h7ab15ed_0    conda-forge
json5                     0.9.6              pyhd3eb1b0_0  
jsonpointer               2.4             py310hff52083_3    conda-forge
jsonschema                4.19.2             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.7.1           pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.19.2             pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0           py310h06a4308_8  
jupyter-lsp               2.2.0              pyhd8ed1ab_0    conda-forge
jupyter-server-proxy      4.1.0              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.0              pyhd8ed1ab_0    conda-forge
jupyter_console           6.6.3           py310h06a4308_0  
jupyter_core              5.5.0           py310hff52083_0    conda-forge
jupyter_events            0.9.0              pyhd8ed1ab_0    conda-forge
jupyter_server            2.10.0             pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
jupyterlab                4.0.8              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.25.1          py310h06a4308_0  
jupyterlab_widgets        3.0.9              pyhd8ed1ab_0    conda-forge
jxrlib                    1.1                  h7f98852_2    conda-forge
kealib                    1.5.2                hcd42e92_1    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py310hd41b1e2_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lame                      3.100                h7b6447c_0  
lazy_loader               0.3                pyhd8ed1ab_0    conda-forge
lcms2                     2.15                 hb7c19ff_3    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20230802.1      cxx17_h59595ed_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libarchive                3.7.2                h039dbb9_0    conda-forge
libarrow                  12.0.1          h1935d02_14_cpu    conda-forge
libavif16                 1.0.1                hed45d22_3    conda-forge
libblas                   3.9.0           19_linux64_openblas    conda-forge
libboost-headers          1.82.0               ha770c72_6    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcap                    2.69                 h0f662aa_0    conda-forge
libcblas                  3.9.0           19_linux64_openblas    conda-forge
libclang                  15.0.7          default_h7634d5b_3    conda-forge
libclang13                15.0.7          default_h9986a30_3    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcublas                 11.11.3.6                     0    nvidia
libcublas-dev             11.11.3.6                     0    nvidia
libcucim                  23.10.00        cuda11_231011_g8dd1727_0    rapidsai
libcudf                   23.10.01        cuda11_231107_g1a0c07670f_0    rapidsai
libcudf_kafka             23.10.01        cuda11_231107_g1a0c07670f_0    rapidsai
libcufft                  10.9.0.58                     0    nvidia
libcufile                 1.4.0.31                      0    nvidia
libcufile-dev             1.4.0.31                      0    nvidia
libcugraph                23.10.00        cuda11_231011_gc3dc860d_0    rapidsai
libcugraph_etl            23.10.00        cuda11_231011_gc3dc860d_0    rapidsai
libcugraphops             23.10.00        cuda11_231011_gca4b54ac_0    nvidia
libcuml                   23.10.00        cuda11_231011_g623996072_0    rapidsai
libcumlprims              23.10.00        cuda11_231011_ge818397_0    nvidia
libcups                   2.3.3                h4637d8d_4    conda-forge
libcurand                 10.3.0.86                     0    nvidia
libcurand-dev             10.3.0.86                     0    nvidia
libcurl                   8.4.0                hca28451_0    conda-forge
libcusolver               11.4.1.48                     0    nvidia
libcusolver-dev           11.4.1.48                     0    nvidia
libcusparse               11.7.5.86                     0    nvidia
libcusparse-dev           11.7.5.86                     0    nvidia
libcuspatial              23.10.00        cuda11_231011_g29cddcdc_0    rapidsai
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.12               hf998b51_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libflac                   1.4.3                h59595ed_0    conda-forge
libgcc-ng                 13.2.0               h807b86a_3    conda-forge
libgcrypt                 1.10.2               hd590300_0    conda-forge
libgdal                   3.7.3                h6f3d308_3    conda-forge
libgfortran-ng            13.2.0               h69a702a_3    conda-forge
libgfortran5              13.2.0               ha4646dd_3    conda-forge
libglib                   2.78.1               hebfc3b9_0    conda-forge
libgomp                   13.2.0               h807b86a_3    conda-forge
libgoogle-cloud           2.12.0               h8d7e28b_2    conda-forge
libgpg-error              1.47                 h71f35ed_0    conda-forge
libgrpc                   1.57.0               ha4d0f93_2    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
libkml                    1.3.0             h01aab08_1018    conda-forge
libkvikio                 23.10.00        cuda11_231011_g5ea0525_0    rapidsai
liblapack                 3.9.0           19_linux64_openblas    conda-forge
libllvm14                 14.0.6               hcd5def8_4    conda-forge
libllvm15                 15.0.7               h5cf9203_3    conda-forge
libnetcdf                 4.9.2           nompi_h80fb2b6_112    conda-forge
libnghttp2                1.58.0               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libntlm                   1.4               h7f98852_1002    conda-forge
libnuma                   2.0.16               h0b41bf4_1    conda-forge
libogg                    1.3.5                h27cfd23_1  
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libopus                   1.3.1                h7b6447c_0  
libpng                    1.6.39               h753d276_0    conda-forge
libpq                     16.1                 hfc447b1_0    conda-forge
libprotobuf               4.23.4               hf27288f_6    conda-forge
libraft                   23.10.00        cuda11_231011_gafdddfb3_0    rapidsai
libraft-headers           23.10.00        cuda11_231011_gafdddfb3_0    rapidsai
libraft-headers-only      23.10.00        cuda11_231011_gafdddfb3_0    rapidsai
librdkafka                1.9.2                ha5a0de0_2    conda-forge
librmm                    23.10.00        cuda11_231011_gf8ac6f8e_0    rapidsai
librttopo                 1.1.0               hb58d41b_14    conda-forge
libsndfile                1.2.2                hc60ed4a_1    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libspatialindex           1.9.3                h9c3ff4c_4    conda-forge
libspatialite             5.1.0                h090f1da_1    conda-forge
libsqlite                 3.44.0               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_3    conda-forge
libsystemd0               254                  h3516f8a_0    conda-forge
libthrift                 0.19.0               hb90f79a_1    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libutf8proc               2.8.0                h166bdaf_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libuv                     1.46.0               hd590300_0    conda-forge
libvorbis                 1.3.7                h7b6447c_0  
libwebp                   1.3.2                h658648e_1    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxgboost                1.7.6           rapidsai_h795055c_2310    rapidsai
libxkbcommon              1.6.0                h5d7e998_0    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzip                    1.10.1               h2629f0a_3    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
libzopfli                 1.0.3                h9c3ff4c_0    conda-forge
linkify-it-py             2.0.0              pyhd8ed1ab_0    conda-forge
llvmlite                  0.40.1          py310h1b8f574_0    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
lz4                       4.3.2           py310h350c4a5_1    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
mapclassify               2.6.1              pyhd8ed1ab_0    conda-forge
markdown                  3.5.1              pyhd8ed1ab_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3           py310h2372a71_1    conda-forge
matplotlib-base           3.8.1           py310h62c0568_0    conda-forge
matplotlib-inline         0.1.6           py310h06a4308_0  
mdit-py-plugins           0.4.0              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.0              pyhd8ed1ab_0    conda-forge
minizip                   4.0.2                h0ab5242_0    conda-forge
mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
mpg123                    1.32.3               h59595ed_0    conda-forge
msgpack-python            1.0.6           py310hd41b1e2_0    conda-forge
multidict                 6.0.4           py310h2372a71_1    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munch                     4.0.0              pyhd8ed1ab_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
mysql-common              8.0.33               hf1915f5_6    conda-forge
mysql-libs                8.0.33               hca2cd23_6    conda-forge
nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
nbconvert                 7.11.0             pyhd8ed1ab_0    conda-forge
nbconvert-core            7.11.0             pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.11.0             pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
nccl                      2.19.3.1             h6103f9b_0    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
nest-asyncio              1.5.6           py310h06a4308_0  
networkx                  3.2.1              pyhd8ed1ab_0    conda-forge
nodejs                    20.8.1               h1990674_0    conda-forge
notebook                  7.0.6              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.3           py310h06a4308_0  
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.94                 h1d7d5a4_0    conda-forge
numba                     0.57.1          py310h0f6aa51_0    conda-forge
numpy                     1.24.4          py310ha4c1d20_0    conda-forge
nvcomp                    2.6.1                h838ba91_3    conda-forge
nvtx                      0.2.8           py310h2372a71_1    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openslide                 3.4.1               hcb9e6c7_11    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
orc                       1.9.0                h52d3b3c_2    conda-forge
overrides                 7.4.0              pyhd8ed1ab_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    1.5.3           py310h9b08913_1    conda-forge
pandoc                    3.1.3                h32600fe_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
panel                     1.3.1              pyhd8ed1ab_0    conda-forge
param                     2.0.1              pyhca7485f_0    conda-forge
parso                     0.8.3              pyhd3eb1b0_0  
partd                     1.4.1              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pexpect                   4.8.0              pyhd3eb1b0_3  
pickleshare               0.7.5           pyhd3eb1b0_1003  
pillow                    10.1.0          py310h01dd4db_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pixman                    0.42.2               h59595ed_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.0.0              pyhd8ed1ab_0    conda-forge
ply                       3.11            py310h06a4308_0  
poppler                   23.10.0              h590f24d_0    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
postgresql                16.1                 h8972f4a_0    conda-forge
proj                      9.3.0                h1d62c97_2    conda-forge
prometheus_client         0.18.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.36          py310h06a4308_0  
prompt_toolkit            3.0.36               hd3eb1b0_0  
protobuf                  4.23.4          py310h620c231_3    conda-forge
psutil                    5.9.5           py310h2372a71_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptxcompiler               0.8.1           py310h70a93da_2    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pulseaudio-client         16.1                 hb77b528_5    conda-forge
pure_eval                 0.2.2              pyhd3eb1b0_0  
py-xgboost                1.7.6           rapidsai_py310h1395376_2310    rapidsai
pyarrow                   12.0.1          py310hf9e7431_14_cpu    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyct                      0.4.6                      py_0    conda-forge
pyct-core                 0.4.6                      py_0    conda-forge
pyee                      8.1.0              pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pylibcugraph              23.10.00        cuda11_py310_231011_gc3dc860d_0    rapidsai
pylibraft                 23.10.00        cuda11_py310_231011_gafdddfb3_0    rapidsai
pynvml                    11.4.1             pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pyppeteer                 1.0.2              pyhd8ed1ab_0    conda-forge
pyproj                    3.6.1           py310h32c33b7_4    conda-forge
pyqt                      5.15.10         py310h6a678d5_0  
pyqt5-sip                 12.13.0         py310h5eee18b_0  
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.10.13         hd12c33a_0_cpython    conda-forge
python-confluent-kafka    1.9.2           py310h5764c6d_2    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.18.1             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python_abi                3.10                    4_cp310    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyviz_comms               3.0.0              pyhd8ed1ab_0    conda-forge
pywavelets                1.4.1           py310h1f7b6fc_1    conda-forge
pyyaml                    6.0.1           py310h2372a71_1    conda-forge
pyzmq                     25.1.1          py310h795f18f_2    conda-forge
qt-main                   5.15.8              h82b777d_17    conda-forge
qtconsole                 5.4.2           py310h06a4308_0  
qtpy                      2.2.0           py310h06a4308_0  
raft-dask                 23.10.00        cuda11_py310_231011_gafdddfb3_0    rapidsai
rapids                    23.10.00        cuda11_py310_231011_gd3958fe_0    rapidsai
rapids-xgboost            23.10.00        cuda11_py310_231011_gd3958fe_0    rapidsai
rav1e                     0.6.6                he8a937b_2    conda-forge
rdma-core                 28.9                 h59595ed_1    conda-forge
re2                       2023.03.02           h8c504da_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.30.2             pyhd8ed1ab_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rich                      13.6.0             pyhd8ed1ab_0    conda-forge
rmm                       23.10.00        cuda11_py310_231011_gf8ac6f8e_0    rapidsai
rpds-py                   0.12.0          py310hcb5633a_0    conda-forge
rtree                     1.1.0           py310hbdcdc62_0    conda-forge
s2n                       1.3.54               h06160fa_0    conda-forge
scikit-image              0.21.0          py310hc6cd4ac_0    conda-forge
scikit-learn              1.3.2           py310h1fdf081_1    conda-forge
scipy                     1.11.3          py310hb13e2d6_1    conda-forge
send2trash                1.8.2              pyh41d4057_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
shapely                   2.0.2           py310h7dcad9a_0    conda-forge
simpervisor               1.0.0              pyhd8ed1ab_0    conda-forge
sip                       6.7.12          py310h6a678d5_0  
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
spdlog                    1.11.0               h9b3ece8_1    conda-forge
sqlite                    3.44.0               h2c6b66d_0    conda-forge
stack_data                0.2.0              pyhd3eb1b0_0  
streamz                   0.6.4              pyh6c4a22f_0    conda-forge
svt-av1                   1.7.0                h59595ed_0    conda-forge
tblib                     2.0.0              pyhd8ed1ab_0    conda-forge
terminado                 0.18.0             pyh0d859eb_0    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tifffile                  2023.9.26          pyhd8ed1ab_0    conda-forge
tiledb                    2.16.3               h8c794c1_3    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli                     2.0.1           py310h06a4308_0  
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py310h2372a71_1    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
traitlets                 5.13.0             pyhd8ed1ab_0    conda-forge
treelite                  3.9.1           py310h4a6579d_0    conda-forge
treelite-runtime          3.9.1                    pypi_0    pypi
types-python-dateutil     2.8.19.14          pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzcode                    2023c                h0b41bf4_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
uc-micro-py               1.0.1              pyhd8ed1ab_0    conda-forge
ucx                       1.14.1               h64cca9d_5    conda-forge
ucx-proc                  1.0.0                       gpu    rapidsai
ucx-py                    0.34.00         py310_231011_g17dceab_0    rapidsai
unicodedata2              15.1.0          py310h2372a71_0    conda-forge
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
uriparser                 0.9.7                hcb278e6_1    conda-forge
urllib3                   1.26.18            pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.5              pyhd3eb1b0_0  
webcolors                 1.13               pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.6.4              pyhd8ed1ab_0    conda-forge
websockets                10.4            py310h5764c6d_1    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.5           py310h06a4308_0  
xarray                    2023.10.1          pyhd8ed1ab_0    conda-forge
xcb-util                  0.4.0                hd590300_1    conda-forge
xcb-util-image            0.4.0                h8ee46fc_1    conda-forge
xcb-util-keysyms          0.4.0                h8ee46fc_1    conda-forge
xcb-util-renderutil       0.3.9                hd590300_1    conda-forge
xcb-util-wm               0.4.1                h8ee46fc_1    conda-forge
xerces-c                  3.2.4                hac6953d_3    conda-forge
xgboost                   1.7.6           rapidsai_py310h1395376_2310    rapidsai
xkeyboard-config          2.40                 hd590300_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xf86vidmodeproto     2.3.1             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xyzservices               2023.10.1          pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yarl                      1.9.2           py310h2372a71_1    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zfp                       1.0.0                h59595ed_4    conda-forge
zict                      3.0.0              pyhd8ed1ab_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zlib-ng                   2.0.7                h0b41bf4_0    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

No errors reported.

__Warning log__
Warning (no file): /sys/fs/cgroup/cpuacct/cpu.cfs_quota_us
Warning (no file): /sys/fs/cgroup/cpuacct/cpu.cfs_period_us
--------------------------------------------------------------------------------
If requested, please copy and paste the information between
the dashed (----) lines, or from a given specific section as
appropriate.

=============================================================
IMPORTANT: Please ensure that you are happy with sharing the
contents of the information present, any information that you
wish to keep private you should remove before sharing.
=============================================================

I followed this installation guide from NVIDIA: https://docs.nvidia.com/cuda/wsl-user-guide/index.html. Other applications using CUDA such as stable diffusion are working well.

Please comment on how I can provide more information if needed to troubleshoot this.

bdice commented 10 months ago

@NTNguyen13 Can you also post the result of apt list? Also take a look at this issue from Numba: https://github.com/numba/numba/issues/6777

NTNguyen13 commented 10 months ago

my apt list is quite long, could you suggest some grep keyword to filter it? I tried the command NUMBA_CUDA_LOG_LEVEL=DEBUG python -c "import cudf; cudf.Series([1,2,3])" the output is: 

== CUDA (ptxcompiler) [653] DEBUG -- CUDA Driver version 12.2
== CUDA (ptxcompiler) [653] DEBUG -- CUDA Runtime version 11.8
== CUDA [803] DEBUG -- call runtime api: cudaRuntimeGetVersion
== CUDA [2490]  INFO -- init
== CUDA [2491] DEBUG -- call driver api: cuInit
== CUDA [2491] DEBUG -- call driver api: cuCtxGetCurrent
== CUDA [2491] DEBUG -- call driver api: cuDeviceGetCount
== CUDA [2492] DEBUG -- call driver api: cuDeviceGet
== CUDA [2492] DEBUG -- call driver api: cuDeviceGetAttribute
== CUDA [2492] DEBUG -- call driver api: cuDeviceGetAttribute
== CUDA [2492] DEBUG -- call driver api: cuDeviceGetName
== CUDA [2492] DEBUG -- call driver api: cuDeviceGetUuid_v2
== CUDA [2492] DEBUG -- call driver api: cuDevicePrimaryCtxRetain
== CUDA [2963] DEBUG -- call driver api: cuCtxPushCurrent_v2
== CUDA [2963] DEBUG -- call driver api: cuMemGetInfo_v2

Edit: After adding the export to bashrc AND restart my VScode runtime, it works fine now

Zakk-Yang commented 10 months ago

Hi, Problem solved by following: under the wsl instance nano ~/.bashrc

export LD_LIBRARY_PATH="/usr/lib/wsl/lib/"
export NUMBA_CUDA_DRIVER="/usr/lib/wsl/lib/libcuda.so.1"
export LD_LIBRARY_PATH=/usr/local/cuda-12.3/targets/x86_64-linux/lib:$LD_LIBRARY_PATH
export PATH=/usr/local/cuda-12.3/bin:$PATH
shwina commented 10 months ago

@Zakk-Yang thanks for sharing your fix!

bdice commented 8 months ago

Closing as resolved.

quaid281 commented 1 week ago

I know this is closed, but I must say that installing CUDA and cuDNN is a nightmare on WSL, and the likelihood of this issue is that CUDF can not find your CUDA. If anyone is facing this issue, best of luck in resolving it.