search

rapidsai / cudf

cuDF - GPU DataFrame Library
https://docs.rapids.ai/api/cudf/stable/
Apache License 2.0
8.3k stars 886 forks source link

ModuleNotFoundError: No module named 'dask.dataframe.dispatch' #10036

Closed anamariaUIC closed 2 years ago

anamariaUIC commented 2 years ago

Hello,

I installed Rapids via:

conda clean --all

module load Python/3.7.4-GCCcore-8.3.0 module load CUDA/11.0.2-GCC-9.3.0

conda create -n rapids-21.12 -c rapidsai -c nvidia -c conda-forge rapids=21.12 python=3.7 cudatoolkit=11.0 dask-sql

I use Rapids via:

conda activate rapids-21.12

(rapids-21.12) anamaria@gpu-2-0.saber:~ $ python Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:53) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import dask_cudf Traceback (most recent call last): File "", line 1, in File "/home/anamaria/.conda/envs/rapids-21.12/lib/python3.7/site-packages/dask_cudf/init.py", line 6, in from . import backends File "/home/anamaria/.conda/envs/rapids-21.12/lib/python3.7/site-packages/dask_cudf/backends.py", line 11, in from dask.dataframe.dispatch import ( ModuleNotFoundError: No module named 'dask.dataframe.dispatch' import dask; print(dask.file) /home/anamaria/.local/lib/python3.7/site-packages/dask/init.py quit() (rapids-21.12) anamaria@gpu-2-0.saber:~ $ conda install dask distributed Collecting package metadata (current_repodata.json): done Solving environment: done

All requested packages already installed.

(rapids-21.12) anamaria@gpu-2-0.saber:~ $ python Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:53) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import dask_cudf Traceback (most recent call last): File "", line 1, in File "/home/anamaria/.conda/envs/rapids-21.12/lib/python3.7/site-packages/dask_cudf/init.py", line 6, in from . import backends File "/home/anamaria/.conda/envs/rapids-21.12/lib/python3.7/site-packages/dask_cudf/backends.py", line 11, in from dask.dataframe.dispatch import ( ModuleNotFoundError: No module named 'dask.dataframe.dispatch'

Please advise how to resolve this issue?

Thanks Ana

galipremsagar commented 2 years ago

@anamariaUIC Can you share the output of conda list?

I'm not able to get the errors you have mentioned:

pgali@dt07:/nvme/0/pgali/cudf$ python
Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:53) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cudf
cu>>> cudf.__version__
'21.12.02'
>>> import dask_cudf
>>> dask_cudf.__version__
'21.12.02'
anamariaUIC commented 2 years ago

@galipremsagar dask-cuda is there, please see bellow:

(base) anamaria@login-2[SABER]: ~ $ ssh gpu-2-0.saber Last login: Thu Jan 13 16:26:22 2022 from login-2.saber.ib

conda activate rapids-21.12 export PATH=/home/anamaria/.conda/envs/rapids-21.12/bin:$PATH(base) anamaria@gpu-2-0.saber:~ $ (base) anamaria@gpu-2-0.saber:~ $ conda activate rapids-21.12 (rapids-21.12) anamaria@gpu-2-0.saber:~ $ export PATH=/home/anamaria/.conda/envs/rapids-21.12/bin:$PATH (rapids-21.12) anamaria@gpu-2-0.saber:~ $ conda list

packages in environment at /home/anamaria/.conda/envs/rapids-21.12:

#

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge abseil-cpp 20210324.2 h9c3ff4c_0 conda-forge aiohttp 3.8.1 py37h5e8e339_0 conda-forge aiosignal 1.2.0 pyhd8ed1ab_0 conda-forge alsa-lib 1.2.3 h516909a_0 conda-forge anyio 3.4.0 py37h89c1867_0 conda-forge appdirs 1.4.4 pyh9f0ad1d_0 conda-forge argcomplete 2.0.0 pyhd8ed1ab_0 conda-forge argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge argon2-cffi-bindings 21.2.0 py37h5e8e339_1 conda-forge arrow-cpp 5.0.0 py37h6ab10f6_20_cuda conda-forge arrow-cpp-proc 3.0.0 cuda conda-forge asgiref 3.4.1 pyhd8ed1ab_0 conda-forge async-timeout 4.0.2 pyhd8ed1ab_0 conda-forge async_generator 1.10 py_0 conda-forge asynctest 0.13.0 py_0 conda-forge attrs 21.4.0 pyhd8ed1ab_0 conda-forge aws-c-auth 0.6.8 hadad3cd_1 conda-forge aws-c-cal 0.5.12 h70efedd_7 conda-forge aws-c-common 0.6.17 h7f98852_0 conda-forge aws-c-compression 0.2.14 h7c7754b_7 conda-forge aws-c-event-stream 0.2.7 hd2be095_32 conda-forge aws-c-http 0.6.10 h416565a_3 conda-forge aws-c-io 0.10.14 he836878_0 conda-forge aws-c-mqtt 0.7.10 h885097b_0 conda-forge aws-c-s3 0.1.29 h8d70ed6_0 conda-forge aws-c-sdkutils 0.1.1 h7c7754b_4 conda-forge aws-checksums 0.1.12 h7c7754b_6 conda-forge aws-crt-cpp 0.17.10 h6ab17b9_5 conda-forge aws-sdk-cpp 1.9.160 h36ff4c5_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 py_2 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge backports.zoneinfo 0.2.1 py37h5e8e339_4 conda-forge bleach 4.1.0 pyhd8ed1ab_0 conda-forge blosc 1.21.0 h9c3ff4c_0 conda-forge bokeh 2.4.0 py37h89c1867_0 conda-forge boost 1.74.0 py37h796e4cb_4 conda-forge boost-cpp 1.74.0 h312852a_4 conda-forge brotli 1.0.9 h7f98852_6 conda-forge brotli-bin 1.0.9 h7f98852_6 conda-forge brotlipy 0.7.0 py37h5e8e339_1003 conda-forge brunsli 0.1 h9c3ff4c_0 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.18.1 h7f98852_0 conda-forge c-blosc2 2.0.4 h5f21a17_1 conda-forge ca-certificates 2021.10.8 ha878542_0 conda-forge cachetools 5.0.0 pyhd8ed1ab_0 conda-forge cairo 1.16.0 h6cf1ce9_1008 conda-forge certifi 2021.10.8 py37h89c1867_1 conda-forge cffi 1.15.0 py37h036bc23_0 conda-forge cfitsio 3.470 hb418390_7 conda-forge charls 2.2.0 h9c3ff4c_0 conda-forge charset-normalizer 2.0.9 pyhd8ed1ab_0 conda-forge click 8.0.3 py37h89c1867_1 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.7.2 pyhd8ed1ab_1 conda-forge cloudpickle 2.0.0 pyhd8ed1ab_0 conda-forge colorama 0.4.4 pyh9f0ad1d_0 conda-forge colorcet 3.0.0 pyhd8ed1ab_0 conda-forge cryptography 36.0.1 py37hf1a17b8_0 conda-forge cucim 21.12.00 cuda_11_py37_g6d1f082_0 rapidsai cudatoolkit 11.0.221 h6bb024c_0 nvidia cudf 21.12.02 cuda_11_py37_g06540b9b37_0 rapidsai cudf_kafka 21.12.02 py37_g06540b9b37_0 rapidsai cugraph 21.12.00 cuda11_py37_g3a43e9d0_0 rapidsai cuml 21.12.00 cuda11_py37_g04c4927f3_0 rapidsai cupy 9.6.0 py37h580ddce_0 conda-forge curl 7.81.0 h2574ce0_0 conda-forge cusignal 21.12.00 py37_g2bf865c_0 rapidsai cuspatial 21.12.00 py37_gab6748f_0 rapidsai custreamz 21.12.02 py37_g06540b9b37_0 rapidsai cuxfilter 21.12.00 py37_g2e0fb5a_0 rapidsai cycler 0.11.0 pyhd8ed1ab_0 conda-forge cyrus-sasl 2.1.27 h230043b_5 conda-forge cytoolz 0.11.2 py37h5e8e339_1 conda-forge dask 2021.11.2 pyhd8ed1ab_0 conda-forge dask-core 2021.11.2 pyhd8ed1ab_0 conda-forge dask-cuda 21.12.00 py37_0 rapidsai dask-cudf 21.12.02 cuda_11_py37_g06540b9b37_0 rapidsai dask-sql 2021.12.0 py37h89c1867_0 conda-forge datashader 0.11.1 pyh9f0ad1d_0 conda-forge datashape 0.5.4 py_1 conda-forge debugpy 1.5.1 py37hcd2ae1e_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge distributed 2021.11.2 py37h89c1867_0 conda-forge dlpack 0.5 h9c3ff4c_0 conda-forge entrypoints 0.3 pyhd8ed1ab_1003 conda-forge expat 2.4.2 h9c3ff4c_0 conda-forge faiss-proc 1.0.0 cuda rapidsai fastapi 0.70.1 pyhd8ed1ab_0 conda-forge fastavro 1.4.8 py37h5e8e339_0 conda-forge fastrlock 0.8 py37hcd2ae1e_1 conda-forge fiona 1.8.20 py37hb7e2723_2 conda-forge flit-core 3.6.0 pyhd8ed1ab_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.13.1 hba837de_1005 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.28.5 py37h5e8e339_0 conda-forge freetype 2.10.4 h0708190_1 conda-forge freexl 1.0.6 h7f98852_0 conda-forge frozenlist 1.2.0 py37h5e8e339_1 conda-forge fsspec 2021.11.1 pyhd8ed1ab_0 conda-forge gdal 3.3.2 py37hd5a0ba4_3 conda-forge geopandas 0.9.0 pyhd8ed1ab_1 conda-forge geopandas-base 0.9.0 pyhd8ed1ab_1 conda-forge geos 3.9.1 h9c3ff4c_2 conda-forge geotiff 1.7.0 h08e826d_2 conda-forge gettext 0.19.8.1 h73d1719_1008 conda-forge gflags 2.2.2 he1b5a44_1004 conda-forge giflib 5.2.1 h36c2ea0_2 conda-forge glog 0.5.0 h48cff8f_0 conda-forge graphite2 1.3.13 h58526e2_1001 conda-forge grpc-cpp 1.42.0 ha1441d3_1 conda-forge h11 0.12.0 pyhd8ed1ab_0 conda-forge harfbuzz 2.9.1 h83ec7ef_1 conda-forge hdf4 4.2.15 h10796ff_3 conda-forge hdf5 1.12.1 nompi_h2750804_103 conda-forge heapdict 1.0.1 py_0 conda-forge icu 68.2 h9c3ff4c_0 conda-forge idna 3.1 pyhd3deb0d_0 conda-forge imagecodecs 2021.8.26 py37hfe5a812_1 conda-forge imageio 2.13.5 pyh239f2a4_0 conda-forge importlib-metadata 4.10.0 py37h89c1867_0 conda-forge importlib_metadata 4.10.0 hd8ed1ab_0 conda-forge importlib_resources 5.4.0 pyhd8ed1ab_0 conda-forge ipykernel 6.6.1 py37h6531663_0 conda-forge ipython 7.31.0 py37h89c1867_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 7.6.5 pyhd8ed1ab_0 conda-forge jbig 2.1 h7f98852_2003 conda-forge jedi 0.18.1 py37h89c1867_0 conda-forge jinja2 3.0.3 pyhd8ed1ab_0 conda-forge joblib 1.1.0 pyhd8ed1ab_0 conda-forge jpeg 9d h36c2ea0_0 conda-forge jpype1 1.3.0 py37h2527ec5_2 conda-forge json-c 0.15 h98cffda_0 conda-forge jsonschema 4.3.3 pyhd8ed1ab_0 conda-forge jupyter-server-proxy 3.2.0 pyhd8ed1ab_0 conda-forge jupyter_client 7.1.0 pyhd8ed1ab_0 conda-forge jupyter_core 4.9.1 py37h89c1867_1 conda-forge jupyter_server 1.13.1 pyhd8ed1ab_0 conda-forge jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge jupyterlab_widgets 1.0.2 pyhd8ed1ab_0 conda-forge jxrlib 1.1 h7f98852_2 conda-forge kealib 1.4.14 h87e4c3c_3 conda-forge kiwisolver 1.3.2 py37h2527ec5_1 conda-forge krb5 1.19.2 hcc1bbae_3 conda-forge lcms2 2.12 hddcbb42_0 conda-forge ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge lerc 3.0 h9c3ff4c_0 conda-forge libaec 1.0.6 h9c3ff4c_0 conda-forge libblas 3.9.0 12_linux64_openblas conda-forge libbrotlicommon 1.0.9 h7f98852_6 conda-forge libbrotlidec 1.0.9 h7f98852_6 conda-forge libbrotlienc 1.0.9 h7f98852_6 conda-forge libcblas 3.9.0 12_linux64_openblas conda-forge libcucim 21.12.00 cuda11_g6d1f082_0 rapidsai libcudf 21.12.02 cuda11_g06540b9b37_0 rapidsai libcudf_kafka 21.12.02 g06540b9b37_0 rapidsai libcugraph 21.12.00 cuda11_g3a43e9d0_0 rapidsai libcuml 21.12.00 cuda11_g04c4927f3_0 rapidsai libcumlprims 21.12.00 cuda11_g0a7f19f_0 nvidia libcurl 7.81.0 h2574ce0_0 conda-forge libcusolver 11.3.2.107 hc875929_0 nvidia libcuspatial 21.12.00 cuda11_gab6748f_0 rapidsai libdap4 3.20.6 hd7c4107_2 conda-forge libdeflate 1.8 h7f98852_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 h9b69904_4 conda-forge libfaiss 1.7.0 cuda110h8045045_8_cuda conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 11.2.0 h1d223b6_11 conda-forge libgcrypt 1.9.4 h7f98852_0 conda-forge libgdal 3.3.2 h6acdded_3 conda-forge libgfortran-ng 11.2.0 h69a702a_11 conda-forge libgfortran5 11.2.0 h5c6108e_11 conda-forge libglib 2.70.2 h174f98d_1 conda-forge libgomp 11.2.0 h1d223b6_11 conda-forge libgpg-error 1.42 h9c3ff4c_0 conda-forge libgsasl 1.10.0 h5b4c23d_0 conda-forge libhwloc 2.3.0 h5e5b7d1_1 conda-forge libiconv 1.16 h516909a_0 conda-forge libkml 1.3.0 h238a007_1014 conda-forge liblapack 3.9.0 12_linux64_openblas conda-forge libllvm11 11.1.0 hf817b99_2 conda-forge libnetcdf 4.8.1 nompi_hb3fd0d9_101 conda-forge libnghttp2 1.43.0 h812cca2_1 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libntlm 1.4 h7f98852_1002 conda-forge libopenblas 0.3.18 pthreads_h8fe5266_0 conda-forge libpng 1.6.37 h21135ba_2 conda-forge libpq 13.5 hd57d9b9_1 conda-forge libprotobuf 3.19.2 h780b84a_0 conda-forge librdkafka 1.6.1 hc49e61c_1 conda-forge librmm 21.12.00 cuda11_g957ad04_0 rapidsai librttopo 1.1.0 h1185371_6 conda-forge libsodium 1.0.18 h36c2ea0_1 conda-forge libspatialindex 1.9.3 h9c3ff4c_4 conda-forge libspatialite 5.0.1 h5cf074c_8 conda-forge libssh2 1.10.0 ha56f1ee_2 conda-forge libstdcxx-ng 11.2.0 he4da1e4_11 conda-forge libthrift 0.15.0 he6d91bd_1 conda-forge libtiff 4.3.0 h6f004c6_2 conda-forge libutf8proc 2.7.0 h7f98852_0 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libuv 1.42.0 h7f98852_0 conda-forge libwebp 1.2.1 h3452ae3_0 conda-forge libwebp-base 1.2.1 h7f98852_0 conda-forge libxcb 1.13 h7f98852_1004 conda-forge libxgboost 1.5.0dev.rapidsai21.12 cuda11.0_0 rapidsai libxml2 2.9.12 h72842e0_0 conda-forge libzip 1.8.0 h4de3113_1 conda-forge libzlib 1.2.11 h36c2ea0_1013 conda-forge libzopfli 1.0.3 h9c3ff4c_0 conda-forge llvmlite 0.37.0 py37h9d7f4d0_1 conda-forge locket 0.2.0 py_2 conda-forge lz4-c 1.9.3 h9c3ff4c_1 conda-forge mapclassify 2.4.3 pyhd8ed1ab_0 conda-forge markdown 3.3.6 pyhd8ed1ab_0 conda-forge markupsafe 2.0.1 py37h5e8e339_1 conda-forge matplotlib-base 3.5.1 py37h1058ff1_0 conda-forge matplotlib-inline 0.1.3 pyhd8ed1ab_0 conda-forge mistune 0.8.4 py37h5e8e339_1005 conda-forge msgpack-python 1.0.3 py37h2527ec5_0 conda-forge multidict 5.2.0 py37h5e8e339_1 conda-forge multipledispatch 0.6.0 py_0 conda-forge munch 2.5.0 py_0 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge nbclient 0.5.9 pyhd8ed1ab_0 conda-forge nbconvert 6.4.0 py37h89c1867_0 conda-forge nbformat 5.1.3 pyhd8ed1ab_0 conda-forge nccl 2.11.4.1 h96e36e3_0 conda-forge ncurses 6.2 h58526e2_4 conda-forge nest-asyncio 1.5.4 pyhd8ed1ab_0 conda-forge networkx 2.6.3 pyhd8ed1ab_1 conda-forge nodejs 14.17.4 h92b4a50_0 conda-forge notebook 6.4.6 pyha770c72_0 conda-forge nspr 4.32 h9c3ff4c_1 conda-forge nss 3.73 hb5efdd6_0 conda-forge numba 0.54.1 py37h2d894fd_0 conda-forge numpy 1.20.3 py37h038b26d_1 conda-forge nvtx 0.2.3 py37h5e8e339_1 conda-forge olefile 0.46 pyh9f0ad1d_1 conda-forge openjdk 11.0.9.1 h5cc2fde_1 conda-forge openjpeg 2.4.0 hb52868f_1 conda-forge openssl 1.1.1l h7f98852_0 conda-forge orc 1.7.1 h1be678f_1 conda-forge packaging 21.3 pyhd8ed1ab_0 conda-forge pandas 1.3.5 py37he8f5f7f_0 conda-forge pandoc 2.16.2 h7f98852_0 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge panel 0.12.4 pyhd8ed1ab_0 conda-forge param 1.12.0 pyh6c4a22f_0 conda-forge parquet-cpp 1.5.1 2 conda-forge parso 0.8.3 pyhd8ed1ab_0 conda-forge partd 1.2.0 pyhd8ed1ab_0 conda-forge pcre 8.45 h9c3ff4c_0 conda-forge pexpect 4.8.0 pyh9f0ad1d_2 conda-forge pickle5 0.0.12 py37h5e8e339_0 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 8.4.0 py37h0f21c89_0 conda-forge pip 21.3.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge pooch 1.5.2 pyhd8ed1ab_0 conda-forge poppler 21.09.0 ha39eefc_3 conda-forge poppler-data 0.4.11 hd8ed1ab_0 conda-forge postgresql 13.5 h2510834_1 conda-forge proj 8.1.0 h277dcde_1 conda-forge prometheus_client 0.12.0 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.24 pyha770c72_0 conda-forge protobuf 3.19.2 py37hcd2ae1e_0 conda-forge psutil 5.9.0 py37h5e8e339_0 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge ptxcompiler 0.2.0 py37h81e21aa_0 rapidsai ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge py-xgboost 1.5.0dev.rapidsai21.12 cuda11.0py37_0 rapidsai pyarrow 5.0.0 py37h63cede7_20_cuda conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pyct 0.4.6 py_0 conda-forge pyct-core 0.4.6 py_0 conda-forge pydantic 1.9.0 py37h5e8e339_0 conda-forge pydeck 0.5.0 pyh9f0ad1d_0 conda-forge pyee 8.1.0 pyh9f0ad1d_0 conda-forge pygments 2.11.1 pyhd8ed1ab_0 conda-forge pynvml 11.4.1 pyhd8ed1ab_0 conda-forge pyopenssl 21.0.0 pyhd8ed1ab_0 conda-forge pyparsing 3.0.6 pyhd8ed1ab_0 conda-forge pyppeteer 0.2.6 pyhd8ed1ab_0 conda-forge pyproj 3.1.0 py37hdf91f24_4 conda-forge pyrsistent 0.18.0 py37h5e8e339_0 conda-forge pysocks 1.7.1 py37h89c1867_4 conda-forge python 3.7.12 hb7a2778_100_cpython conda-forge python-confluent-kafka 1.6.0 py37h5e8e339_1 conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-tzdata 2021.5 pyhd8ed1ab_0 conda-forge python_abi 3.7 2_cp37m conda-forge pytz 2021.3 pyhd8ed1ab_0 conda-forge pytz-deprecation-shim 0.1.0.post0 py37h89c1867_1 conda-forge pyviz_comms 2.1.0 pyhd8ed1ab_0 conda-forge pywavelets 1.2.0 py37hb1e94ed_1 conda-forge pyyaml 6.0 py37h5e8e339_3 conda-forge pyzmq 22.3.0 py37h336d617_1 conda-forge rapids 21.12.00 cuda11.0_py37_gc46440c_94 rapidsai rapids-xgboost 21.12.00 cuda11.0_py37_gc46440c_94 rapidsai re2 2021.11.01 h9c3ff4c_0 conda-forge readline 8.1 h46c0cb4_0 conda-forge requests 2.27.0 pyhd8ed1ab_0 conda-forge rmm 21.12.00 cuda11_py37_g957ad04_0_no_cma rapidsai rtree 0.9.7 py37h0b55af0_3 conda-forge s2n 1.3.0 h9b69904_0 conda-forge scikit-image 0.18.1 py37hdc94413_0 conda-forge scikit-learn 1.0.2 py37hf9e9bfc_0 conda-forge scipy 1.7.3 py37hf2a6cf1_0 conda-forge send2trash 1.8.0 pyhd8ed1ab_0 conda-forge setuptools 60.2.0 py37h89c1867_0 conda-forge shapely 1.8.0 py37h48c49eb_0 conda-forge simpervisor 0.4 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.8 he1b5a44_3 conda-forge sniffio 1.2.0 py37h89c1867_2 conda-forge sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge spdlog 1.8.5 h4bd325d_0 conda-forge sqlite 3.37.0 h9cd32fc_0 conda-forge starlette 0.16.0 pyhd8ed1ab_0 conda-forge streamz 0.6.3 pyh6c4a22f_0 conda-forge tabulate 0.8.9 pyhd8ed1ab_0 conda-forge tblib 1.7.0 pyhd8ed1ab_0 conda-forge terminado 0.12.1 py37h89c1867_1 conda-forge testpath 0.5.0 pyhd8ed1ab_0 conda-forge threadpoolctl 3.0.0 pyh8a188c0_0 conda-forge tifffile 2021.11.2 pyhd8ed1ab_0 conda-forge tiledb 2.3.4 he87e0bf_0 conda-forge tk 8.6.11 h27826a3_1 conda-forge toolz 0.11.2 pyhd8ed1ab_0 conda-forge tornado 6.1 py37h5e8e339_2 conda-forge tqdm 4.62.3 pyhd8ed1ab_0 conda-forge traitlets 5.1.1 pyhd8ed1ab_0 conda-forge treelite 2.1.0 py37h4b3d254_0 conda-forge treelite-runtime 2.1.0 pypi_0 pypi typing-extensions 4.0.1 hd8ed1ab_0 conda-forge typing_extensions 4.0.1 pyha770c72_0 conda-forge tzcode 2021e h7f98852_0 conda-forge tzdata 2021e he74cb21_0 conda-forge tzlocal 4.1 py37h89c1867_1 conda-forge ucx 1.11.2+gef2bbcf cuda11.0_0 rapidsai ucx-proc 1.0.0 gpu rapidsai ucx-py 0.23.0 py37_gef2bbcf_0 rapidsai unicodedata2 14.0.0 py37h5e8e339_0 conda-forge urllib3 1.26.7 pyhd8ed1ab_0 conda-forge uvicorn 0.16.0 py37h89c1867_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 py_1 conda-forge websocket-client 1.2.3 pyhd8ed1ab_0 conda-forge websockets 9.1 py37h5e8e339_0 conda-forge wheel 0.37.1 pyhd8ed1ab_0 conda-forge widgetsnbextension 3.5.2 py37h89c1867_1 conda-forge xarray 0.20.2 pyhd8ed1ab_0 conda-forge xerces-c 3.2.3 h9d8b166_3 conda-forge xgboost 1.5.0dev.rapidsai21.12 cuda11.0py37_0 rapidsai xorg-fixesproto 5.0 h7f98852_1002 conda-forge xorg-inputproto 2.3.2 h7f98852_1002 conda-forge xorg-kbproto 1.0.7 h7f98852_1002 conda-forge xorg-libice 1.0.10 h7f98852_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.7.2 h7f98852_0 conda-forge xorg-libxau 1.0.9 h7f98852_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xorg-libxext 1.3.4 h7f98852_1 conda-forge xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge xorg-libxi 1.7.10 h7f98852_0 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-libxtst 1.2.3 h7f98852_1002 conda-forge xorg-recordproto 1.14.2 h7f98852_1002 conda-forge xorg-renderproto 0.11.1 h7f98852_1002 conda-forge xorg-xextproto 7.3.0 h7f98852_1002 conda-forge xorg-xproto 7.0.31 h7f98852_1007 conda-forge xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h7f98852_2 conda-forge yarl 1.7.2 py37h5e8e339_1 conda-forge zeromq 4.3.4 h9c3ff4c_1 conda-forge zfp 0.5.5 h9c3ff4c_8 conda-forge zict 2.0.0 py_0 conda-forge zipp 3.6.0 pyhd8ed1ab_0 conda-forge zlib 1.2.11 h36c2ea0_1013 conda-forge zstd 1.5.1 ha95c52a_0 conda-forge (rapids-21.12) anamaria@gpu-2-0.saber:~ $

anamariaUIC commented 2 years ago

@galipremsagar additional related question: in order to use DASK do I need to have NVLink on my GPUs?

anamariaUIC commented 2 years ago

@galipremsagar are there are any updates on these questions?

beckernick commented 2 years ago

Using Dask does not require NVLink.

Your conda environment appears to have Dask (dask 2021.11.2 pyhd8ed1ab_0 conda-forge). Is it possible your environment is not correctly set up on your system (do other libraries work), or that you may be picking up a different Dask from somewhere else on the system?

anamariaUIC commented 2 years ago

@beckernick I can import dask.dataframe but not dask.dataframe.dispatch. please see the following:

(rapids-21.12) anamaria@gpu-2-0.saber:~ $ python Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:53) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import dask import dask.dataframe.dispatch Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'dask.dataframe.dispatch' import dask.dataframe import dask.dataframe.dispatch Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'dask.dataframe.dispatch'

(rapids-21.12) anamaria@gpu-2-0.saber:~ $ env PROJ_LIB=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids-21.12/share/proj CUDA_PATH=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids-21.12 MANPATH=/software/linux-el7-x86_64/tools/lmod/lmod/share/man:: XDG_SESSION_ID=3727 JAVA_LD_LIBRARY_PATH=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids-21.12/lib/server ModuleTable003=dmljZXMvaG9kLy5sb2NhbC9lYXN5YnVpbGQvbW9kdWxlcy9hbGw6L3NvZnR3YXJlL2xpbnV4LWVsNy14ODZfNjQvdG9vbHMvbG1vZC9sbW9kL21vZHVsZWZpbGVzL0NvcmUiLH0= HOSTNAME=gpu-2-0.saber SELINUX_ROLE_REQUESTED= TERM=xterm-256color SHELL=/bin/bash HISTSIZE=1000 MODULEPATH_ROOT=/software/linux-el7-x86_64/tools/modulefiles SSH_CLIENT=192.168.15.4 57388 22 CONDA_SHLVL=1 CONDA_PROMPT_MODIFIER=(rapids-21.12) SELINUX_USE_CURRENT_RANGE= LMOD_PKG=/software/linux-el7-x86_64/tools/lmod/lmod QTDIR=/usr/lib64/qt-3.3 LMOD_VERSION=7.7 QTINC=/usr/lib64/qt-3.3/include SSH_TTY=/dev/pts/1 __LMOD_REF_COUNT_LOADEDMODULES=apps/anaconda3:1 QT_GRAPHICSSYSTEM_CHECKED=1 USER=anamaria LMOD_sys=Linux LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:.tar=38;5;9:.tgz=38;5;9:.arc=38;5;9:.arj=38;5;9:.taz=38;5;9:.lha=38;5;9:.lz4=38;5;9:.lzh=38;5;9:.lzma=38;5;9:.tlz=38;5;9:.txz=38;5;9:.tzo=38;5;9:.t7z=38;5;9:.zip=38;5;9:.z=38;5;9:.Z=38;5;9:.dz=38;5;9:.gz=38;5;9:.lrz=38;5;9:.lz=38;5;9:.lzo=38;5;9:.xz=38;5;9:.bz2=38;5;9:.bz=38;5;9:.tbz=38;5;9:.tbz2=38;5;9:.tz=38;5;9:.deb=38;5;9:.rpm=38;5;9:.jar=38;5;9:.war=38;5;9:.ear=38;5;9:.sar=38;5;9:.rar=38;5;9:.alz=38;5;9:.ace=38;5;9:.zoo=38;5;9:.cpio=38;5;9:.7z=38;5;9:.rz=38;5;9:.cab=38;5;9:.jpg=38;5;13:.jpeg=38;5;13:.gif=38;5;13:.bmp=38;5;13:.pbm=38;5;13:.pgm=38;5;13:.ppm=38;5;13:.tga=38;5;13:.xbm=38;5;13:.xpm=38;5;13:.tif=38;5;13:.tiff=38;5;13:.png=38;5;13:.svg=38;5;13:.svgz=38;5;13:.mng=38;5;13:.pcx=38;5;13:.mov=38;5;13:.mpg=38;5;13:.mpeg=38;5;13:.m2v=38;5;13:.mkv=38;5;13:.webm=38;5;13:.ogm=38;5;13:.mp4=38;5;13:.m4v=38;5;13:.mp4v=38;5;13:.vob=38;5;13:.qt=38;5;13:.nuv=38;5;13:.wmv=38;5;13:.asf=38;5;13:.rm=38;5;13:.rmvb=38;5;13:.flc=38;5;13:.avi=38;5;13:.fli=38;5;13:.flv=38;5;13:.gl=38;5;13:.dl=38;5;13:.xcf=38;5;13:.xwd=38;5;13:.yuv=38;5;13:.cgm=38;5;13:.emf=38;5;13:.axv=38;5;13:.anx=38;5;13:.ogv=38;5;13:.ogx=38;5;13:.aac=38;5;45:.au=38;5;45:.flac=38;5;45:.mid=38;5;45:.midi=38;5;45:.mka=38;5;45:.mp3=38;5;45:.mpc=38;5;45:.ogg=38;5;45:.ra=38;5;45:.wav=38;5;45:.axa=38;5;45:.oga=38;5;45:.spx=38;5;45:*.xspf=38;5;45: CONDA_EXE=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin/conda LMOD_REF_COUNTLMFILES_=/software/linux-el7-x86_64/modules/apps/anaconda3:1 _CE_CONDA= CPL_ZIP_ENCODING=UTF-8 ModuleTable001=X01vZHVsZVRhYmxlXz17WyJNVHZlcnNpb24iXT0zLFsiY19yZWJ1aWxkVGltZSJdPWZhbHNlLFsiY19zaG9ydFRpbWUiXT1mYWxzZSxkZXB0aFQ9e30sZmFtaWx5PXt9LG1UPXtbImFwcHMvYW5hY29uZGEzIl09e1siZm4iXT0iL3NvZnR3YXJlL2xpbnV4LWVsNy14ODZfNjQvbW9kdWxlcy9hcHBzL2FuYWNvbmRhMyIsWyJmdWxsTmFtZSJdPSJhcHBzL2FuYWNvbmRhMyIsWyJsb2FkT3JkZXIiXT0xLHByb3BUPXt9LFsic3RhY2tEZXB0aCJdPTAsWyJzdGF0dXMiXT0iYWN0aXZlIixbInVzZXJOYW1lIl09ImFwcHMvYW5hY29uZGEzIix9LH0sbXBhdGhBPXsiL3NvZnR3YXJlL2xpbnV4LWVsNy14ODZfNjQvdG9vbHMvRWFzeUJ1aWxkLTQuMS4wL21vZHVsZXMvYWxsIiwiL3NvZnR3 MAIL=/var/spool/mail/anamaria PATH=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids-21.12/bin:/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin:/software/linux-el7-x86_64/apps/anaconda3/anaconda3/condabin:/usr/local/bin:/usr/lib64/qt-3.3/bin:/usr/bin:/usr/local/sbin:/usr/sbin PROJ_NETWORK=ON CONDA_PREFIX=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids-21.12 LMOD_SETTARG_CMD=: PWD=/home/anamaria LMFILES=/software/linux-el7-x86_64/modules/apps/anaconda3 JAVA_HOME=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids-21.12 LANG=en_US.UTF-8 MODULEPATH=/software/linux-el7-x86_64/tools/EasyBuild-4.1.0/modules/all:/software/linux-el7-x86_64/modules:/software/linux-el7-nvidia/modules:/software/linux-el7-x86_64/services/hod/.local/easybuild/modules/all:/software/linux-el7-x86_64/tools/lmod/lmod/modulefiles/Core _ModuleTableSz=3 LOADEDMODULES=apps/anaconda3 KDEDIRS=/usr SELINUX_LEVEL_REQUESTED= GDAL_DATA=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids-21.12/share/gdal LMOD_CMD=/software/linux-el7-x86_64/tools/lmod/lmod/libexec/lmod _CE_M= SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass HISTCONTROL=ignoredups SHLVL=1 HOME=/home/anamaria __LMOD_REF_COUNT_PATH=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin:1;/software/linux-el7-x86_64/apps/anaconda3/anaconda3/condabin:1;/usr/local/bin:2;/usr/lib64/qt-3.3/bin:1;/usr/bin:1;/usr/local/sbin:1;/usr/sbin:1 ModuleTable002=YXJlL2xpbnV4LWVsNy14ODZfNjQvbW9kdWxlcyIsIi9zb2Z0d2FyZS9saW51eC1lbDctbnZpZGlhL21vZHVsZXMiLCIvc29mdHdhcmUvbGludXgtZWw3LXg4Nl82NC9zZXJ2aWNlcy9ob2QvLmxvY2FsL2Vhc3lidWlsZC9tb2R1bGVzL2FsbCIsIi9zb2Z0d2FyZS9saW51eC1lbDcteDg2XzY0L3Rvb2xzL2xtb2QvbG1vZC9tb2R1bGVmaWxlcy9Db3JlIix9LFsic3lzdGVtQmFzZU1QQVRIIl09Ii9zb2Z0d2FyZS9saW51eC1lbDcteDg2XzY0L3Rvb2xzL0Vhc3lCdWlsZC00LjEuMC9tb2R1bGVzL2FsbDovc29mdHdhcmUvbGludXgtZWw3LXg4Nl82NC9tb2R1bGVzOi9zb2Z0d2FyZS9saW51eC1lbDctbnZpZGlhL21vZHVsZXM6L3NvZnR3YXJlL2xpbnV4LWVsNy14ODZfNjQvc2Vy BASH_ENV=/software/linux-el7-x86_64/tools/lmod/lmod/init/bash CONDA_PYTHON_EXE=/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin/python LOGNAME=anamaria QTLIB=/usr/lib64/qt-3.3/lib XDG_DATA_DIRS=/home/anamaria/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share SSH_CONNECTION=192.168.15.4 57388 192.168.12.36 22 MODULESHOME=/software/linux-el7-x86_64/tools/lmod/lmod JAVA_HOME_CONDA_BACKUP=/software/linux-el7-x86_64/compilers/jdk-1.8.0_131 CONDA_DEFAULT_ENV=rapids-21.12 LESSOPEN=||/usr/bin/lesspipe.sh %s LMOD_FULL_SETTARG_SUPPORT=no XDG_RUNTIME_DIR=/run/user/554479 QT_PLUGIN_PATH=/usr/lib64/kde4/plugins:/usr/lib/kde4/plugins JAVA_LD_LIBRARY_PATH_BACKUP= LMOD_DIR=/software/linux-el7-x86_64/tools/lmod/lmod/libexec BASH_FUNC_module()=() { eval $($LMOD_CMD bash "$@") && eval $(${LMOD_SETTARG_CMD:-:} -s sh) } BASH_FUNC_ml()=() { eval $($LMOD_DIR/mlcmd "$@") } =/usr/bin/env

anamariaUIC commented 2 years ago

Hello,

Can someone please advise me on this: import dask_cudf

ModuleNotFoundError Traceback (most recent call last) /tmp/ipykernel_59808/2855451223.py in ----> 1 import dask_cudf

/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/lib/python3.7/site-packages/dask_cudf/init.py in 4 from cudf._version import get_versions 5 ----> 6 from . import backends 7 from .core import DataFrame, Series, concat, from_cudf, from_dask_dataframe 8 from .groupby import groupby_agg

/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/lib/python3.7/site-packages/dask_cudf/backends.py in 9 10 from dask.dataframe.core import get_parallel_type, meta_nonempty ---> 11 from dask.dataframe.dispatch import ( 12 categorical_dtype_dispatch, 13 concat_dispatch,

ModuleNotFoundError: No module named 'dask.dataframe.dispatch'

anamariaUIC commented 2 years ago

This is what I have in .bashrc

# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/software/linux-el7-x86_64/apps/anaconda3/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/software/linux-el7-x86_64/apps/anaconda3/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
anamariaUIC commented 2 years ago

I am submitting this as well, if it is of any help to debug this:

`(rapids) anamaria@gpu-2-0.saber:~ $ python Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:53) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import os os.environ environ({'PROJ_LIB': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/share/proj', 'CUDA_PATH': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids', 'MANPATH': '/software/linux-el7-x86_64/tools/lmod/lmod/share/man::', 'XDG_SESSION_ID': '4116', 'JAVA_LD_LIBRARY_PATH': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/lib/server', 'ModuleTable003': 'dmljZXMvaG9kLy5sb2NhbC9lYXN5YnVpbGQvbW9kdWxlcy9hbGw6L3NvZnR3YXJlL2xpbnV4LWVsNy14ODZfNjQvdG9vbHMvbG1vZC9sbW9kL21vZHVsZWZpbGVzL0NvcmUiLH0=', 'HOSTNAME': 'gpu-2-0.saber', 'SELINUX_ROLE_REQUESTED': '', 'TERM': 'xterm-256color', 'SHELL': '/bin/bash', 'HISTSIZE': '1000', 'MODULEPATH_ROOT': '/software/linux-el7-x86_64/tools/modulefiles', 'SSH_CLIENT': '192.168.15.4 58410 22', 'CONDA_SHLVL': '1', 'CONDA_PROMPT_MODIFIER': '(rapids) ', 'SELINUX_USE_CURRENT_RANGE': '', 'LMOD_PKG': '/software/linux-el7-x86_64/tools/lmod/lmod', 'QTDIR': '/usr/lib64/qt-3.3', 'LMOD_VERSION': '7.7', 'QTINC': '/usr/lib64/qt-3.3/include', 'SSH_TTY': '/dev/pts/1', '__LMOD_REF_COUNT_LOADEDMODULES': 'apps/anaconda3:1', 'QT_GRAPHICSSYSTEM_CHECKED': '1', 'USER': 'anamaria', 'LMOD_sys': 'Linux', 'LS_COLORS': 'rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:.tar=38;5;9:.tgz=38;5;9:.arc=38;5;9:.arj=38;5;9:.taz=38;5;9:.lha=38;5;9:.lz4=38;5;9:.lzh=38;5;9:.lzma=38;5;9:.tlz=38;5;9:.txz=38;5;9:.tzo=38;5;9:.t7z=38;5;9:.zip=38;5;9:.z=38;5;9:.Z=38;5;9:.dz=38;5;9:.gz=38;5;9:.lrz=38;5;9:.lz=38;5;9:.lzo=38;5;9:.xz=38;5;9:.bz2=38;5;9:.bz=38;5;9:.tbz=38;5;9:.tbz2=38;5;9:.tz=38;5;9:.deb=38;5;9:.rpm=38;5;9:.jar=38;5;9:.war=38;5;9:.ear=38;5;9:.sar=38;5;9:.rar=38;5;9:.alz=38;5;9:.ace=38;5;9:.zoo=38;5;9:.cpio=38;5;9:.7z=38;5;9:.rz=38;5;9:.cab=38;5;9:.jpg=38;5;13:.jpeg=38;5;13:.gif=38;5;13:.bmp=38;5;13:.pbm=38;5;13:.pgm=38;5;13:.ppm=38;5;13:.tga=38;5;13:.xbm=38;5;13:.xpm=38;5;13:.tif=38;5;13:.tiff=38;5;13:.png=38;5;13:.svg=38;5;13:.svgz=38;5;13:.mng=38;5;13:.pcx=38;5;13:.mov=38;5;13:.mpg=38;5;13:.mpeg=38;5;13:.m2v=38;5;13:.mkv=38;5;13:.webm=38;5;13:.ogm=38;5;13:.mp4=38;5;13:.m4v=38;5;13:.mp4v=38;5;13:.vob=38;5;13:.qt=38;5;13:.nuv=38;5;13:.wmv=38;5;13:.asf=38;5;13:.rm=38;5;13:.rmvb=38;5;13:.flc=38;5;13:.avi=38;5;13:.fli=38;5;13:.flv=38;5;13:.gl=38;5;13:.dl=38;5;13:.xcf=38;5;13:.xwd=38;5;13:.yuv=38;5;13:.cgm=38;5;13:.emf=38;5;13:.axv=38;5;13:.anx=38;5;13:.ogv=38;5;13:.ogx=38;5;13:.aac=38;5;45:.au=38;5;45:.flac=38;5;45:.mid=38;5;45:.midi=38;5;45:.mka=38;5;45:.mp3=38;5;45:.mpc=38;5;45:.ogg=38;5;45:.ra=38;5;45:.wav=38;5;45:.axa=38;5;45:.oga=38;5;45:.spx=38;5;45:*.xspf=38;5;45:', 'CONDA_EXE': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin/conda', 'LMOD_REF_COUNTLMFILES_': '/software/linux-el7-x86_64/modules/apps/anaconda3:1', '_CE_CONDA': '', 'CPL_ZIP_ENCODING': 'UTF-8', 'ModuleTable001': 'X01vZHVsZVRhYmxlXz17WyJNVHZlcnNpb24iXT0zLFsiY19yZWJ1aWxkVGltZSJdPWZhbHNlLFsiY19zaG9ydFRpbWUiXT1mYWxzZSxkZXB0aFQ9e30sZmFtaWx5PXt9LG1UPXtbImFwcHMvYW5hY29uZGEzIl09e1siZm4iXT0iL3NvZnR3YXJlL2xpbnV4LWVsNy14ODZfNjQvbW9kdWxlcy9hcHBzL2FuYWNvbmRhMyIsWyJmdWxsTmFtZSJdPSJhcHBzL2FuYWNvbmRhMyIsWyJsb2FkT3JkZXIiXT0xLHByb3BUPXt9LFsic3RhY2tEZXB0aCJdPTAsWyJzdGF0dXMiXT0iYWN0aXZlIixbInVzZXJOYW1lIl09ImFwcHMvYW5hY29uZGEzIix9LH0sbXBhdGhBPXsiL3NvZnR3YXJlL2xpbnV4LWVsNy14ODZfNjQvdG9vbHMvRWFzeUJ1aWxkLTQuMS4wL21vZHVsZXMvYWxsIiwiL3NvZnR3', 'MAIL': '/var/spool/mail/anamaria', 'PATH': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/bin:/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin:/software/linux-el7-x86_64/apps/anaconda3/anaconda3/condabin:/usr/local/bin:/usr/lib64/qt-3.3/bin:/usr/bin:/usr/local/sbin:/usr/sbin', 'PROJ_NETWORK': 'ON', 'CONDA_PREFIX': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids', 'LMOD_SETTARG_CMD': ':', 'PWD': '/home/anamaria', 'JAVA_HOME': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids', 'LMFILES': '/software/linux-el7-x86_64/modules/apps/anaconda3', 'LANG': 'en_US.UTF-8', 'MODULEPATH': '/software/linux-el7-x86_64/tools/EasyBuild-4.1.0/modules/all:/software/linux-el7-x86_64/modules:/software/linux-el7-nvidia/modules:/software/linux-el7-x86_64/services/hod/.local/easybuild/modules/all:/software/linux-el7-x86_64/tools/lmod/lmod/modulefiles/Core', '_ModuleTableSz': '3', 'LOADEDMODULES': 'apps/anaconda3', 'KDEDIRS': '/usr', 'SELINUX_LEVEL_REQUESTED': '', 'GDAL_DATA': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/share/gdal', 'LMOD_CMD': '/software/linux-el7-x86_64/tools/lmod/lmod/libexec/lmod', '_CE_M': '', 'SSH_ASKPASS': '/usr/libexec/openssh/gnome-ssh-askpass', 'HISTCONTROL': 'ignoredups', 'SHLVL': '1', 'HOME': '/home/anamaria', '__LMOD_REF_COUNT_PATH': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin:1;/software/linux-el7-x86_64/apps/anaconda3/anaconda3/condabin:1;/usr/local/bin:2;/usr/lib64/qt-3.3/bin:1;/usr/bin:1;/usr/local/sbin:1;/usr/sbin:1', 'ModuleTable002': 'YXJlL2xpbnV4LWVsNy14ODZfNjQvbW9kdWxlcyIsIi9zb2Z0d2FyZS9saW51eC1lbDctbnZpZGlhL21vZHVsZXMiLCIvc29mdHdhcmUvbGludXgtZWw3LXg4Nl82NC9zZXJ2aWNlcy9ob2QvLmxvY2FsL2Vhc3lidWlsZC9tb2R1bGVzL2FsbCIsIi9zb2Z0d2FyZS9saW51eC1lbDcteDg2XzY0L3Rvb2xzL2xtb2QvbG1vZC9tb2R1bGVmaWxlcy9Db3JlIix9LFsic3lzdGVtQmFzZU1QQVRIIl09Ii9zb2Z0d2FyZS9saW51eC1lbDcteDg2XzY0L3Rvb2xzL0Vhc3lCdWlsZC00LjEuMC9tb2R1bGVzL2FsbDovc29mdHdhcmUvbGludXgtZWw3LXg4Nl82NC9tb2R1bGVzOi9zb2Z0d2FyZS9saW51eC1lbDctbnZpZGlhL21vZHVsZXM6L3NvZnR3YXJlL2xpbnV4LWVsNy14ODZfNjQvc2Vy', 'BASH_ENV': '/software/linux-el7-x86_64/tools/lmod/lmod/init/bash', 'CONDA_PYTHON_EXE': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/bin/python', 'LOGNAME': 'anamaria', 'QTLIB': '/usr/lib64/qt-3.3/lib', 'XDG_DATA_DIRS': '/home/anamaria/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share', 'SSH_CONNECTION': '192.168.15.4 58410 192.168.12.36 22', 'MODULESHOME': '/software/linux-el7-x86_64/tools/lmod/lmod', 'JAVA_HOME_CONDA_BACKUP': '', 'CONDA_DEFAULT_ENV': 'rapids', 'LESSOPEN': '||/usr/bin/lesspipe.sh %s', 'LMOD_FULL_SETTARG_SUPPORT': 'no', 'XDG_RUNTIME_DIR': '/run/user/554479', 'QT_PLUGIN_PATH': '/usr/lib64/kde4/plugins:/usr/lib/kde4/plugins', 'JAVA_LD_LIBRARY_PATH_BACKUP': '', 'LMOD_DIR': '/software/linux-el7-x86_64/tools/lmod/lmod/libexec', 'BASH_FUNC_module()': '() { eval $($LMOD_CMD bash "$@") && eval $(${LMOD_SETTARG_CMD:-:} -s sh)\n}', 'BASH_FUNC_ml()': '() { eval $($LMOD_DIR/mlcmd "$@")\n}', '': '/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/bin/python'})

`

anamariaUIC commented 2 years ago

Ok, I resolved this and here is solution for all people who might be facing the same issue:

I obtained these lines from a person on the same cluster for whom importing dask_cudf did work:

>>> import dask
>>> print(dask.__file__)
/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/lib/python3.7/site-packages/dask/__init__.py

This is what I was getting:

>>> import dask
>>> print(dask.__file__)
/home/anamaria/.local/lib/python3.7/site-packages/dask/__init__.py

My issue was resolved by putting in my .bashrc:

export PYTHONPATH=$PYTHONPATH:"/software/linux-el7-x86_64/apps/anaconda3/anaconda3/envs/rapids/lib/python3.7/site-packages”

github-actions[bot] commented 2 years ago

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.