search

rapidsai / cuml

cuML - RAPIDS Machine Learning Library
https://docs.rapids.ai/api/cuml/stable/
Apache License 2.0
4.16k stars 525 forks source link

[BUG] Received unsupported input type: <class 'dask_cudf.core.DataFrame'> when doing cuml_model.predict(X_test_dask) on Google Colab #3050

Open BioGeek opened 3 years ago

BioGeek commented 3 years ago

Describe the bug I used the rapidsai-csp-utils to install RAPIDS and related libraries on Google Colab. I also installed dask-cuda via conda. Then I copy/pasted and executed the code from the Random Forests Multi-node, Multi-GPU demo notebook.

In the last cell, I get the error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-16-79cdc37bea3f> in <module>()
      1 skl_y_pred = skl_model.predict(X_test)
----> 2 cuml_y_pred = cuml_model.predict(X_test_dask).compute().to_array()
      3 
      4 # Due to randomness in the algorithm, you may see slight variation in accuracies
      5 print("SKLearn accuracy:  ", accuracy_score(y_test, skl_y_pred))

12 frames
cuml/ensemble/randomforestclassifier.pyx in cuml.ensemble.randomforestclassifier.RandomForestClassifier._predict_get_all()

/usr/local/lib/python3.6/site-packages/cuml/common/input_utils.py in convert_dtype()
    413 
    414     else:
--> 415         raise TypeError("Received unsupported input type: %s" % type(X))
    416 
    417     return X

TypeError: Received unsupported input type: <class 'dask_cudf.core.DataFrame'>

Steps/Code to reproduce bug

Execute the cells in this Google Colab notebook. The error happens in cell 16.

Expected behavior

cuml_model predicts the labels for X_test_dask.

Environment details (please complete the following information):

Click here to see environment details

     **git***
     Not inside a git repository

     ***OS Information***
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=18.04
     DISTRIB_CODENAME=bionic
     DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
     NAME="Ubuntu"
     VERSION="18.04.5 LTS (Bionic Beaver)"
     ID=ubuntu
     ID_LIKE=debian
     PRETTY_NAME="Ubuntu 18.04.5 LTS"
     VERSION_ID="18.04"
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     VERSION_CODENAME=bionic
     UBUNTU_CODENAME=bionic
     Linux 2426acb48820 4.19.112+ #1 SMP Thu Jul 23 08:00:38 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Fri Oct 23 13:51:28 2020
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 455.23.05    Driver Version: 418.67       CUDA Version: 10.1     |
     |-------------------------------+----------------------+----------------------+
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |
     | N/A   46C    P0    27W /  70W |    854MiB / 15079MiB |      0%      Default |
     |                               |                      |                 ERR! |
     +-------------------------------+----------------------+----------------------+

     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     |  No running processes found                                                 |
     +-----------------------------------------------------------------------------+

     ***CPU***
     Architecture:        x86_64
     CPU op-mode(s):      32-bit, 64-bit
     Byte Order:          Little Endian
     CPU(s):              2
     On-line CPU(s) list: 0,1
     Thread(s) per core:  2
     Core(s) per socket:  1
     Socket(s):           1
     NUMA node(s):        1
     Vendor ID:           GenuineIntel
     CPU family:          6
     Model:               79
     Model name:          Intel(R) Xeon(R) CPU @ 2.20GHz
     Stepping:            0
     CPU MHz:             2200.000
     BogoMIPS:            4400.00
     Hypervisor vendor:   KVM
     Virtualization type: full
     L1d cache:           32K
     L1i cache:           32K
     L2 cache:            256K
     L3 cache:            56320K
     NUMA node0 CPU(s):   0,1
     Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt arat md_clear arch_capabilities

     ***CMake***
     /usr/local/bin/cmake
     cmake version 3.12.0

     CMake suite maintained and supported by Kitware (kitware.com/cmake).

     ***g++***
     /usr/bin/g++
     g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
     Copyright (C) 2017 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

     ***nvcc***
     /usr/local/cuda/bin/nvcc
     nvcc: NVIDIA (R) Cuda compiler driver
     Copyright (c) 2005-2019 NVIDIA Corporation
     Built on Sun_Jul_28_19:07:16_PDT_2019
     Cuda compilation tools, release 10.1, V10.1.243

     ***Python***
     /usr/local/bin/python
     Python 3.6.12

     ***Environment Variables***
     PATH                            : /usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin:/opt/bin
     LD_LIBRARY_PATH                 : /usr/lib64-nvidia
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    :
     PYTHON_PATH                     :

     ***conda packages***
     /usr/local/bin/conda
     # packages in environment at /usr/local:
     #
     # Name                    Version                   Build  Channel
     _libgcc_mutex             0.1                 conda_forge    conda-forge
     _openmp_mutex             4.5                       1_gnu    conda-forge
     aiohttp                   3.6.2            py36h8c4c3a4_1    conda-forge
     arrow-cpp                 0.15.0           py36h090bef1_2    conda-forge
     async-timeout             3.0.1                   py_1000    conda-forge
     attrs                     20.2.0             pyh9f0ad1d_0    conda-forge
     blinker                   1.4                        py_1    conda-forge
     bokeh                     2.2.3            py36h9f0ad1d_0    conda-forge
     boost                     1.70.0           py36h9de70de_1    conda-forge
     boost-cpp                 1.70.0               h8e57a91_2    conda-forge
     brotli                    1.0.9                he1b5a44_2    conda-forge
     brotlipy                  0.7.0           py36he6145b8_1001    conda-forge
     bzip2                     1.0.8                h516909a_3    conda-forge
     c-ares                    1.16.1               h516909a_3    conda-forge
     ca-certificates           2020.10.14                    0
     cachetools                4.1.1                      py_0    conda-forge
     cairo                     1.16.0            hcf35c78_1003    conda-forge
     certifi                   2020.6.20        py36h9880bd3_2    conda-forge
     cffi                      1.11.5                   py36_0    conda-forge
     cfitsio                   3.470                h0392d32_7    conda-forge
     chardet                   3.0.4           py36h9880bd3_1008    conda-forge
     click                     7.1.2              pyh9f0ad1d_0    conda-forge
     cloudpickle               1.6.0                      py_0    conda-forge
     conda                     4.9.0            py36h9880bd3_1    conda-forge
     conda-package-handling    1.7.2            py36he6145b8_0    conda-forge
     contextvars               2.4                        py_0    conda-forge
     cryptography              3.1.1            py36h5d711a5_1    conda-forge
     cudatoolkit               10.1.243             h6bb024c_0    nvidia
     cudf                      0.14.0                   py36_0    rapidsai/label/main
     cudnn                     7.6.0                cuda10.1_0    nvidia
     cugraph                   0.14.0                   py36_0    rapidsai/label/main
     cuml                      0.14.0          cuda10.1_py36_0    rapidsai/label/main
     cupy                      7.8.0            py36h5c369b2_1    conda-forge
     curl                      7.71.1               he644dc0_8    conda-forge
     cusignal                  0.16.0          py38_g61980b6_0    rapidsai/label/main
     cuspatial                 0.14.0                   py36_0    rapidsai/label/main
     cytoolz                   0.11.0           py36h8c4c3a4_1    conda-forge
     dask                      2.30.0                     py_0    conda-forge
     dask-core                 2.30.0                     py_0    conda-forge
     dask-cuda                 0.14.1                   py36_0    rapidsai
     dask-cudf                 0.14.0                   py36_0    rapidsai/label/main
     distributed               2.30.0           py36h9f0ad1d_1    conda-forge
     dlpack                    0.3                  he1b5a44_1    conda-forge
     double-conversion         3.1.5                he1b5a44_2    conda-forge
     expat                     2.2.9                he1b5a44_2    conda-forge
     fastavro                  1.0.0.post1      py36h8c4c3a4_1    conda-forge
     fastrlock                 0.5              py36h831f99a_1    conda-forge
     fontconfig                2.13.1            h86ecdb6_1001    conda-forge
     freetype                  2.10.4               he06d7ca_0    conda-forge
     freexl                    1.0.5             h516909a_1002    conda-forge
     fsspec                    0.8.4                      py_0    conda-forge
     gcsfs                     0.7.1                      py_0    conda-forge
     gdal                      3.0.4           py36hd60729c_10    conda-forge
     geos                      3.8.1                he1b5a44_0    conda-forge
     geotiff                   1.6.0                h05acad5_0    conda-forge
     gflags                    2.2.2             he1b5a44_1004    conda-forge
     giflib                    5.2.1                h516909a_2    conda-forge
     glib                      2.66.1               h92f7085_0
     glog                      0.4.0                h49b9bf7_3    conda-forge
     google-auth               1.22.0                     py_0    conda-forge
     google-auth-oauthlib      0.4.1                      py_2    conda-forge
     grpc-cpp                  1.23.0               h18db393_0    conda-forge
     hdf4                      4.2.13            hf30be14_1003    conda-forge
     hdf5                      1.10.6          nompi_h1022a3e_1110    conda-forge
     heapdict                  1.0.1                      py_0    conda-forge
     icu                       64.2                 he1b5a44_1    conda-forge
     idna                      2.10               pyh9f0ad1d_0    conda-forge
     idna_ssl                  1.1.0           py36h9f0ad1d_1001    conda-forge
     immutables                0.14             py36h8c4c3a4_1    conda-forge
     jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge
     joblib                    0.17.0                     py_0    conda-forge
     jpeg                      9d                   h516909a_0    conda-forge
     json-c                    0.13.1            hbfbb72e_1002    conda-forge
     kealib                    1.4.13               h33137a7_1    conda-forge
     krb5                      1.17.1               hfafb76e_3    conda-forge
     lcms2                     2.11                 hbd6801e_0    conda-forge
     ld_impl_linux-64          2.35                 h769bd43_9    conda-forge
     libblas                   3.8.0               17_openblas    conda-forge
     libcblas                  3.8.0               17_openblas    conda-forge
     libcudf                   0.14.0               cuda10.1_0    rapidsai/label/main
     libcugraph                0.14.0               cuda10.1_0    rapidsai/label/main
     libcuml                   0.14.0               cuda10.1_0    rapidsai/label/main
     libcumlprims              0.14.1               cuda10.1_0    nvidia
     libcurl                   7.71.1               hcdd3856_8    conda-forge
     libcuspatial              0.14.0               cuda10.1_0    rapidsai/label/main
     libdap4                   3.20.6               h1d1bd15_1    conda-forge
     libedit                   3.1.20191231         he28a2e2_2    conda-forge
     libev                     4.33                 h516909a_1    conda-forge
     libevent                  2.1.10               hcdb4288_3    conda-forge
     libffi                    3.3                  he6710b0_2
     libgcc-ng                 9.3.0               h5dbcf3e_17    conda-forge
     libgdal                   3.0.4               he6a97d6_10    conda-forge
     libgfortran-ng            9.3.0               he4bcb1c_17    conda-forge
     libgfortran5              9.3.0               he4bcb1c_17    conda-forge
     libgomp                   9.3.0               h5dbcf3e_17    conda-forge
     libhwloc                  2.1.0                h3c4fd83_0    conda-forge
     libiconv                  1.16                 h516909a_0    conda-forge
     libkml                    1.3.0             h4fcabce_1010    conda-forge
     liblapack                 3.8.0               17_openblas    conda-forge
     libllvm9                  9.0.1                he513fc3_1    conda-forge
     libnetcdf                 4.7.4           nompi_hefab0ff_106    conda-forge
     libnghttp2                1.41.0               h8cfc5f6_2    conda-forge
     libnvstrings              0.14.0               cuda10.1_0    rapidsai/label/main
     libopenblas               0.3.10          pthreads_h4812303_5    conda-forge
     libpng                    1.6.37               hed695b0_2    conda-forge
     libpq                     12.3                 h5513abc_2    conda-forge
     libprotobuf               3.8.0                h8b12597_0    conda-forge
     librmm                    0.14.0               cuda10.1_0    rapidsai/label/main
     libspatialite             4.3.0a            h2482549_1038    conda-forge
     libssh2                   1.9.0                hab1572f_5    conda-forge
     libstdcxx-ng              9.3.0               h2ae2ef3_17    conda-forge
     libtiff                   4.1.0                hc7e4089_6    conda-forge
     libuuid                   2.32.1            h14c3975_1000    conda-forge
     libwebp-base              1.1.0                h516909a_3    conda-forge
     libxcb                    1.13              h14c3975_1002    conda-forge
     libxgboost                1.1.0dev.rapidsai0.14      cuda10.1_0    rapidsai/label/main
     libxml2                   2.9.10               hee79883_0    conda-forge
     llvmlite                  0.33.0           py36hfa65bc7_1    conda-forge
     locket                    0.2.0                      py_2    conda-forge
     lz4-c                     1.8.3             he1b5a44_1001    conda-forge
     markupsafe                1.1.1            py36he6145b8_2    conda-forge
     msgpack-python            1.0.0            py36hdb11119_2    conda-forge
     multidict                 4.7.5            py36h8c4c3a4_2    conda-forge
     nccl                      2.5.7.1              h51cf6c1_0    conda-forge
     ncurses                   6.2                  he1b5a44_2    conda-forge
     numba                     0.50.1           py36h0573a6f_1
     numpy                     1.19.2           py36h68c22af_1    conda-forge
     nvstrings                 0.14.0                   py36_0    rapidsai/label/main
     oauthlib                  3.0.1                      py_0    conda-forge
     olefile                   0.46               pyh9f0ad1d_1    conda-forge
     openjpeg                  2.3.1                h981e76c_3    conda-forge
     openssl                   1.1.1h               h516909a_0    conda-forge
     packaging                 20.4               pyh9f0ad1d_0    conda-forge
     pandas                    0.25.3           py36hb3f55d8_0    conda-forge
     parquet-cpp               1.5.1                         2    conda-forge
     partd                     1.1.0                      py_0    conda-forge
     pcre                      8.44                 he1b5a44_0    conda-forge
     pillow                    8.0.1            py36h8328e55_0    conda-forge
     pip                       20.2.4                     py_0    conda-forge
     pixman                    0.38.0            h516909a_1003    conda-forge
     poppler                   0.87.0               h4190859_1    conda-forge
     poppler-data              0.4.10                        0    conda-forge
     postgresql                12.3                 h8573dbc_2    conda-forge
     proj                      7.0.0                h966b41f_5    conda-forge
     psutil                    5.7.2            py36he6145b8_1    conda-forge
     pthread-stubs             0.4               h14c3975_1001    conda-forge
     py-xgboost                1.1.0dev.rapidsai0.14  cuda10.1py36_0    rapidsai/label/main
     pyarrow                   0.15.0           py36h8b68381_1    conda-forge
     pyasn1                    0.4.8                      py_0    conda-forge
     pyasn1-modules            0.2.7                      py_0    conda-forge
     pycosat                   0.6.3           py36he6145b8_1005    conda-forge
     pycparser                 2.20               pyh9f0ad1d_2    conda-forge
     pyjwt                     1.7.1                      py_0    conda-forge
     pynvml                    8.0.4                      py_1    conda-forge
     pyopenssl                 19.1.0                     py_1    conda-forge
     pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
     pysocks                   1.7.1            py36h9880bd3_2    conda-forge
     python                    3.6.12               hcff3b4d_2
     python-dateutil           2.8.1                      py_0    conda-forge
     python_abi                3.6                     1_cp36m    conda-forge
     pytz                      2020.1             pyh9f0ad1d_0    conda-forge
     pyyaml                    5.3.1            py36he6145b8_1    conda-forge
     re2                       2020.04.01           he1b5a44_0    conda-forge
     readline                  8.0                  he28a2e2_2    conda-forge
     requests                  2.24.0             pyh9f0ad1d_0    conda-forge
     requests-oauthlib         1.3.0              pyh9f0ad1d_0    conda-forge
     rmm                       0.14.0                   py36_0    rapidsai/label/main
     rsa                       4.6                pyh9f0ad1d_0    conda-forge
     ruamel_yaml               0.15.80         py36he6145b8_1003    conda-forge
     scikit-learn              0.23.2           py36h9e5ed8f_1    conda-forge
     scipy                     1.5.2            py36h82117a9_2    conda-forge
     setuptools                49.6.0           py36h9880bd3_2    conda-forge
     six                       1.15.0             pyh9f0ad1d_0    conda-forge
     snappy                    1.1.8                he1b5a44_3    conda-forge
     sortedcontainers          2.2.2              pyh9f0ad1d_0    conda-forge
     spdlog                    1.8.1                hc9558a2_0    conda-forge
     sqlite                    3.33.0               h4cf870e_1    conda-forge
     tbb                       2018.0.5             h2d50403_0    conda-forge
     tblib                     1.6.0                      py_0    conda-forge
     threadpoolctl             2.1.0              pyh5ca1d4c_0    conda-forge
     thrift-cpp                0.12.0            hf3afdfd_1004    conda-forge
     tiledb                    1.7.7                hcde45ca_0    conda-forge
     tk                        8.6.10               hed695b0_1    conda-forge
     toolz                     0.11.1                     py_0    conda-forge
     tornado                   6.0.4            py36h8c4c3a4_2    conda-forge
     tqdm                      4.50.2             pyh9f0ad1d_0    conda-forge
     typing-extensions         3.7.4.3                       0    conda-forge
     typing_extensions         3.7.4.3                    py_0    conda-forge
     tzcode                    2020a                h516909a_0    conda-forge
     ucx                       1.8.0+gf6ec8d4      cuda10.1_20    rapidsai/label/main
     ucx-py                    0.14.0+gf6ec8d4          py36_0    rapidsai/label/main
     uriparser                 0.9.3                he1b5a44_1    conda-forge
     urllib3                   1.25.11                    py_0    conda-forge
     wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
     xerces-c                  3.2.2             h8412b87_1004    conda-forge
     xgboost                   1.1.0dev.rapidsai0.14  cuda10.1py36_0    rapidsai/label/main
     xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
     xorg-libice               1.0.10               h516909a_0    conda-forge
     xorg-libsm                1.2.3             h84519dc_1000    conda-forge
     xorg-libx11               1.6.12               h516909a_0    conda-forge
     xorg-libxau               1.0.9                h14c3975_0    conda-forge
     xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
     xorg-libxext              1.3.4                h516909a_0    conda-forge
     xorg-libxrender           0.9.10            h516909a_1002    conda-forge
     xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
     xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
     xorg-xproto               7.0.31            h14c3975_1007    conda-forge
     xz                        5.2.5                h516909a_1    conda-forge
     yaml                      0.2.5                h516909a_0    conda-forge
     yarl                      1.6.2            py36h8c4c3a4_0    conda-forge
     zict                      2.0.0                      py_0    conda-forge
     zlib                      1.2.11            h516909a_1010    conda-forge
     zstd                      1.4.4                h3b9ef0a_2    conda-forge
viclafargue commented 3 years ago

Thank you for opening the issue. It seems like you are running an up to date (0.17) notebook with RAPIDS softwares in version 0.14. This can unfortunately fail as API specifications change with different versions.

viclafargue commented 3 years ago

Maybe @Salonijain27 has more information on this?

github-actions[bot] commented 3 years ago

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

github-actions[bot] commented 3 years ago

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.