datametrician
commented
5 years ago @joehoeller I don't believe anyone on the RAPIDS team uses PyCharm. If you want to experiment and provide docs on this that would be amazing!
Closed daddydrac closed 5 years ago
datametrician
commented
5 years ago @joehoeller I don't believe anyone on the RAPIDS team uses PyCharm. If you want to experiment and provide docs on this that would be amazing!
daddydrac
commented
5 years ago Rapids then is a concept. Nothing I can build a system or an app with. It’s an immature product is what you’re telling me, and not quite ready for prime time.
daddydrac
commented
5 years ago I’m closing this out. I’ll come back when it’s ready to be used to build software.
mrocklin
commented
5 years ago @joehoeller I'm curious how do you set up PyCharm to use Pandas?
The same procedure should also work with RAPIDS.
daddydrac
commented
5 years ago Fair comeback. In all seriousness I did that, and it doesn’t come up at all in settings > project interpreter > Anaconda
daddydrac
commented
5 years ago Also, I get this: The conda install line is taken right from the instructions on the cuml repo, where it says, "cuML can be installed using the rapidsai conda channel".
`conda install -c nvidia -c rapidsai -c conda-forge -c pytorch -c defaults cuml Collecting package metadata: done Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
Current channels:
To search for alternate channels that may provide the conda package you're looking for, navigate to
https://anaconda.organd use the search bar at the top of the page. `
daddydrac
commented
5 years ago Because none of the automated installs worked (conda install *), I decided to build from source. During the test phase, these 3 failed:
[ FAILED ] 3 tests, listed below:
[ FAILED ] KmeansTests/KmeansTestF.Fit/0, where GetParam() = 16-byte object <02-00 00-00 CD-CC 4C-3D 04-00 00-00 02-00 00-00>
[ FAILED ] KmeansTests/KmeansTestD.Fit/0, where GetParam() = 24-byte object <02-00 00-00 00-00 00-00 9A-99 99-99 99-99 A9-3F 04-00 00-00 02-00 00-00>
[ FAILED ] PcaTests/PcaTestDataVecF.Result/0, where GetParam() = I also ran into these errors after running py.test:
================================ ERRORS ====================================
__________________ ERROR collecting cuML/test/test_dbscan.py ___________________
ImportError while importing test module '/home/joe/Desktop/rapids/cuml/python/cuML/test/test_dbscan.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
cuML/test/test_dbscan.py:17: in <module>
from cuml import DBSCAN as cuDBSCAN
E ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
_______________ ERROR collecting cuML/test/test_kalman_filter.py _______________
ImportError while importing test module '/home/joe/Desktop/rapids/cuml/python/cuML/test/test_kalman_filter.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
cuML/test/test_kalman_filter.py:17: in <module>
from cuml import KalmanFilter
E ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
__________________ ERROR collecting cuML/test/test_kmeans.py ___________________
ImportError while importing test module '/home/joe/Desktop/rapids/cuml/python/cuML/test/test_kmeans.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
cuML/test/test_kmeans.py:20: in <module>
import cuml
E ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
_______________ ERROR collecting cuML/test/test_linear_model.py ________________
ImportError while importing test module '/home/joe/Desktop/rapids/cuml/python/cuML/test/test_linear_model.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
cuML/test/test_linear_model.py:17: in <module>
from cuml import LinearRegression as cuLinearRegression
E ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
____________________ ERROR collecting cuML/test/test_pca.py ____________________
ImportError while importing test module '/home/joe/Desktop/rapids/cuml/python/cuML/test/test_pca.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
cuML/test/test_pca.py:17: in <module>
from cuml import PCA as cuPCA
E ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
___________________ ERROR collecting cuML/test/test_tsvd.py ____________________
ImportError while importing test module '/home/joe/Desktop/rapids/cuml/python/cuML/test/test_tsvd.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
cuML/test/test_tsvd.py:16: in <module>
from cuml import TruncatedSVD as cuTSVD
E ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
!!!!!!!!!!!!!!!!!!! Interrupted: 6 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 6 error in 0.16 seconds ============================Here is some addt'l errata, not sure it will help:
Python 3.7.1 | packaged by conda-forge | (default, Nov 13 2018, 18:33:04)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
from cuml import KNN as cumlKNN
ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
>>>
$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130
NVIDIA-SMI 410.48 Driver Version: 410.48 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce RTX 2080 Ti... Off | 00000000:41:00.0 On | N/A |
| 41% 26C P8 7W / 260W | 650MiB / 10986MiB | 3% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1501 G /usr/lib/xorg/Xorg 40MiB |
| 0 1740 G /usr/bin/gnome-shell 58MiB |
| 0 2732 G /usr/lib/xorg/Xorg 341MiB |
| 0 2862 G /usr/bin/gnome-shell 79MiB |
| 0 3550 G ...quest-channel-token=1174352793230323240 127MiB |
+-----------------------------------------------------------------------------+Maybe you can point me to what I need to do to fix that?
mrocklin
commented
5 years ago Other than the build issues it sounds like you're having an issue getting conda and pycharm to play nicely together. I recommend asking upstream. Rapids libraries distribute through conda like any other package.
On Thu, Jan 31, 2019, 9:55 PM joehoeller <notifications@github.com wrote:
Because none of the automated installs worked (conda install *), I decided to build from source. During the test phase, these 3 failed:
[ FAILED ] 3 tests, listed below: [ FAILED ] KmeansTests/KmeansTestF.Fit/0, where GetParam() = 16-byte object <02-00 00-00 CD-CC 4C-3D 04-00 00-00 02-00 00-00> [ FAILED ] KmeansTests/KmeansTestD.Fit/0, where GetParam() = 24-byte object <02-00 00-00 00-00 00-00 9A-99 99-99 99-99 A9-3F 04-00 00-00 02-00 00-00> [ FAILED ] PcaTests/PcaTestDataVecF.Result/0, where GetParam() =
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rapidsai/cuml/issues/174#issuecomment-459615435, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszGmVsk_tm4G5SJPDJc34GhIrdncZks5vI9bigaJpZM4aclG3 .
daddydrac
commented
5 years ago @mrocklin - please see all the errors, one in particular is:
from cuml import KNN as cumlKNN ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
mrocklin
commented
5 years ago Do you have a contact URL for upstream?
Nope. You'll have to find out how to report things to either PyCharm or conda yourself as appropriate. I suspect that this is particular to your setup though. Sorry I can't be of more help. Good luck!
daddydrac
commented
5 years ago It’s just Ubuntu 18, Anaconda 5/Python 3.7, & CUDA 10.
kkraus14
commented
5 years ago It looks like cuBLAS isn't installed / recognized based on what I'm seeing but I would defer to @dantegd or @cjnolet here.
dantegd
commented
5 years ago I think @kkraus14 is right it looks like an issue finding cuBLAS during runtime of the python package (not the c++ libcuml). I have a couple of ideas of what could be the issue, but can’t say for sure until I’ve seen the details of the environment. @joehoeller could you post the full output of the c++ tests, your environment variables and conda packages (with the conda environment you used for building)? Thanks.
daddydrac
commented
5 years ago @dantegd, on my lunch break now, but when I get off from my FT i'll be sure to do that this evening. Thank you for pointing this out, much appreciated.
daddydrac
commented
5 years ago @dantegd
Env var's:
CLUTTER_IM_MODULE=xim
CONDA_SHLVL=1
CONDA_EXE=/home/joe/anaconda3/bin/conda
LESSCLOSE=/usr/bin/lesspipe %s %s
XDG_MENU_PREFIX=gnome-
LANG=en_US.UTF-8
DISPLAY=:1
GNOME_SHELL_SESSION_MODE=ubuntu
COLORTERM=truecolor
USERNAME=joe
CONDA_PREFIX=/home/joe/anaconda3
XDG_VTNR=2
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
MANDATORY_PATH=/usr/share/gconf/ubuntu.mandatory.path
XDG_SESSION_ID=3
USER=joe
DESKTOP_SESSION=ubuntu
QT4_IM_MODULE=xim
TEXTDOMAINDIR=/usr/share/locale/
GNOME_TERMINAL_SCREEN=/org/gnome/Terminal/screen/e46940b2_6ee4_4b1c_bec1_037fbb3aa3e3
DEFAULTS_PATH=/usr/share/gconf/ubuntu.default.path
PWD=/home/joe
HOME=/home/joe
CONDA_PYTHON_EXE=/home/joe/anaconda3/bin/python
TEXTDOMAIN=im-config
SSH_AGENT_PID=7956
QT_ACCESSIBILITY=1
LIBVIRT_DEFAULT_URI=qemu:///system
XDG_SESSION_TYPE=x11
XDG_DATA_DIRS=/usr/share/ubuntu:/usr/local/share:/usr/share:/var/lib/snapd/desktop
XDG_SESSION_DESKTOP=ubuntu
GJS_DEBUG_OUTPUT=stderr
CONDA_PROMPT_MODIFIER=(base)
GTK_MODULES=gail:atk-bridge
WINDOWPATH=2
TERM=xterm-256color
SHELL=/bin/bash
VTE_VERSION=5202
QT_IM_MODULE=xim
XMODIFIERS=@im=ibus
IM_CONFIG_PHASE=2
XDG_CURRENT_DESKTOP=ubuntu:GNOME
GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
GNOME_TERMINAL_SERVICE=:1.107
XDG_SEAT=seat0
SHLVL=1
GDMSESSION=ubuntu
GNOME_DESKTOP_SESSION_ID=this-is-deprecated
LOGNAME=joe
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
XDG_RUNTIME_DIR=/run/user/1000
XAUTHORITY=/run/user/1000/gdm/Xauthority
XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/etc/xdg
PATH=/usr/local/cuda-10.0/bin:/home/joe/anaconda3/bin:/home/joe/anaconda3/condabin:/home/joe/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/joe/.local/bin
CONDA_DEFAULT_ENV=base
GJS_DEBUG_TOPICS=JS ERROR;JS LOG
SESSION_MANAGER=local/uptrend:@/tmp/.ICE-unix/7859,unix/uptrend:/tmp/.ICE-unix/7859
LESSOPEN=| /usr/bin/lesspipe %s
GTK_IM_MODULE=ibus
_=/usr/bin/printenv
@dantegd - $PATH vars:
echo $PATH
/usr/local/cuda-10.0/bin:/home/joe/anaconda3/bin:/home/joe/anaconda3/condabin:/home/joe/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/joe/.local/bin@dantegd List of Anaconda packages:
conda list
# packages in environment at /home/joe/anaconda3:
#
# Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py37_0
alabaster 0.7.11 py37_0
anaconda 5.3.1 py37_0
anaconda-client 1.7.2 py37_0
anaconda-navigator 1.9.2 py37_0
anaconda-project 0.8.2 py37_0
appdirs 1.4.3 py37h28b3542_0
arrow 0.12.1 py37_1002 conda-forge
arrow-cpp 0.12.0 py37h0e61e49_0 conda-forge
asn1crypto 0.24.0 py37_0
astroid 2.0.4 py37_0
astropy 3.0.4 py37h14c3975_0
atomicwrites 1.2.1 py37_0
attrs 18.2.0 py37h28b3542_0
automat 0.7.0 py37_0
babel 2.6.0 py37_0
backcall 0.1.0 py37_0
backports 1.0 py37_1
backports.shutil_get_terminal_size 1.0.0 py37_2
beautifulsoup4 4.6.3 py37_0
binaryornot 0.4.4 py_1 conda-forge
bitarray 0.8.3 py37h14c3975_0
bkcharts 0.2 py37_0
blas 1.0 mkl
blaze 0.11.3 py37_0
bleach 2.1.4 py37_0
blosc 1.14.4 hdbcaa40_0
bokeh 0.13.0 py37_0
boost-cpp 1.68.0 h11c811c_1000 conda-forge
boto 2.49.0 py37_0
bottleneck 1.2.1 py37h035aef0_1
bzip2 1.0.6 h14c3975_5
ca-certificates 2018.12.5 0 anaconda
cairo 1.14.12 h8948797_3
certifi 2018.11.29 py37_0 anaconda
cffi 1.11.5 py37he75722e_1
chardet 3.0.4 py37_1
click 6.7 py37_0
cloudpickle 0.5.5 py37_0
clyent 1.2.2 py37_1
colorama 0.3.9 py37_0
conda 4.6.2 py37_0
conda-build 3.15.1 py37_0
conda-env 2.6.0 1
constantly 15.1.0 py37h28b3542_0
contextlib2 0.5.5 py37_0
cookiecutter 1.6.0 py37_1000 conda-forge
cryptography 2.5 py37hb7f436b_0 conda-forge
cuda100 1.0 0 pytorch
cudatoolkit 9.0 h13b8566_0
cudf 0.5.0 py37_18 rapidsai/label/cuda10.0
cudf-cuda100 0.5.0 pypi_0 pypi
cudnn 7.3.1 cuda9.0_0
cuml 0.6.0.dev0+6.gd39cf85 pypi_0 pypi
cuml-cuda100 0.5.0 pypi_0 pypi
curl 7.26.0 1 anaconda
cycler 0.10.0 py37_0
cython 0.29.2 py37he6710b0_0 anaconda
cytoolz 0.9.0.1 py37h14c3975_1
dask 0.19.1 py37_0
dask-core 0.19.1 py37_0
datashape 0.5.4 py37_1
dbus 1.13.2 h714fa37_1
decorator 4.3.0 py37_0
defusedxml 0.5.0 py37_1
distributed 1.23.1 py37_0
docutils 0.14 py37_0
entrypoints 0.2.3 py37_2
et_xmlfile 1.0.1 py37_0
expat 2.2.6 he6710b0_0
faiss-gpu 1.5.0 py37_cuda10.0_1 [cuda100] pytorch
fastcache 1.0.2 py37h14c3975_2
filelock 3.0.8 py37_0
flask 1.0.2 py37_1
flask-cors 3.0.6 py37_0
fontconfig 2.13.0 h9420a91_0
freetype 2.9.1 h8a8886c_1
fribidi 1.0.5 h7b6447c_0
future 0.17.1 py37_1000 conda-forge
get_terminal_size 1.0.0 haa9412d_0
gevent 1.3.6 py37h7b6447c_0
glib 2.56.2 hd408876_0
glob2 0.6 py37_0
gmp 6.1.2 h6c8ec71_1
gmpy2 2.0.8 py37h10f8cd9_2
graphite2 1.3.12 h23475e2_2
greenlet 0.4.15 py37h7b6447c_0
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
h5py 2.8.0 py37h989c5e5_3
harfbuzz 1.8.8 hffaf4a1_0
hdf5 1.10.2 hba1933b_1
heapdict 1.0.0 py37_2
html5lib 1.0.1 py37_0
hyperlink 18.0.0 py37_0
icu 58.2 h9c2bf20_1
idna 2.7 py37_0
imageio 2.4.1 py37_0
imagesize 1.1.0 py37_0
incremental 17.5.0 py37_0
intel-openmp 2019.0 118
ipykernel 4.9.0 py37_1
ipython 6.5.0 py37_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.4.1 py37_0
isort 4.3.4 py37_0
itsdangerous 0.24 py37_1
jbig 2.1 hdba287a_0
jdcal 1.4 py37_0
jedi 0.12.1 py37_0
jeepney 0.3.1 py37_0
jinja2 2.10 py37_0
jinja2-time 0.2.0 py_2 conda-forge
jpeg 9b h024ee3a_2
jsonschema 2.6.0 py37_0
jupyter 1.0.0 py37_7
jupyter_client 5.2.3 py37_0
jupyter_console 5.2.0 py37_1
jupyter_core 4.4.0 py37_0
jupyterlab 0.34.9 py37_0
jupyterlab_launcher 0.13.1 py37_0
keyring 13.2.1 py37_0
kiwisolver 1.0.1 py37hf484d3e_0
krb5 1.16.2 hc83ff2d_1000 conda-forge
lazy-object-proxy 1.3.1 py37h14c3975_2
libcudf 0.5.0 cuda10.0_18 rapidsai/label/cuda10.0
libcudf_cffi 0.5.0 cuda10.0_py37_18 rapidsai/label/cuda10.0
libcurl 7.63.0 h01ee5af_1000 conda-forge
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libgfortran-ng 7.3.0 hdf63c60_0
libpng 1.6.35 hbc83047_0 anaconda
libprotobuf 3.6.1 hdbcaa40_1000 conda-forge
libsodium 1.0.16 h1bed415_0
libssh2 1.8.0 h1ad7b7a_1003 conda-forge
libstdcxx-ng 8.2.0 hdf63c60_1
libtiff 4.0.9 he85c1e1_2
libtool 2.4.6 h544aabb_3
libuuid 1.0.3 h1bed415_2
libxcb 1.13 h1bed415_1
libxml2 2.9.8 h26e45fe_1
libxslt 1.1.32 h1312cb7_0
llvmlite 0.27.0 py37hd408876_0
locket 0.2.0 py37_1
lxml 4.2.5 py37hefd8a0e_0
lzo 2.10 h49e0be7_2
markupsafe 1.0 py37h14c3975_1
matplotlib 2.2.3 py37hb69df0a_0
mccabe 0.6.1 py37_1
mistune 0.8.3 py37h14c3975_1
mkl 2018.0.3 1
mkl-service 1.1.2 py37h90e4bf4_5
mkl_fft 1.0.6 py37h7dd41cf_0 anaconda
mkl_random 1.0.1 py37h4414c95_1 anaconda
more-itertools 4.3.0 py37_0
mpc 1.1.0 h10f8cd9_1
mpfr 4.0.1 hdf1c602_3
mpmath 1.0.0 py37_2
msgpack-python 0.5.6 py37h6bb024c_1
multipledispatch 0.6.0 py37_0
navigator-updater 0.2.1 py37_0
nbconvert 5.4.0 py37_1
nbformat 4.4.0 py37_0
nccl 1.3.5 cuda9.0_0
ncurses 6.1 hf484d3e_0
networkx 2.1 py37_0
ninja 1.8.2 py37h6bb024c_1
nltk 3.3.0 py37_0
nose 1.3.7 py37_2
notebook 5.6.0 py37_0
numba 0.42.0 py37h962f231_0
numexpr 2.6.8 py37hd89afb7_0
numpy 1.15.4 py37h1d66e8a_0 anaconda
numpy-base 1.15.4 py37h81de0dd_0 anaconda
numpydoc 0.8.0 py37_0
nvstrings 0.2.0 cuda10.0_py37_0 nvidia/label/cuda10.0
nvstrings-cuda100 0.2.0.post2 pypi_0 pypi
odo 0.5.1 py37_0
olefile 0.46 py37_0
openpyxl 2.5.6 py37_0
openssl 1.0.2p h14c3975_0 anaconda
packaging 17.1 py37_0
pandas 0.24.0 py37he6710b0_0
pandoc 1.19.2.1 hea2e7c5_1
pandocfilters 1.4.2 py37_1
pango 1.42.4 h049681c_0
parquet-cpp 1.5.1 4 conda-forge
parso 0.3.1 py37_0
partd 0.3.8 py37_0
patchelf 0.9 hf484d3e_2
path.py 11.1.0 py37_0
pathlib2 2.3.2 py37_0
patsy 0.5.0 py37_0
pcre 8.42 h439df22_0
pep8 1.7.1 py37_0
pexpect 4.6.0 py37_0
pickleshare 0.7.4 py37_0
pillow 5.2.0 py37heded4f4_0
pip 19.0.1 pypi_0 pypi
pixman 0.34.0 hceecf20_3
pkginfo 1.4.2 py37_1
pluggy 0.7.1 py37h28b3542_0
ply 3.11 py37_0
poyo 0.4.2 py_0 conda-forge
prometheus_client 0.3.1 py37h28b3542_0
prompt_toolkit 1.0.15 py37_0
psutil 5.4.7 py37h14c3975_0
ptyprocess 0.6.0 py37_0
py 1.6.0 py37_0
pyarrow 0.11.1 pypi_0 pypi
pyasn1 0.4.4 py37h28b3542_0
pyasn1-modules 0.2.2 py37_0
pycodestyle 2.4.0 py37_0
pycosat 0.6.3 py37h14c3975_0
pycparser 2.19 pypi_0 pypi
pycrypto 2.6.1 py37h14c3975_9
pycurl 7.43.0.2 py37hb7f436b_0
pyflakes 2.0.0 py37_0
pygments 2.2.0 py37_0
pyhamcrest 1.9.0 pypi_0 pypi
pylint 2.1.1 py37_0
pyodbc 4.0.24 py37he6710b0_0
pyopenssl 18.0.0 py37_0
pyparsing 2.2.0 py37_1
pyqt 5.9.2 py37h05f1152_2
pysocks 1.6.8 py37_0
pytables 3.4.4 py37ha205bf6_0
pytest 3.8.0 py37_0
pytest-arraydiff 0.2 py37h39e3cac_0
pytest-astropy 0.4.0 py37_0
pytest-doctestplus 0.1.3 py37_0
pytest-openfiles 0.3.0 py37_0
pytest-remotedata 0.3.0 py37_0
python 3.7.1 hd21baee_1000 conda-forge
python-dateutil 2.7.5 py37_0
pytorch 1.0.0 py3.7_cuda10.0.130_cudnn7.4.1_1 [cuda100] pytorch
pytz 2018.9 py37_0
pywavelets 1.0.0 py37hdd07704_0
pyyaml 3.13 py37h14c3975_0
pyzmq 17.1.2 py37h14c3975_0
qt 5.9.6 h8703b6f_2
qtawesome 0.4.4 py37_0
qtconsole 4.4.1 py37_0
qtpy 1.5.0 py37_0
readline 7.0 h7b6447c_5
requests 2.19.1 py37_0
rope 0.11.0 py37_0
ruamel_yaml 0.15.46 py37h14c3975_0
scikit-image 0.14.0 py37hf484d3e_1
scikit-learn 0.19.2 py37h4989274_0
scipy 1.1.0 py37hfa4b5c9_1
seaborn 0.9.0 py37_0
secretstorage 3.1.0 py37_0
send2trash 1.5.0 py37_0
service_identity 17.0.0 py37h28b3542_0
setuptools 40.6.3 py37_0
simplegeneric 0.8.1 py37_2
singledispatch 3.4.0.3 py37_0
sip 4.19.8 py37hf484d3e_0
six 1.12.0 py37_0
snappy 1.1.7 hbae5bb6_3
snowballstemmer 1.2.1 py37_0
sortedcollections 1.0.1 py37_0
sortedcontainers 2.0.5 py37_0
sphinx 1.7.9 py37_0
sphinxcontrib 1.0 py37_1
sphinxcontrib-websupport 1.1.0 py37_1
spyder 3.3.1 py37_1
spyder-kernels 0.2.6 py37_0
sqlalchemy 1.2.11 py37h7b6447c_0
sqlite 3.25.3 h7b6447c_0 anaconda
statsmodels 0.9.0 py37h035aef0_0
sympy 1.2 py37_0
tblib 1.3.2 py37_0
terminado 0.8.1 py37_1
testpath 0.3.1 py37_0
thrift-cpp 0.12.0 h23e226f_1000 conda-forge
tk 8.6.8 hbc83047_0
toolz 0.9.0 py37_0
torchvision 0.2.1 py_2 pytorch
tornado 5.1 py37h14c3975_0
tqdm 4.26.0 py37h28b3542_0
traitlets 4.3.2 py37_0
twisted 18.7.0 py37h14c3975_1
unicodecsv 0.14.1 py37_0
unixodbc 2.3.7 h14c3975_0
urllib3 1.23 py37_0
wcwidth 0.1.7 py37_0
webencodings 0.5.1 py37_1
werkzeug 0.14.1 py37_0
wheel 0.32.3 py37_0
whichcraft 0.5.2 py_1 conda-forge
widgetsnbextension 3.4.1 py37_0
wrapt 1.10.11 py37h14c3975_2
xlrd 1.1.0 py37_1
xlsxwriter 1.1.0 py37_0
xlwt 1.3.0 py37_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zeromq 4.2.5 hf484d3e_1
zict 0.1.3 py37_0
zlib 1.2.11 h7b6447c_3 anaconda
zope 1.0 py37_1
zope.interface 4.5.0 py37h14c3975_0 @dantegd - Something doesn't feel right here, thoughts?
cuda100 1.0 0 pytorch
cudatoolkit 9.0 h13b8566_0
cudf 0.5.0 py37_18 rapidsai/label/cuda10.0
cudf-cuda100 0.5.0 pypi_0 pypi
cudnn 7.3.1 cuda9.0_0
cuml 0.6.0.dev0+6.gd39cf85 pypi_0 pypi
cuml-cuda100 0.5.0 pypi_0 pypiWhen I am running:
(base) joe@ml-dev-box:~/NVIDIA_CUDA-10.0_Samples $ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130@joehoeller I haven't had time to reproduce yet, but f I'm not mistaken the cudatoolkit installed by conda is the issue (which you well identified). Since it installed the toolkit 9.0 it is conflicting/hiding the system one (10.0), so that is the reason that the needed cuda 10 cublas is not being found. So my first thought is to uninstall the conda cudatoolkit package to solve the issue (might need to rebuild). If that's not enough let us know to look more into it to recreate the environment and see if there are any potential issues in our build instructions/files. Thanks!
kkraus14
commented
5 years ago @joehoeller I haven't had time to reproduce yet, but f I'm not mistaken the
cudatoolkitinstalled by conda is the issue (which you well identified). Since it installed the toolkit 9.0 it is conflicting/hiding the system one (10.0), so that is the reason that the needed cuda 10 cublas is not being found. So my first thought is to uninstall the condacudatoolkitpackage to solve the issue (might need to rebuild). If that's not enough let us know to look more into it to recreate the environment and see if there are any potential issues in our build instructions/files. Thanks!
The conda cudatoolkit package doesn't ship the nvcc compiler while it does ship the libraries, so it's likely that while nvcc is reporting CUDA 10 that the libraries being found are from the conda package on CUDA 9.0. @joehoeller it would be great if you could run numba -s and dump the output here for us to confirm.
daddydrac
commented
5 years ago Yeah, built container here: https://github.com/joehoeller/computer-vision-container/ Still have same problem, also note that all the other frameworks work in the CUDA 10 container.
cc/ @mt-jones per our LinkedIn convo as well
After I launch container and goto demo, I get:
import numpy as np
import pandas as pd
import cudf
import os
from sklearn.neighbors import NearestNeighbors as skKNN
from cuml.neighbors.nearest_neighbors import NearestNeighbors as cumlKNN
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-1-05ab86581f23> in <module>
1 import numpy as np
2 import pandas as pd
----> 3 import cudf
4 import os
5
/usr/local/lib/python3.6/dist-packages/cudf/__init__.py in <module>
1 # Copyright (c) 2018, NVIDIA CORPORATION.
2
----> 3 from cudf import dataframe
4 from cudf import datasets
5 from cudf.dataframe import DataFrame, from_pandas, merge
/usr/local/lib/python3.6/dist-packages/cudf/dataframe/__init__.py in <module>
----> 1 from cudf.dataframe import (buffer, dataframe, series,
2 index, numerical, datetime, categorical, string)
3
4 from cudf.dataframe.dataframe import DataFrame, from_pandas, merge
5 from cudf.dataframe.index import (Index, GenericIndex,
/usr/local/lib/python3.6/dist-packages/cudf/dataframe/buffer.py in <module>
1 import numpy as np
2
----> 3 from librmm_cffi import librmm as rmm
4
5 from cudf.utils import cudautils, utils
/usr/local/lib/python3.6/dist-packages/librmm_cffi/__init__.py in <module>
43 return path
44
---> 45 librmm_api = ffi.dlopen(_get_lib_name())
46 librmm = _RMMWrapper(ffi, librmm_api)
47
OSError: cannot load library 'librmm.so': librmm.so: cannot open shared object file: No such file or directory@kkraus14 ->
Time Stamp 2019-04-01 13:24:42.697640
Hardware Information Machine : x86_64 CPU Name : skylake Number of accessible CPU cores : 12 Listed accessible CPUs cores : 0-11 CFS restrictions : None
OS Information Platform : Linux-4.18.0-15-generic-x86_64-with-debian-buster-sid Release : 4.18.0-15-generic System Name : Linux Version : #16~18.04.1-Ubuntu SMP Thu Feb 7 14:06:04 UTC 2019 OS specific info : debianbuster/sid glibc info : glibc 2.10
Python Information Python Compiler : GCC 7.3.0 Python Implementation : CPython Python Version : 3.7.1 Python Locale : en_US UTF-8
LLVM information LLVM version : 6.0.0
CUDA Information Found 1 CUDA devices id 0 b'GeForce RTX 2080 Ti' [SUPPORTED] compute capability: 7.5 pci device id: 0 pci bus id: 1 Summary: 1/1 devices are supported CUDA driver version : 10010 CUDA libraries: Finding cublas ERROR: can't locate lib Finding cusparse ERROR: can't locate lib Finding cufft ERROR: can't locate lib Finding curand ERROR: can't locate lib Finding nvvm ERROR: can't locate lib finding libdevice for compute_20... ERROR: can't open libdevice for compute_20 finding libdevice for compute_30... ERROR: can't open libdevice for compute_30 finding libdevice for compute_35... ERROR: can't open libdevice for compute_35 finding libdevice for compute_50... ERROR: can't open libdevice for compute_50
ROC Information ROC available : False Error initialising ROC due to : No ROC toolchains found. No HSA Agents found, encountered exception when searching: Error at driver init: NUMBA_HSA_DRIVER /opt/rocm/lib/libhsa-runtime64.so is not a valid file path. Note it must be a filepath of the .so/.dll/.dylib or the driver:
SVML Information SVML state, config.USING_SVML : False SVML library found and loaded : False llvmlite using SVML patched LLVM : True SVML operational : False
Threading Layer Information TBB Threading layer available : False +--> Disabled due to : Unknown import problem. OpenMP Threading layer available : True Workqueue Threading layer available : True
Numba Environment Variable Information None set.
Conda Information conda_build_version : 3.17.6 conda_env_version : 4.5.12 platform : linux-64 python_version : 3.7.1.final.0 root_writable : False
`
How do you set up PyCharm to use rapids?