search

rapidsai / cuml

cuML - RAPIDS Machine Learning Library
https://docs.rapids.ai/api/cuml/stable/
Apache License 2.0
4.26k stars 535 forks source link

[BUG] Jacobi solver is significantly slower than the full solver #1641

Open atarashansky opened 4 years ago

atarashansky commented 4 years ago

I'm running this notebook for PCA with the following parameters:

n_samples = 2**17
n_features = 3000
n_components = 150
whiten = False
svd_solver = "jacobi"
random_state = 23

The wall time for the below code is 7.05 seconds using the jacobi solver, and 4.94 seconds using the full solver. I tried fiddling with the tolerance and iterated power to increase the speed of jacobi solver, but I had to really up the tolerance (1e-2) to get jacobi to be only slightly faster than the full solver (4.59 seconds), at which point it isn't worth it.

%%time
pca_cuml = cuPCA(n_components=n_components,
                 svd_solver='jacobi',
                 whiten=whiten,
                 random_state=random_state)

result_cuml = pca_cuml.fit_transform(device_data)

The documentation says jacobi should be much faster, right?

Here is the output of conda list:

# packages in environment at /home/alec/anaconda3/envs/cuml:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       0_gnu    conda-forge
arrow-cpp                 0.15.0           py36h090bef1_2    conda-forge
attrs                     19.3.0                   pypi_0    pypi
backcall                  0.1.0                    pypi_0    pypi
bleach                    3.1.0                    pypi_0    pypi
bokeh                     1.4.0                    py36_0    conda-forge
boost-cpp                 1.70.0               h8e57a91_2    conda-forge
brotli                    1.0.7             he1b5a44_1000    conda-forge
bzip2                     1.0.8                h516909a_2    conda-forge
c-ares                    1.15.0            h516909a_1001    conda-forge
ca-certificates           2020.1.1                      0    anaconda
cairo                     1.16.0            hfb77d84_1002    conda-forge
certifi                   2019.11.28               py36_0    anaconda
cfitsio                   3.470                hb60a0a2_2    conda-forge
click                     7.0                        py_0    conda-forge
cloudpickle               1.2.2                      py_1    conda-forge
cudatoolkit               10.0.130                      0    anaconda
cudf                      0.12.0                   py36_0    rapidsai
cudnn                     7.6.0                cuda10.0_0    nvidia
cugraph                   0.12.0                   py36_0    rapidsai
cuml                      0.12.0          cuda10.0_py36_0    rapidsai
cupy                      7.1.1            py36he57b8b9_1    conda-forge
curl                      7.68.0               hf8cf82a_0    conda-forge
cuspatial                 0.12.0                   py36_0    rapidsai
cytoolz                   0.10.1           py36h516909a_0    conda-forge
dask                      2.10.1                     py_0    conda-forge
dask-core                 2.10.1                     py_0    conda-forge
dask-cuda                 0.12.0                   py36_0    rapidsai
dask-cudf                 0.12.0                   py36_0    rapidsai
dask-xgboost              0.2.0.dev28      cuda10.0py36_0    rapidsai
decorator                 4.4.1                    pypi_0    pypi
defusedxml                0.6.0                    pypi_0    pypi
distributed               2.10.0                     py_0    conda-forge
dlpack                    0.2                  he1b5a44_1    conda-forge
double-conversion         3.1.5                he1b5a44_2    conda-forge
entrypoints               0.3                      pypi_0    pypi
expat                     2.2.9                he1b5a44_2    conda-forge
fastavro                  0.22.9           py36h516909a_0    conda-forge
fastrlock                 0.4             py36he1b5a44_1000    conda-forge
fontconfig                2.13.1            h86ecdb6_1001    conda-forge
freetype                  2.10.0               he983fc9_1    conda-forge
freexl                    1.0.5             h14c3975_1002    conda-forge
fsspec                    0.6.2                      py_0    conda-forge
gdal                      2.4.4            py36h5f563d9_0    conda-forge
geos                      3.8.0                he1b5a44_0    conda-forge
geotiff                   1.5.1                h38872f0_8    conda-forge
gettext                   0.19.8.1          hc5be6a0_1002    conda-forge
gflags                    2.2.2             he1b5a44_1002    conda-forge
giflib                    5.1.7                h516909a_1    conda-forge
glib                      2.58.3          py36h6f030ca_1002    conda-forge
glog                      0.4.0                he1b5a44_1    conda-forge
grpc-cpp                  1.23.0               h18db393_0    conda-forge
hdf4                      4.2.13            hf30be14_1003    conda-forge
hdf5                      1.10.5          nompi_h3c11f04_1104    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
icu                       64.2                 he1b5a44_1    conda-forge
importlib-metadata        1.5.0                    pypi_0    pypi
ipykernel                 5.1.4                    pypi_0    pypi
ipython                   7.12.0                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                7.5.1                    pypi_0    pypi
jedi                      0.16.0                   pypi_0    pypi
jinja2                    2.11.1                     py_0    conda-forge
joblib                    0.14.1                     py_0    conda-forge
jpeg                      9c                h14c3975_1001    conda-forge
json-c                    0.13.1            h14c3975_1001    conda-forge
jsonschema                3.2.0                    pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            5.3.4                    pypi_0    pypi
jupyter-console           6.1.0                    pypi_0    pypi
jupyter-core              4.6.1                    pypi_0    pypi
kealib                    1.4.10            h58c409b_1005    conda-forge
krb5                      1.16.4               h2fd8d38_0    conda-forge
ld_impl_linux-64          2.33.1               h53a641e_7  
libblas                   3.8.0               14_openblas    conda-forge
libcblas                  3.8.0               14_openblas    conda-forge
libcudf                   0.12.0               cuda10.0_0    rapidsai
libcugraph                0.12.0               cuda10.0_0    rapidsai
libcuml                   0.12.0               cuda10.0_0    rapidsai
libcumlprims              0.12                 cuda10.0_9    nvidia
libcurl                   7.68.0               hda55be3_0    conda-forge
libcuspatial              0.12.0               cuda10.0_0    rapidsai
libdap4                   3.20.4               hd3bb157_0    conda-forge
libedit                   3.1.20181209         hc058e9b_0  
libevent                  2.1.10               h72c5cf5_0    conda-forge
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 9.2.0                h24d8f2e_2    conda-forge
libgdal                   2.4.4                h2b6fda6_0    conda-forge
libgfortran-ng            7.3.0                hdf63c60_5    conda-forge
libgomp                   9.2.0                h24d8f2e_2    conda-forge
libhwloc                  2.1.0                h3c4fd83_0    conda-forge
libiconv                  1.15              h516909a_1005    conda-forge
libkml                    1.3.0             h4fcabce_1010    conda-forge
liblapack                 3.8.0               14_openblas    conda-forge
libllvm8                  8.0.1                hc9558a2_0    conda-forge
libnetcdf                 4.7.3           nompi_h9f9fd6a_101    conda-forge
libnvstrings              0.12.0               cuda10.0_0    rapidsai
libopenblas               0.3.7                h5ec1e0e_6    conda-forge
libpng                    1.6.37               hed695b0_0    conda-forge
libpq                     12.1                 hd9ab2ff_0    conda-forge
libprotobuf               3.8.0                h8b12597_0    conda-forge
librmm                    0.12.0               cuda10.0_0    rapidsai
libspatialite             4.3.0a            ha48a99a_1034    conda-forge
libssh2                   1.8.2                h22169c7_2    conda-forge
libstdcxx-ng              9.1.0                hdf63c60_0  
libtiff                   4.1.0                hfc65ed5_0    conda-forge
libuuid                   2.32.1            h14c3975_1000    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxgboost                1.0.0dev.rapidsai0.12      cuda10.0_1    rapidsai
libxml2                   2.9.10               hee79883_0    conda-forge
llvmlite                  0.30.0           py36h8b12597_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.8.3             he1b5a44_1001    conda-forge
markupsafe                1.1.1            py36h516909a_0    conda-forge
mistune                   0.8.4                    pypi_0    pypi
msgpack-python            0.6.2            py36hc9558a2_0    conda-forge
nbconvert                 5.6.1                    pypi_0    pypi
nbformat                  5.0.4                    pypi_0    pypi
nccl                      2.4.6.1              cuda10.0_0    nvidia
ncurses                   6.1                  he6710b0_1  
notebook                  6.0.3                    pypi_0    pypi
numba                     0.46.0           py36hb3f55d8_1    conda-forge
numpy                     1.18.1           py36h95a1406_0    conda-forge
nvstrings                 0.12.0                   py36_0    rapidsai
olefile                   0.46                       py_0    conda-forge
openjpeg                  2.3.1                h981e76c_3    conda-forge
openssl                   1.1.1                h7b6447c_0    anaconda
packaging                 20.1                       py_0    conda-forge
pandas                    0.25.3           py36hb3f55d8_0    conda-forge
pandocfilters             1.4.2                    pypi_0    pypi
parquet-cpp               1.5.1                         2    conda-forge
parso                     0.6.1                    pypi_0    pypi
partd                     1.1.0                      py_0    conda-forge
pcre                      8.43                 he1b5a44_0    conda-forge
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    7.0.0            py36hefe7db6_0    conda-forge
pip                       20.0.2                   py36_1  
pixman                    0.38.0            h516909a_1003    conda-forge
poppler                   0.67.0               h14e79db_8    conda-forge
poppler-data              0.4.9                         1    conda-forge
postgresql                12.1                 hc63931a_0    conda-forge
proj                      6.3.0                hc80f0dc_0    conda-forge
prometheus-client         0.7.1                    pypi_0    pypi
prompt-toolkit            3.0.3                    pypi_0    pypi
psutil                    5.6.7            py36h516909a_0    conda-forge
pthread-stubs             0.4               h14c3975_1001    conda-forge
ptyprocess                0.6.0                    pypi_0    pypi
py-xgboost                1.0.0dev.rapidsai0.12  cuda10.0py36_1    rapidsai
pyarrow                   0.15.0           py36h8b68381_1    conda-forge
pygments                  2.5.2                    pypi_0    pypi
pynvml                    8.0.4                      py_0    conda-forge
pyparsing                 2.4.6                      py_0    conda-forge
pyrsistent                0.15.7                   pypi_0    pypi
python                    3.6.7             h357f687_1006    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
pytz                      2019.3                     py_0    conda-forge
pyyaml                    5.3              py36h516909a_0    conda-forge
pyzmq                     18.1.1                   pypi_0    pypi
qtconsole                 4.6.0                    pypi_0    pypi
rapids                    0.12.0          cuda10.0_py36_1    rapidsai
rapids-xgboost            0.12.0          cuda10.0_py36_1    rapidsai
re2                       2020.01.01           he1b5a44_0    conda-forge
readline                  8.0                  hf8c457e_0    conda-forge
rmm                       0.12.0                   py36_0    rapidsai
scikit-learn              0.22.1           py36hcdab131_1    conda-forge
scipy                     1.4.1            py36h921218d_0    conda-forge
send2trash                1.5.0                    pypi_0    pypi
setuptools                45.1.0                   py36_0  
six                       1.14.0                   py36_0    conda-forge
snappy                    1.1.8                he1b5a44_1    conda-forge
sortedcontainers          2.1.0                      py_0    conda-forge
sqlite                    3.31.1               h7b6447c_0  
tblib                     1.6.0                      py_0    conda-forge
terminado                 0.8.3                    pypi_0    pypi
testpath                  0.4.4                    pypi_0    pypi
thrift-cpp                0.12.0            hf3afdfd_1004    conda-forge
tk                        8.6.10               hed695b0_0    conda-forge
toolz                     0.10.0                     py_0    conda-forge
tornado                   6.0.3            py36h516909a_0    conda-forge
traitlets                 4.3.3                    pypi_0    pypi
tzcode                    2019a             h516909a_1002    conda-forge
ucx                       1.7.0+g9d06c3a       cuda10.0_0    rapidsai
ucx-py                    0.12.0+g9d06c3a          py36_0    rapidsai
uriparser                 0.9.3                he1b5a44_1    conda-forge
wcwidth                   0.1.8                    pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
wheel                     0.34.2                   py36_0  
widgetsnbextension        3.5.1                    pypi_0    pypi
xerces-c                  3.2.2             h8412b87_1004    conda-forge
xgboost                   1.0.0dev.rapidsai0.12  cuda10.0py36_1    rapidsai
xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
xorg-libice               1.0.10               h516909a_0    conda-forge
xorg-libsm                1.2.3             h84519dc_1000    conda-forge
xorg-libx11               1.6.9                h516909a_0    conda-forge
xorg-libxau               1.0.9                h14c3975_0    conda-forge
xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
xorg-libxext              1.3.4                h516909a_0    conda-forge
xorg-libxrender           0.9.10            h516909a_1002    conda-forge
xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
xorg-xproto               7.0.31            h14c3975_1007    conda-forge
xz                        5.2.4                h14c3975_4  
yaml                      0.2.2                h516909a_1    conda-forge
zict                      1.0.0                      py_0    conda-forge
zipp                      2.1.0                    pypi_0    pypi
zlib                      1.2.11               h7b6447c_3  
zstd                      1.4.3                h3b9ef0a_0    conda-forge
divyegala commented 4 years ago

@atarashansky could you tell me what dataset you are using so I could try to repro this?