search

rapidsai / cuml

cuML - RAPIDS Machine Learning Library
https://docs.rapids.ai/api/cuml/stable/
Apache License 2.0
4.26k stars 535 forks source link

[QST] list of cuml-cpu available module #6074

Open busFred opened 2 months ago

busFred commented 2 months ago

Here is how I set up conda environment

$ conda create -n diss_cpu python=3.10
$ conda install -c rapidsai -c nvidia -c conda-forge cuml-cpu numpy-base=1.26.4

After I import cuml, here is the list of all available module, which is a subset of its gpu counterpart.

(diss_cpu) $ python
Python 3.10.14 (main, May  6 2024, 19:42:50) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cuml
>>> cuml.
cuml.Base(                    cuml.Lasso(                   cuml.Ridge(                   cuml.cluster                  cuml.linear_model             cuml.using_output_type(
cuml.CD(                      cuml.LinearRegression(        cuml.SGD(                     cuml.common                   cuml.manifold                 
cuml.ElasticNet(              cuml.LogisticRegression(      cuml.TruncatedSVD(            cuml.decomposition            cuml.prims                    
cuml.GlobalSettings()         cuml.PCA(                     cuml.UMAP(                    cuml.internals                cuml.set_global_output_type(  
cuml.HDBSCAN(                 cuml.QN(                      cuml.UniversalBase(           cuml.is_cuda_available()      cuml.solvers                  
>>> cuml.__version__
'24.08.00'

From the documentation, it seems NearestNeighbor is officially supported on cpu.

(diss_cpu) $ python
>>> import cuml.neighbors
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hungtien/anaconda3/envs/diss_cpu/lib/python3.10/site-packages/cuml/neighbors/__init__.py", line 19, in <module>
    from cuml.neighbors.nearest_neighbors import NearestNeighbors
ModuleNotFoundError: No module named 'cuml.neighbors.nearest_neighbors'

Could someone help me look into this?

busFred commented 2 months ago

I actually did a little troubleshoot on my end and i think the problem is that cuml-cpu would not load certain modules correctly once it detects cuda packages are installed. so we have some remote computing resources with very old gpu, even below cuml minimum cuda computability. However, those gpu can still run pytorch in gpu mode if cuda drivers are installed. so what i was trying to do was install cuml-cpu version while having pytorch-gpu at the same time. I guess this is a corner case that cuml-cpu didn't take into consideration during development

dantegd commented 1 month ago

Thanks for the report @busFred, that corner case indeed is something we did not think about. Have been working significantly on improvements to the are around cuml-cpu, and will be submitting significant improvements in the next few versions, will keep the issue open so we track this usecase as well.