How to install GPU version from source?

[lingvisa]

My Linux has Cuda 10.1 installed and 4 Tesla GPU. I follow the spacy instruction to install from from source, and then do:

spacy.prefer_gpu()
nlp = spacy.load('zh_core_web_lg')
docs = nlp.pipe(texts, disable=['ner', 'parser'], n_process=-1)

My test shows that prefer_gpu doesn't speed up at all on 60MB of text. Is there anything special to configure when installing spacy from source code?

[adrianeboyd]

All you need is to have cupy-cuda101 (or whatever the correct CUDA version is) installed in the same environment when you run spacy. How spacy was compiled or installed doesn't make a difference. If you use spacy.require_gpu() instead of prefer_gpu, you'll get an error if it can't find the GPU so you'll see clearly whether it's using the GPU.

Because of how cupy can be configured, it can only use one GPU at a time, so you'll need completely separate scripts to take advantage of more than one GPU. Use use_gpu from spacy.util to set a particular GPU for a particular script.

I'd recommend against trying to use multiprocessing + GPU, see #5507. We still need to improve the warnings / behavior around this since it isn't clear that it's not going to work well.

[lingvisa]

I switched to spacy.require_gpu, and it reports this error:

 spacy.require_gpu()
  File "/root/.venv/wbnlu/lib/python3.6/site-packages/thinc/neural/util.py", line 87, in require_gpu
    raise ValueError("GPU is not accessible. Was the library installed correctly?")
ValueError: GPU is not accessible. Was the library installed correctly?

Then I did: pip install cupy-cuda101

The message shows that cupy was successfully installed. But then it reported the same message. My GPU info: nvidia-smi Tue Jul 21 08:59:59 2020

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.87.01    Driver Version: 418.87.01    CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  On   | 00000000:00:0D.0 Off |                    0 |
| N/A   34C    P0    25W / 250W |      0MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla P100-PCIE...  On   | 00000000:00:0E.0 Off |                    0 |
| N/A   31C    P0    25W / 250W |      0MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla P100-PCIE...  On   | 00000000:00:0F.0 Off |                    0 |
| N/A   30C    P0    27W / 250W |      0MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla P100-PCIE...  On   | 00000000:00:10.0 Off |                    0 |
| N/A   65C    P0   170W / 250W |  15599MiB / 16280MiB |     98%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

Any other commands I can detect GPU configuration further?

[adrianeboyd]

You can try a simple cupy example:

import cupy
a = cupy.zeros((5, 5))

Or try the code that selects the GPU directly:

https://github.com/explosion/spaCy/blob/a8978ca285fa7ebf0867f54723a6ba5569b1c156/spacy/util.py#L719-L730

[lingvisa]

It looks like Cupy is not installed correctly, which doesn't match the correct cuda version. I was in a ipython terminal and tried to import 'cupy':

In [1]: import cupy                                                                                                                                                                                                       
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
~/.venv/wbnlu/lib/python3.6/site-packages/cupy/__init__.py in <module>
     20                                 message='can\'t resolve package from __spec__')
---> 21         from cupy import core  # NOQA
     22 except ImportError:

~/.venv/wbnlu/lib/python3.6/site-packages/cupy/core/__init__.py in <module>
----> 1 from cupy.core import core  # NOQA
      2 from cupy.core import internal  # NOQA
      3 

ImportError: libcublas.so.10: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
<ipython-input-1-329ec5cf1bc8> in <module>
----> 1 import cupy

~/.venv/wbnlu/lib/python3.6/site-packages/cupy/__init__.py in <module>
     40 original error: {}'''.format(exc_info[1]))  # NOQA
     41 
---> 42     six.reraise(ImportError, ImportError(msg), exc_info[2])
     43 
     44 

~/.venv/wbnlu/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    700                 value = tp()
    701             if value.__traceback__ is not tb:
--> 702                 raise value.with_traceback(tb)
    703             raise value
    704         finally:

~/.venv/wbnlu/lib/python3.6/site-packages/cupy/__init__.py in <module>
     19         warnings.filterwarnings('ignore', category=ImportWarning,
     20                                 message='can\'t resolve package from __spec__')
---> 21         from cupy import core  # NOQA
     22 except ImportError:
     23     # core is a c-extension module.

~/.venv/wbnlu/lib/python3.6/site-packages/cupy/core/__init__.py in <module>
----> 1 from cupy.core import core  # NOQA
      2 from cupy.core import internal  # NOQA
      3 
      4 
      5 # import class and function

ImportError: CuPy is not correctly installed.

If you are using wheel distribution (cupy-cudaXX), make sure that the version of CuPy you installed matches with the version of CUDA on your host.
Also, confirm that only one CuPy package is installed:
  $ pip freeze

If you are building CuPy from source, please check your environment, uninstall CuPy and reinstall it with:
  $ pip install cupy --no-cache-dir -vvvv

Check the Installation Guide for details:
  https://docs-cupy.chainer.org/en/latest/install.html

original error: libcublas.so.10: cannot open shared object file: No such file or directory

[adrianeboyd]

Hmm, something must be wrong with either CUDA or cupy. If you're not confident that your CUDA installation is correct, I'd run the basic CUDA tests from the CUDA installation docs, and then follow the instructions in the cupy warning above if it turns out to be a cupy issue.

[github-actions[bot]]

This issue has been automatically closed because it was answered and there was no follow-up discussion.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.