search

koursaros-ai / nboost

NBoost is a scalable, search-api-boosting platform for deploying transformer models to improve the relevance of search results on different platforms (i.e. Elasticsearch)
Apache License 2.0
675 stars 69 forks source link

Running Nboost on GPU #86

Closed SagarPalyal closed 4 years ago

SagarPalyal commented 4 years ago

I have set up nboost and cuda toolkit on gpu machine but still nboost is running on cpu. Please let me know what extra configurations I need to do, so that nboost will consume gpu.

ngautam0 commented 4 years ago

Hi

There are no extra steps required.

Please run nvidia-smi command to check the GPUs driver. The output should be like this:

Tue Jul 28 12:37:01 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.57       Driver Version: 450.57       CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 106...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   65C    P2    24W /  N/A |   1761MiB /  6078MiB |      5%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      3518      G   /usr/lib/xorg/Xorg                511MiB |
|    0   N/A  N/A      4256      G   /usr/bin/gnome-shell              206MiB |
|    0   N/A  N/A      5796      G   ...AAAAAAAAA= --shared-files      305MiB |
|    0   N/A  N/A      8931      G   ...AAAAEAAAAIAAAAAAAAAAgAAAA       72MiB |
|    0   N/A  N/A     10737      C   ...ix/n-boost/env/bin/python      613MiB |
|    0   N/A  N/A     11492      G   ...oken=16417047233074432355       43MiB |
+-----------------------------------------------------------------------------+

I followed below steps:

  1. Install TensorFlow with GPU - https://www.tensorflow.org/install/gpu
  2. Run nboost using nboost command or from plugins.

You should be able to see something like this:

2020-07-28 12:27:55.417631: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
I:nboost/pt-tinybert-msmarco:[tra:__i: 23]:Loading from checkpoint nboost/pt-tinybert-msmarco
I:nboost/pt-tinybert-msmarco:[tra:__i: 29]:RUNNING ON CUDA
SagarPalyal commented 4 years ago

Hi gautam, Thanks for response. Yes I tried installing Pytorch in my case with gpu and it worked without any extra configuration.