GPU not used in training?

[AlexTS1980]

I set

os.environ["CUDA_VISIBLE_DEVICES"]="0"
GPU_COUNT = 1

but when I check nvidia-smi, no jobs are reported. Why is this happening?

[hj3yoo]

Are you using GPU-enabled Tensorflow?

1008:

if you run 'pip install -r requirements.txt', you will install a tensorflow without gpu, change the requirements.txt, replace tensorflow>=1.3.0 as tensorflow-gpu>=1.3.0

511

[cvKDean]

I tried to replace "tensorflow>=1.3.0" with "tensorflow-gpu>=1.3.0" in the requirements.txt file. But when I try training the model I encounter this error:

OSError                                   Traceback (most recent call last)
b:\bardeloza_files\anaconda3\envs\thesis_gpu\lib\site-packages\tensorflow\python\platform\self_check.py in preload_check()
     74         try:
---> 75           ctypes.WinDLL(build_info.cudart_dll_name)
     76         except OSError:

b:\bardeloza_files\anaconda3\envs\thesis_gpu\lib\ctypes\__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
    347         if handle is None:
--> 348             self._handle = _dlopen(self._name, mode)
    349         else:

OSError: [WinError 126] The specified module could not be found

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
<ipython-input-1-410b9a34c864> in <module>
     20 
     21 from mrcnn.evaluate import build_coco_results, evaluate_coco
---> 22 from mrcnn.dataset import MappingChallengeDataset
     23 
     24 import zipfile

B:\bardeloza_files\crowdai-mask-rcnn\mrcnn\dataset.py in <module>
----> 1 from mrcnn import utils
      2 import numpy as np
      3 
      4 from pycocotools.coco import COCO
      5 from pycocotools.cocoeval import COCOeval

B:\bardeloza_files\crowdai-mask-rcnn\mrcnn\utils.py in <module>
     13 import random
     14 import numpy as np
---> 15 import tensorflow as tf
     16 import scipy
     17 import skimage.color

b:\bardeloza_files\anaconda3\envs\thesis_gpu\lib\site-packages\tensorflow\__init__.py in <module>
     22 
     23 # pylint: disable=g-bad-import-order
---> 24 from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
     25 # pylint: disable=wildcard-import
     26 from tensorflow.tools.api.generator.api import *  # pylint: disable=redefined-builtin

b:\bardeloza_files\anaconda3\envs\thesis_gpu\lib\site-packages\tensorflow\python\__init__.py in <module>
     47 import numpy as np
     48 
---> 49 from tensorflow.python import pywrap_tensorflow
     50 
     51 # Protocol buffers

b:\bardeloza_files\anaconda3\envs\thesis_gpu\lib\site-packages\tensorflow\python\pywrap_tensorflow.py in <module>
     28 # Perform pre-load sanity checks in order to produce a more actionable error
     29 # than we get from an error during SWIG import.
---> 30 self_check.preload_check()
     31 
     32 # pylint: disable=wildcard-import,g-import-not-at-top,unused-import,line-too-long

b:\bardeloza_files\anaconda3\envs\thesis_gpu\lib\site-packages\tensorflow\python\platform\self_check.py in preload_check()
     80               "environment variable. Download and install CUDA %s from "
     81               "this URL: https://developer.nvidia.com/cuda-toolkit"
---> 82               % (build_info.cudart_dll_name, build_info.cuda_version_number))
     83 
     84       if hasattr(build_info, "cudnn_dll_name") and hasattr(

ImportError: Could not find 'cudart64_90.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable. Download and install CUDA 9.0 from this URL: https://developer.nvidia.com/cuda-toolkit

I realize that using tensorflow-gpu only does not install CUDA. Is there a way to install the necessary CUDA dependencies via pip?

[monocongo]

1. Install CUDA:

   $ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
   $ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
   $ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
   $ sudo add-apt-repository "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /"
   $ sudo apt-get update
   $ sudo apt-get -y install cuda

2. Install cuDNN:
   1. Login to the NVIDIA Developer Network
   2. Download the cuDNN Runtime Library for Ubuntu18.04

      $ sudo apt install ./libcudnn7_7.6.5.32-1+cuda10.2_amd64.deb