Fuji GPU running error.

smatsumori commented 6 years ago

(tensorflow3) smatsumori@fuji:~/Workspace/nec$ python train.py
2018-02-21 00:25:05.957002: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-02-21 00:25:05.957029: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-02-21 00:25:05.957050: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-02-21 00:25:05.957055: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-02-21 00:25:05.957059: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2018-02-21 00:25:06.200697: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties: 
name: TITAN X (Pascal)
major: 6 minor: 1 memoryClockRate (GHz) 1.531
pciBusID 0000:02:00.0
Total memory: 11.90GiB
Free memory: 11.75GiB
2018-02-21 00:25:06.200725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 
2018-02-21 00:25:06.200730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y 
2018-02-21 00:25:06.200736: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id: 0000:02:00.0)
Traceback (most recent call last):
  File "/home/smatsumori/.virtualenvs/tensorflow3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
    return fn(*args)
  File "/home/smatsumori/.virtualenvs/tensorflow3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1297, in _run_fn
    self._extend_graph()
  File "/home/smatsumori/.virtualenvs/tensorflow3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1358, in _extend_graph
    self._session, graph_def.SerializeToString(), status)
  File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/home/smatsumori/.virtualenvs/tensorflow3/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation 'MEMORY_MODULE_2/VALUES': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
Colocation Debug Info:
Colocation group had the following types and devices: 
RefSwitch: GPU CPU 
Assign: GPU CPU 
StridedSliceAssign: CPU 
Identity: GPU CPU 
VariableV2: GPU CPU 
     [[Node: MEMORY_MODULE_2/VALUES = VariableV2[container="", dtype=DT_FLOAT, shape=[100000], shared_name="", _device="/device:GPU:0"]()]]

smatsumori commented 6 years ago

CUDA library libcupti.so.9.1. LD_LIBRARY_PATH: /usr/local/cuda/lib64: 2018-02-21 01:07:54.329530: F ./tensorflow/streamexecutor/lib/statusor.h:212] Non-OK-status: status status: Failed precondition: could not dlopen DSO: libcupti.so.9.1; dlerror: libcupti.so.9.1: cannot open shared object file: No such file or directory

smatsumori commented 6 years ago

ADD

/usr/local/cuda/extras/CUPTI/lib64/ to LD_LIBRARY_PATH

imai-laboratory / nec

Fuji GPU running error. #9