Open mengxia1994 opened 5 years ago
Hi, I'm nobody but I think the 5 lines that have with tf.device('/cpu:0'): are the only lines computed by cpu, the other big part are computed by GPUs. If so it's normal that the speed doesn't change because it's a little part.
About Nvidia-smi I don't know.
with tf.device('/cpu:0'): change to: with tf.device('/gpu:0'):
I have refered to some similar issues but they do not help. I changed "/cpu:0" to "gpu:0", and run train.py, several lines of message show up like below, which doesn't show before the change.
Found device 0 with properties: name: Tesla V100-PCIE-32GB major: 7 minor: 0 memoryClockRate(GHz): 1.38 pciBusID: 0000:3b:00.0 totalMemory: 31.72GiB freeMemory: 29.18GiB 2019-07-03 10:39:13.197815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0 2019-07-03 10:39:13.199634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix: 2019-07-03 10:39:13.199660: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0 2019-07-03 10:39:13.199675: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0: N 2019-07-03 10:39:13.200644: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 28382 MB memory) -> physical GPU (device: 0, name: Tesla V100-PCIE-32GB, pci bus id: 0000:3b:00.0, compute capability: 7.0) 2019-07-03 10:39:13.203163: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x55a8136b0a40 executing computations on platform CUDA. Devices: 2019-07-03 10:39:13.203199: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): Tesla V100-PCIE-32GB, Compute Capability 7.0
it seems correct, however, the speed doesn't change and when i check nvidia-smi, i found no process. How could i do? i am using anaconda to manage my virtual environments, i checked and the tensorflow exists.