Closed digirak closed 5 years ago
Are you using tensorflow-gpu
/ keras-gpu
? Because your framework simply doesn't seem to detect the GPUs. Which it should.
Joeri
One thing you can do to check if you installed your software stack correctly is to ssh into one of the nodes, open a Python interpreter and do:
from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())
Hi I just realized I hadn't installed keras-gpu. Thanks!
Describe the issue I have asked for two GPUs and the code clearly request so does the shell script. However I get the following error. 019-05-18 17:03:44.558783: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance. Traceback (most recent call last): File "/home/rnath/Code/DeepNet.py", line 62, in
conv1d=get_model()
File "/home/rnath/Code/DeepNet.py", line 58, in get_model
model = keras.utils.multi_gpu_model(model, gpus=2)
File "/home/rnath/miniconda3/lib/python3.7/site-packages/keras/utils/multi_gpu_utils.py", line 181, in multi_gpu_model
available_devices))
ValueError: To call
multi_gpu_model
withgpus=2
, we expect the following devices to be available: ['/cpu:0', '/gpu:0', '/gpu:1']. However this machine only has: ['/cpu:0', '/xla_cpu:0']. Try reducinggpus
.Context This is after the job has been assigned.
!/bin/bash
SBATCH --job-name=DeepSpec
SBATCH --time=24:10:00 # hh:mm:ss
SBATCH --output=output_val.txt
SBATCH --ntasks=3
SBATCH --gres=gpu:2
SBATCH --mem-per-cpu=5120 # 5GB
SBATCH --mail-user=rakesh.nath@uliege.be
SBATCH --mail-type=ALL
SBATCH --partition=gpu2080ti
SBATCH --comment=DeepLSpectra
python /home/rnath/Code/DeepNet.py