Open Saladino93 opened 5 years ago
Sorry for no reply. tf.distribute.Strategy also offers a solution for multiple GPU, which might be worth a shot.
There is a setting under tf.config that prints out device assignments as ops get scheduled. That could be useful for proving to yourself that it's using all the GPUs.
Hi all,
I the following network
`import tensorflow as tf
from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense, Dropout, LSTM, Bidirectional, TimeDistributed
from tensorflow.keras.utils import to_categorical, multi_gpu_model
import tensorflow_probability as tfp
.....
neural_net = tf.keras.Sequential([ (LSTM(num_units, activation = tf.nn.tanh, input_shape = (input_shape), return_sequences = True)), Dropout(0.2), Bidirectional(LSTM(num_units, activation = tf.nn.tanh)), Dropout(0.2), tfp.layers.DenseFlipout(84, activation = tf.nn.relu), tfp.layers.DenseFlipout(n_classes_binary, activation = tf.nn.softmax) ])`
and I am want to implement multi gpu support.
If I follow tf recommendation, to share weights on cpu, with
multi_neural_net = tf.keras.utils.multi_gpu_model(neural_net, gpus = gpu_n, cpu_relocation = True)
I get
`--------------------------------------------------------------------------- NameError Traceback (most recent call last)