Closed lose4578 closed 5 years ago
@lose4578 Thank you for interested in my repository. I have planned for multi-GPU training before, but current tf2.0 distribute strategy is slower than single GPU.
if you want to see how to distribute training, see dist_train.py
Thank you.
Hello, thank you for your code contribution. Do you have any plans for multi-gpu parallel computing recently? I have encountered some problems when trying to change the code to parallel computing.
Traceback (most recent call last): File "train.py", line 72, in
mt.compile(optimizer=opt, loss=callback.transformer_dist_train_loss)
File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
result = method(self, *args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 263, in compile
'We currently do not support distribution strategy with a '
ValueError: We currently do not support distribution strategy with a
Sequential
model that is created withoutinput_shape
/input_dim
set in its first layer or a subclassed model.