search

jason9693 / MusicTransformer-tensorflow2.0

implementation of music transformer with tensorflow-2.0 (ICLR2019)
MIT License
352 stars 81 forks source link

Multi-GPU Programming #3

Closed lose4578 closed 5 years ago

lose4578 commented 5 years ago

Hello, thank you for your code contribution. Do you have any plans for multi-gpu parallel computing recently? I have encountered some problems when trying to change the code to parallel computing.

Traceback (most recent call last): File "train.py", line 72, in mt.compile(optimizer=opt, loss=callback.transformer_dist_train_loss) File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper result = method(self, *args, **kwargs) File "/opt/conda/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 263, in compile 'We currently do not support distribution strategy with a ' ValueError: We currently do not support distribution strategy with a Sequential model that is created without input_shape/input_dim set in its first layer or a subclassed model.

jason9693 commented 5 years ago

@lose4578 Thank you for interested in my repository. I have planned for multi-GPU training before, but current tf2.0 distribute strategy is slower than single GPU.

if you want to see how to distribute training, see dist_train.py

Thank you.