How to use multi-gpus for training?

[vozhuo]

I have 4 gpus and I noticed that libml/utils.py have some function (para_list, para_mean, etc) for parallel training, but it's seems not to be used. I'm not familiar with multi gpus training, how can I use these functions, should I modify mixmatch.py?

[david-berthelot]

Indeed it is not used, I wrote this function in case I needed to use multiple GPUs but since it was fast enough with a single one, I ended up not using them. That being said the change should be easy, something like:

logits = utils.para_cat(lambda x: classifier(x, training=True), images)

[vozhuo]

Thank you!