Multi-gpu training

[boboyyy]

Thank you for the paper and code. I tried multi-gpu using horovod for faster training, but results tend out to be much lower than single-gpu training results. Would you release a version of multi-gpu training?

[xbpeng]

We've never tried multi-gpu training before, so that likely wont be supported anytime soon.

[Robokan]

Curious. What procedure did you use to get horovod working? Did you something like what's shown here: https://horovod.readthedocs.io/en/stable/summary_include.html#install

I was thinking to work on this but if you already tried it I would be interested to hear what you discovered.