Profiling Caffe2 Distributed Training

[Hiroki11x]

I'd like to profile caffe2 distributed training resent50

First of all I have to know how faster

• caffe2 fp32 single gpu training

then, I will compare

• caffe2 fp16 single gpu training
• caffe2 fp32 single node (multi gpus) training
• caffe2 fp16 single node (multi gpus) training
• caffe2 fp32 multi node (multi gpus) training
• caffe2 fp16 multi node (multi gpus) training

Before that, I should research someone's benchmarks about resnet50 training

[Hiroki11x]

use cudnn7 https://github.com/caffe2/caffe2/issues/1025

[Hiroki11x]

refer https://github.com/caffe2/caffe2/issues/902#issuecomment-315677437

[Hiroki11x]

I note in private https://github.com/Hiroki11x/TeXFiles/blob/master/README.md