search

mlcommons / training_policies

Issues related to MLPerf™ training policies, including rules and suggested changes
https://mlcommons.org/en/groups/training
Apache License 2.0
92 stars 66 forks source link

Whether Distributed Optimizer wrapper meets the requirements? #302

Open shjwudp opened 4 years ago

shjwudp commented 4 years ago

Hi, there. I want to ask whether the current rules allow the use of distributed optimizer wrappers like hvd.DistributedOptimizer ?

I see that we have a list of optimizers allowed by the competition. But maybe the distributed optimizer wrapper is a special case? My reasons as follow:

  1. For most reference implementations, the optimizer packaging has no special effect on the algorithm, such as Mask-RCNN + torch.optim.SGD.
  2. Distributed training communication framework is an important part of training. Many companies are also studying the efficiency optimization of the framework. Supporting distributed training framework wrapper is conducive to promoting the technical exchange of distributed communication.

Thanks

bitfort commented 4 years ago

SWG:

AI(Everyone who wants) Weigh in on this.