DDP Backend Support

[lorr1]

Description of the problems or issues

• Added support to specify the DDP backend between NCCL and GLOO. GLOO supports sparse tensors (common for embeddings) while NCCL does not.
• DDP requires that all modules have learnable parameters. When DDP was called on each module separately, an error would be thrown if one module had no learnable params.

Description of the proposed changes

• Add a distributed_backend flag to model_config. This could be moved to learner_config if desired.
• Added a check to pass on calling DDP on modules without any requires_grad param.

Test plan

• Updated parse arg tests to add the distributed backend config.
• DDP isn't currently in the tests, but I did test out locally running a DDP model with the gloo backend successfully.

[codecov[bot]]

Codecov Report

Merging #86 (16b2490) into master (7744916) will decrease coverage by 0.09%. The diff coverage is 25.00%.

@@            Coverage Diff             @@
##           master      #86      +/-   ##
==========================================
- Coverage   87.83%   87.73%   -0.10%     
==========================================
  Files          38       38              
  Lines        1742     1745       +3     
  Branches      374      375       +1     
==========================================
+ Hits         1530     1531       +1     
- Misses        112      114       +2     
  Partials      100      100              

Flag	Coverage Δ
unittests	87.73% <25.00%> (-0.10%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/emmental/meta.py	80.00% <0.00%> (ø)
src/emmental/model.py	81.37% <0.00%> (-0.57%)	:arrow_down:
src/emmental/utils/parse_args.py	99.26% <100.00%> (+<0.01%)	:arrow_up: