Synchronize loading of models when in distributed training

fastai / fastai_dev

fast.ai early development experiments

Apache License 2.0

641 stars 351 forks source link

Closed philtrade closed 5 years ago

philtrade commented 5 years ago

A port from a recent PR in the fastai/fastai repo (https://github.com/fastai/fastai/pull/2393)

When in distributed training (multiple gpus), synchronize the load() of the model with a barrier synchronizer.

Regenerated related notebooks.

review-notebook-app[bot] commented 5 years ago

Check out this pull request on

You'll be able to see Jupyter notebook diff and discuss changes. Powered by ReviewNB.

sgugger commented 5 years ago

Thanks!