Disable find_unused_parameters by default

facebookresearch / ClassyVision

An end-to-end PyTorch framework for image and video classification

https://classyvision.ai

MIT License

1.59k stars 278 forks source link

Disable find_unused_parameters by default #707

Closed mannatsingh closed 3 years ago

mannatsingh commented 3 years ago

Summary: It is not worth it to have this enabled by default, since -

it can potentially slow things down
gradient checkpointing doesn't work
sharded ddp runs into issues
...

The only advantage was that users who did have unused parameters didn't have to worry about anything and things just worked. Such users will have to pass the following parameter in the run after the diff -

{
    "distributed": {"find_unused_parameters": true}
}

Differential Revision: D26653963

facebook-github-bot commented 3 years ago

This pull request was exported from Phabricator. Differential Revision: D26653963