Remove syncing logic from train pipeline and use single pipeline in DLRM - Githubissues

facebookresearch / dlrm

An implementation of a deep learning recommendation model (DLRM)

MIT License

3.72k stars 825 forks source link

Remove syncing logic from train pipeline and use single pipeline in DLRM #339

Closed joshuadeng closed 1 year ago

joshuadeng commented 1 year ago

Summary: This diff removes the need for syncing between multiple TrainPipelines for the same model, used for the DLRM case. We enable the execute all batches mode in the train pipeline and exhaust each iterator before switching between train/val/test

Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040

facebook-github-bot commented 1 year ago

This pull request was exported from Phabricator. Differential Revision: D43550040