search

pytorch / data

A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.
BSD 3-Clause "New" or "Revised" License
1.08k stars 142 forks source link

Update state_dict() request at time 0 to include worker state #1260

Closed andrewkho closed 3 weeks ago

andrewkho commented 1 month ago

Please read through our contribution guide prior to creating your pull request.

When state_dict is requested before an iterator is created, we currently return an empty dict. This does not reflect a true initial state of the workers, and may break assumptions made by DCP.

This update will guarantee that anytime state_dict is requested, you will get a "full" state_dict that includes main state, and all worker initial states, including dataset/iterator states. We achieve this by including an ack signal from the workers which includes an initial state_dict. The main process now waits for these acks and stores the initial states for any subsequent state_dict requests, as well as for resetting the iterator.

Fixes #{issue number}

Changes

pytorch-bot[bot] commented 1 month ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/data/1260

Note: Links to docs will display an error until the docs builds have been completed.

:white_check_mark: No Failures

As of commit ed42b9f63e81804b4d8ee1a68f0b6a993244f904 with merge base 82918dd12281e69d39fa80bfeabfc02b1f93561c (image): :green_heart: Looks good so far! There are no failures yet. :green_heart:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot commented 1 month ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 month ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 month ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 month ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 month ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 month ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 month ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 4 weeks ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 4 weeks ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 4 weeks ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 3 weeks ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 3 weeks ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 3 weeks ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

gokulavasan commented 3 weeks ago

Two questions, rest lgtm! Also lint check is failing

facebook-github-bot commented 3 weeks ago

@andrewkho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.