Resolve problems introduced into processing state when using the --continue-on-error flag to skip errors
Update README with current flags and their usage
What's new?
Clean up resources after exceptions thrown while processing an object, in order to start with a clean state for processing the next object in the queue.
Documentation of --continue-on-error flag in README, other minor usage doc updates
How should this be tested?
I used a test set of 15 test objects to migrate, and deleted some datastreams from the F3 filesystem.
First test: no continue-on-error, two datastreams deleted
Expected: migration halts after first missing datastream
Actual = Expected
Second test: continue-on-error, datastreams in two consecutive objects deleted
Expected: migration skips two objects with missing datastreams, rest are processed
Actual = Expected
Third test: continue-on-error, datastreams in two non-consecutive objects deleted
Expected: migration skips first object with missing datastream, processes the next good objects, skips second object with missing datastream, processes the rest
Actual = Expected
Fourth test: continue-on-error, datastreams in every object in test set deleted
Expected: migration attempts and skips every object
Actual = Expected
Additional Notes:
This PR does not address implementation of the `--max-errors feature described in the JIRA ticket.
JIRA Ticket: https://jira.lyrasis.org/browse/FCREPO-3460
What does this Pull Request do?
Resolve problems introduced into processing state when using the
--continue-on-errorflag to skip errors Update README with current flags and their usageWhat's new?
Clean up resources after exceptions thrown while processing an object, in order to start with a clean state for processing the next object in the queue. Documentation of
--continue-on-errorflag in README, other minor usage doc updatesHow should this be tested?
I used a test set of 15 test objects to migrate, and deleted some datastreams from the F3 filesystem.
First test: no continue-on-error, two datastreams deleted Expected: migration halts after first missing datastream Actual = Expected
Second test: continue-on-error, datastreams in two consecutive objects deleted Expected: migration skips two objects with missing datastreams, rest are processed Actual = Expected
Third test: continue-on-error, datastreams in two non-consecutive objects deleted Expected: migration skips first object with missing datastream, processes the next good objects, skips second object with missing datastream, processes the rest Actual = Expected
Fourth test: continue-on-error, datastreams in every object in test set deleted Expected: migration attempts and skips every object Actual = Expected
Additional Notes:
This PR does not address implementation of the
`--max-errorsfeature described in the JIRA ticket.Interested parties
@awoods @dbernstein @pwinckles