Advanced Error Handling

[aclavio]

Capture and handle issues that can occur due to intermittent connectivity issues, or when items fail as part of a batch and should be retired individually.

• expire uris in the driver's in-progress queue after a time threshold
  • allow them to be rescheduled
  • allow time threshold to be configured
• capture errors at the batch level and individual failures when fetching task results
  • log errors
  • retry all failed docs x times
  • add them to a retry task of size 1
  • increment retry count until threshold is reached
  • log when threshold reached