Closed cmurphy closed 3 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 48.44%. Comparing base (
488eb97
) to head (da72164
). Report is 143 commits behind head on main.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Fix error handling for backfill
Abort backfill after limited errors
If there is a persistent problem with one of the network connections, continuing to attempt the insertion is a waste of time, and if the list of errors gets too long before the script finishes or is interrupted, it becomes impossible to tell when it started failing and therefore where to restart the script from. This change sets a limit on the maximum number of failures to tolerate before exiting the script. Parsing errors are related to the format of the data and don't have implications on the overall success of the script on other data, so no change is implemented for parsing errors.
Summary
Release Note
Documentation