sigstore / rekor

Software Supply Chain Transparency Log
https://sigstore.dev
Apache License 2.0
884 stars 163 forks source link

Better error handling in backfill script #2148

Closed cmurphy closed 3 months ago

cmurphy commented 3 months ago

Fix error handling for backfill

Abort backfill after limited errors

If there is a persistent problem with one of the network connections, continuing to attempt the insertion is a waste of time, and if the list of errors gets too long before the script finishes or is interrupted, it becomes impossible to tell when it started failing and therefore where to restart the script from. This change sets a limit on the maximum number of failures to tolerate before exiting the script. Parsing errors are related to the format of the data and don't have implications on the overall success of the script on other data, so no change is implemented for parsing errors.

Summary

Release Note

Documentation

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 48.44%. Comparing base (488eb97) to head (da72164). Report is 143 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2148 +/- ## =========================================== - Coverage 66.46% 48.44% -18.02% =========================================== Files 92 80 -12 Lines 9258 6659 -2599 =========================================== - Hits 6153 3226 -2927 - Misses 2359 3035 +676 + Partials 746 398 -348 ``` | [Flag](https://app.codecov.io/gh/sigstore/rekor/pull/2148/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | Coverage Δ | | |---|---|---| | [e2etests](https://app.codecov.io/gh/sigstore/rekor/pull/2148/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | `?` | | | [unittests](https://app.codecov.io/gh/sigstore/rekor/pull/2148/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | `48.44% <ø> (+0.76%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.