sigstore / rekor

Software Supply Chain Transparency Log
https://sigstore.dev
Apache License 2.0
904 stars 164 forks source link

Unset DisableKeepalive for backfill HTTP client #2137

Closed cmurphy closed 5 months ago

cmurphy commented 6 months ago

Disabling Keep-Alive, as done by the default transport setting in the hashicorp cleanhttp package, seems to conflict with a network setting between the public good Rekor instances and the bastion and results in GET requests stalling or timing out after processing a few entries. This change adds an option to the rekor client to unset the DisableKeepalive setting and has the backfill script utilize that option. Other uses of the rekor client will see no change.

Summary

Release Note

Documentation

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.

Project coverage is 68.61%. Comparing base (488eb97) to head (33bd958). Report is 129 commits behind head on main.

Files Patch % Lines
pkg/client/options.go 0.00% 3 Missing :warning:
pkg/client/rekor_client.go 0.00% 1 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2137 +/- ## ========================================== + Coverage 66.46% 68.61% +2.15% ========================================== Files 92 92 Lines 9258 7293 -1965 ========================================== - Hits 6153 5004 -1149 + Misses 2359 1545 -814 + Partials 746 744 -2 ``` | [Flag](https://app.codecov.io/gh/sigstore/rekor/pull/2137/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | Coverage Δ | | |---|---|---| | [e2etests](https://app.codecov.io/gh/sigstore/rekor/pull/2137/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | `50.34% <0.00%> (+2.79%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/sigstore/rekor/pull/2137/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | `48.91% <0.00%> (+1.23%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

cmurphy commented 6 months ago

@haydentherapper I added another change, this was still hitting a rate limit after 1000 or so requests.