sigstore / rekor

Software Supply Chain Transparency Log
https://sigstore.dev
Apache License 2.0
903 stars 164 forks source link

Fix index scripts #2203

Closed cmurphy closed 3 months ago

cmurphy commented 3 months ago

Fix the cleanup-index script for indexes with capitalized keys (e.g., workload identities for GitHub repositories with capital letters in the name).

Add a new script alongside the backfill script to copy indices directly from Redis to MySQL. This is created as a separate script from the backfill script because they are necessary for different purposes: backfill is appropriate for when there is no data in any search index backend and the only source of data is the Rekor API, where copy is apprpriate when there is index data in the Redis database to pull directly from. This enables the script to copy data that the backfill script would not be able to recreate because the data needed to generate the indexes is not always persisted to the Rekor API.

Summary

Release Note

Documentation

codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 0% with 94 lines in your changes missing coverage. Please review.

Project coverage is 43.02%. Comparing base (488eb97) to head (b486afa). Report is 173 commits behind head on main.

Files Patch % Lines
cmd/copy-index/main.go 0.00% 92 Missing :warning:
cmd/cleanup-index/main.go 0.00% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2203 +/- ## =========================================== - Coverage 66.46% 43.02% -23.44% =========================================== Files 92 189 +97 Lines 9258 19462 +10204 =========================================== + Hits 6153 8374 +2221 - Misses 2359 10332 +7973 - Partials 746 756 +10 ``` | [Flag](https://app.codecov.io/gh/sigstore/rekor/pull/2203/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | Coverage Δ | | |---|---|---| | [e2etests](https://app.codecov.io/gh/sigstore/rekor/pull/2203/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | `?` | | | [unittests](https://app.codecov.io/gh/sigstore/rekor/pull/2203/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore) | `43.02% <0.00%> (-4.66%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sigstore#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.