neondatabase / neon

Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, code-like database branching, and scale to zero.
https://neon.tech
Apache License 2.0
15.22k stars 444 forks source link

scrubber: allow restricting find_garbage to a partial tenant id prefix #9814

Closed arpad-m closed 1 day ago

arpad-m commented 2 days ago

Adds support to the find_garbage command to restrict itself to a partial tenant ID prefix, say a, and then it only traverses tenants with IDs starting with a. One can now pass the --tenant-id-prefix parameter.

That way, one can shard the find_garbage command and make it run in parallel.

The PR also does a change of how remote_storage first removes trailing /s, only to then add them in the listing function. It turns out that this isn't neccessary and it prevents the prefix functionality from working. S3 doesn't do this either.

github-actions[bot] commented 2 days ago

5517 tests run: 5291 passed, 0 failed, 226 skipped (full report)


Code coverage* (full report)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
f5f94ec0366b63fd2cbbe02edc2087dbd893d04d at 2024-11-20T05:37:23.152Z :recycle: