FoundationDB / fdb-record-layer

A record-oriented store built on FoundationDB
Apache License 2.0
570 stars 102 forks source link

Index Scrubbing: support cleanup when dangling index entries are expected #2765

Open jjezra opened 4 weeks ago

jjezra commented 4 weeks ago

On some databases, as it seems, dangling index entries are part of the normal flow. To support routine cleanup of these entries, we should add:

  1. OnlineIndexScrubber API argument to start from scratch (to avoid continuation of previous sessions)
  2. OnlineIndexScrubber API argument to limit the number of deletes per time unit (to avoid FDB flooding)

With the existing way to avoid warning, the index scrubbing can be used as a vacuum method to remove expected dangling index entries.