m3db / proposal

M3 Design Documents
Apache License 2.0
18 stars 1 forks source link

M3DB Index Out Of Order Writes support #16

Open robskillington opened 5 years ago

robskillington commented 5 years ago

Implementation pending, algorithm:

  1. During cold flush, for any series added to the volume, add to an index segment the series (load existing "cold writes index segment" from disk into a new builder to update it if already present)
  2. Send the cold index block the new updated segment, along with a number of series that were just inserted
  3. For loop over all existing in memory cold block documents, checking if it is in the cold writes segment just persisted to disk, if so then write a new version of the in memory cold block segment without the ones that also exist in the cold writes segment from disk