ydb-platform / nbs

Network Block Store
Apache License 2.0
52 stars 21 forks source link

[Blockstore] should add blobs to index in parallel while writing them #122

Closed SvartMetal closed 5 months ago

SvartMetal commented 8 months ago

Currently, WriteBlocks request consists of two consecutive stages:

This approach needs improvement. The idea is simple:

  1. Persisting 'unconfirmed blob' information in parallel during writing the blobs to Blob Storage
  2. Deferring or returning a retriable error for 'unconfirmed ranges' at this time
  3. Confirming the blobs by adding them to the index when it's known that all the blobs have been successfully written to Blob Storage
  4. Finally, cleaning up the 'unconfirmed blobs', previously persisted in step 1

The only subtlety is what to do with partially-written blobs. Suppose the tablet is restarted in the middle of writing data. In this case, some blobs may be partially written and cannot be recovered. Therefore, we need to confirm/recover these blobs at tablet startup. This recovery process is done using the TEvBlobStorage::TEvGet request with the 'restore' flag for each blob.

SvartMetal commented 8 months ago

The majority of the work has been completed, but there are still some tasks to be done. Must have:

Nice to have:

SvartMetal commented 5 months ago

Performance improvement on 128 KB workload, iodepth=40:

Screenshot 2024-04-04 at 21 25 37
qkrorlqr commented 5 months ago

Implemented and partially deployed. Works fine.