search

eBay / HomeObject

Replicated BLOB Store built upon HomeStore
Apache License 2.0
6 stars 15 forks source link

implement chunk tracker #177

Closed JacksonYao287 closed 6 months ago

JacksonYao287 commented 6 months ago

There will always be a very small window after HO get the block address and before sending read to the underlying data service, the chunk got selected by GC, moved the blob, returned to heap chunk selector, select by a new shard and a new blob is written on the same block address. This is very unlikely to happen, but conceptually possible, and once it happens, there will be data mismatch. It can be solved by read verification (cross check the blob_id, shard_id from the key) with the data portion blob header. If the blob is rewritten to other shards, it will report mismatch and we should fail this read and let the gateway to retry. Retry will be successful as it will arrive on a new chunk.

xiaoxichen commented 6 months ago

we dont need this right?

JacksonYao287 commented 6 months ago

yes, I believe we can get anther way to solve this issue, no need to blk_num()