search

0chain / gosdk

A client SDK in Go to interface the blockchain and storage platform, and other smart contracts
MIT License
32 stars 29 forks source link

repair allocation optimisation #1053

Closed boddumanohar closed 1 year ago

boddumanohar commented 1 year ago

currently during allocation repair, we download the entire file, divide into shards as per the updated allocation. This method has 2 problems:

  1. This is solution is not scalable with respect to the size of file.
  2. This solution is not applicable in browsers.

An optimisation that can be done is to do repair on the fly by chunks. You could download a few chunks of data and shard it and upload it to the new blobber.

this downloaded chunk of data needs to be stored in ring buffer so that we don't take up any space on the local file system. This technique is very much needed to implement repair in browser.

boddumanohar commented 1 year ago

the main goal of this ticket is to allow repair in browser. I just tested the merged PR, I am still not able to get the perform repair and I getstat .: not implemented on js this error

Hitenjain14 commented 1 year ago

The issue comes from https://github.com/0chain/gosdk/blob/staging/zboxcore/sdk/repairworker.go#L202, should we add an option in wasm where we don't perform a check for this?