azdagron
commented
1 year ago In terms of an API, I think we could get away with adding single RPC, ~ExchangeSegmentPieceOrders~ RetryBeginSegmentPieces or something that allows an uplink to replace piece orders that were originally created by BeginSegment.
The flow could then be as follows:
- Uplink calls BeginSegment and gets back total orders for the in the redundancy scheme.
- Instead of using all piece orders to put pieces like it does today, it employs an (as yet undefined) strategy to use a subset, maybe eventually approaching the total if there are failures or long tails.
- If the segment upload is in danger of failing, because enough pieces have failed, the uplink can call ~ExchangeSegmentPieceOrders~ RetryBeginSegmentPieces with a list of failed piece numbers and get back an amended list of orders with those pieces replaced. The response includes the modified (and re-signed) segmentID, which contains the piece orders and is verified during CommitSegment.
- Uplink can use the new piece orders to retry failed pieces.
- Once it has succeeded, it can call CommitSegment, which will verify the upload results against the signed list of piece orders contained in the segmentID, which is the original, or, if step 3 was exercised, an amended list.
storj-gerrit[bot]
iglesiasbrandon
Right now we have a overly conservative 29/80 scheme that requires a ton of extra bandwidth. Instead of doing something static, it would be nice if the uplink could communicate with the satellite to get peers dynamically according to the network conditions it is experiencing. For example, if it is uploading many parallel segments, it maybe doesn't need as many extra peers to maintain a good quality of service.
Deliverable: