GenesysGo / shadow-drive-rust

Apache License 2.0
21 stars 13 forks source link

Upload Multiple Files #10

Closed cavemanloverboy closed 2 years ago

cavemanloverboy commented 2 years ago

I tried to upload a 2x5 set of files and ran into

ShadowBatchUploadResponse { file_name: "example_10", status: Error("ShadowDriveServerError { status: 400, message: Object({\"error\": String(\"Error: failed to send transaction: Transaction simulation failed: Error processing Instruction 0: Cross-program invocation with unauthorized signer or writable account\")}) }"), location: None, transaction_signature: None }

for one batch of five, and successful upload for the other batch of five. I believe this is because instead of waiting for max confirmations on a batch the library instead does

            //wait 1/2 second before going to next batch
            //without this the latest blockhash doesn't align with the server's recent blockhash
            tokio::time::sleep(Duration::from_millis(500)).await;

before proceeding to the next batch with five retries. I have several suggestions. The first is to change this so that instead of sleeping we simply wait for the previous batch to reach max confirmations. The second is to perhaps move this sleep into the retry loop (with a 50-100 ms wait) so that you aren't just spamming retry. The third is to make this retry parameter an input to the function so that the user decides the number of retries (not super sure about this one so let me know what you think about it first).

cavemanloverboy commented 2 years ago

hold off on this for now. closing issue

VegetarianOrc commented 2 years ago

I know you closed this one for now @cavemanloverboy, but I did take a quick look at it and retest locally. Seems like there may be a race condition in my test where txns aren't finalized server side? We could probably guard against this by inserting a confirmation of the txn id returned from the http call. I might do a little more digging and see if adding that alleviates the intermittent issues I'm seeing.

robbestad commented 1 year ago

Could you re-open and look at implementing it for v2 drives, given that the race issue wrt transactions is only a problem for v1?