CalebEverett / arloader

Rust command line application and client for uploading files to Arweave.
Apache License 2.0
82 stars 23 forks source link

crash when tx status returns internal server error #16

Closed ozgrakkurt closed 2 years ago

ozgrakkurt commented 2 years ago

currently tx status endpoint on arweave http server returns internal server error and this isn't currently handled in arloader so we get this crash: thread 'tokio-runtime-worker' panicked at 'internal error: entered unreachable code', github.com-1ecc6299db9ec823/arloader-0.1.59/src/lib.rs:1271:18

CalebEverett commented 2 years ago

ok - is the endpoint down, or is it timing out because of too many concurrent requests? I'll take a look and handle the error.

ozgrakkurt commented 2 years ago

we are calling https://arweave.net/ so it should be up I think. also we aren't making concurrent requests it is only one

enzotar commented 2 years ago

Here is the code used for the call. Also had a question about running tests with Solana Testnet or Devnet: we can only make it work paying with Arweave and not Sol, hence the match in the code below. Are we doing something wrong? Can move this to a separate issue if you want.

 let (arweave, mut status) = match ctx.solana_net {
            SolanaNet::Mainnet => {
                let arweave = Arweave {
                    name: String::from("arweave"),
                    units: String::from("sol"),
                    base_url: url::Url::parse("https://arweave.net/").unwrap(),
                    crypto: arloader::crypto::Provider::from_keypair_path(arweave_key_path.into())
                        .await?,
                };

                let price_terms = arweave.get_price_terms(reward_mult).await?;

                let status = arweave
                    .upload_file_from_path_with_sol(
                        file_path.into(),
                        None,
                        None,
                        None,
                        price_terms,
                        ctx.solana_net.url(),
                        url::Url::parse("https://arloader.io/sol").unwrap(),
                        &fee_payer,
                    )
                    .await?;

                (arweave, status)
            }
            _ => {
                let arweave = Arweave {
                    name: String::from("arweave"),
                    units: String::from("winstons"),
                    base_url: url::Url::parse("https://arweave.net/").unwrap(),
                    crypto: arloader::crypto::Provider::from_keypair_path(arweave_key_path.into())
                        .await?,
                };

                let price_terms = arweave.get_price_terms(reward_mult).await?;

                let status = arweave
                    .upload_file_from_path(file_path.into(), None, None, None, price_terms)
                    .await?;

                (arweave, status)
            }
        };

        loop {
            match status.status {
                StatusCode::Confirmed => break,
                StatusCode::NotFound => {
                    return Err(Error::ArweaveTxNotFound(status.id.to_string()))
                }
                StatusCode::Submitted | StatusCode::Pending => {
                    tokio::time::sleep(Duration::from_secs(1)).await;
                    status = arweave.get_status(&status.id).await?;
                }
            }
        }
CalebEverett commented 2 years ago

we are calling https://arweave.net/ so it should be up I think. also we aren't making concurrent requests it is only one

ok - I can add an error there that indicates there was a problem with the network.

CalebEverett commented 2 years ago

Hi - yes that is right - the arloader.io endpoint only works with main net. Since there is no Arweave testnet, all the transactions have to be paid for and so real sol has to be used. I usually test with a small number of ar files just on mainnet since the absolute cost of uploads to arweave is low.

CalebEverett commented 2 years ago

@ozgrakkurt Ok, I added an error there that contains the unspecified network status code. Up on master now - v0.1.60 building https://github.com/CalebEverett/arloader/blob/c2c0c4855ea53678fceaf7041deda42ab49f2f29/src/lib.rs#L1272

ozgrakkurt commented 2 years ago

thanks!

ozgrakkurt commented 2 years ago

Here is the code used for the call. Also had a question about running tests with Solana Testnet or Devnet: we can only make it work paying with Arweave and not Sol, hence the match in the code below. Are we doing something wrong? Can move this to a separate issue if you want.

 let (arweave, mut status) = match ctx.solana_net {
            SolanaNet::Mainnet => {
                let arweave = Arweave {
                    name: String::from("arweave"),
                    units: String::from("sol"),
                    base_url: url::Url::parse("https://arweave.net/").unwrap(),
                    crypto: arloader::crypto::Provider::from_keypair_path(arweave_key_path.into())
                        .await?,
                };

                let price_terms = arweave.get_price_terms(reward_mult).await?;

                let status = arweave
                    .upload_file_from_path_with_sol(
                        file_path.into(),
                        None,
                        None,
                        None,
                        price_terms,
                        ctx.solana_net.url(),
                        url::Url::parse("https://arloader.io/sol").unwrap(),
                        &fee_payer,
                    )
                    .await?;

                (arweave, status)
            }
            _ => {
                let arweave = Arweave {
                    name: String::from("arweave"),
                    units: String::from("winstons"),
                    base_url: url::Url::parse("https://arweave.net/").unwrap(),
                    crypto: arloader::crypto::Provider::from_keypair_path(arweave_key_path.into())
                        .await?,
                };

                let price_terms = arweave.get_price_terms(reward_mult).await?;

                let status = arweave
                    .upload_file_from_path(file_path.into(), None, None, None, price_terms)
                    .await?;

                (arweave, status)
            }
        };

        loop {
            match status.status {
                StatusCode::Confirmed => break,
                StatusCode::NotFound => {
                    return Err(Error::ArweaveTxNotFound(status.id.to_string()))
                }
                StatusCode::Submitted | StatusCode::Pending => {
                    tokio::time::sleep(Duration::from_secs(1)).await;
                    status = arweave.get_status(&status.id).await?;
                }
            }
        }

@CalebEverett if we execute this code we get ArweaveTxNotFound error. Is there any reason that can be seen from the code why this would happen?

CalebEverett commented 2 years ago

@ozgrakkurt maybe check to make sure the wallet balance is sufficient - also make sure the reward is high enough. I think if the balance is not high enough, it wouldn't get submitted to arloader.io endpoint, but if the reward is not high enough it will be not found. Also maybe check the id after some time at arweave.net, just to make sure it isn't resulting from a network delay.

ozgrakkurt commented 2 years ago

thanks!