Get klines for all symbols asynchronously

[0ihsan]

I am trying to get klines for all symbols, I have created this function but having some problems as I am new to the Rust language. Do you have any solutions or suggestions, I also realized that there is Igosuki/binance-rs-async, but it lacks of futures API?

My approach to get klines for all symbols is:

1. fetch exchangeinfo
2. fetch klines for each symbol with a custom async reqwest client

Cargo.toml

[dependencies]
# ...
futures = "0.3"

./src/futures/market.rs

// ...

use futures::future;
use crate::api::*;
use crate::futures::general::*;

// ...
impl FuturesMarket {

    // ...

    pub async fn get_all_klines<S1, S2, S3, S4>(
        &self, interval: S1, limit: S2, start_time: S3, end_time: S4,
    ) -> Result<Vec<Result<KlineSummaries>>>
    where
        S1: Into<String>,
        S2: Into<Option<u16>>,
        S3: Into<Option<u64>>,
        S4: Into<Option<u64>>,
    {

        let general: FuturesGeneral = Binance::new(None, None);
        match general.exchange_info() {
            Err(e) => {
                eprintln!("error: {}", e);
                bail!("can't get all symbol names")
            },
            Ok(exchangeinfo) => {

                let client = reqwest::Client::new();
                let mut parameters: BTreeMap<String, String>;
                let urls: Vec<String>;

                // fill urls[]
                for symbol in exchangeinfo.symbols {
                    parameters = BTreeMap::new();
                    parameters.insert("symbol".into(), symbol.symbol.into());
                    parameters.insert("interval".into(), interval.into());
                    // Add three optional parameters
                    if let Some(lt) = limit.into() {
                        parameters.insert("limit".into(), format!("{}", lt));
                    }
                    if let Some(st) = start_time.into() {
                        parameters.insert("startTime".into(), format!("{}", st));
                    }
                    if let Some(et) = end_time.into() {
                        parameters.insert("endTime".into(), format!("{}", et));
                    }
                    urls.push(build_request(parameters));
                }

                let responses = future::join_all(urls.into_iter().map(|url| {
                    let client = &client;
                    let mut url: String = format!("{}{}", "https://fapi.binance.com".into(), String::from(API::Futures(Futures::Klines)));
                    url.push_str(format!("?{}", url).as_str());
                    async move {
                        println!("-> get {:?}", url);
                        let resp = client.get(url).send().await?;
                        resp.bytes().await
                    }
                })).await;

                let mut all_klines: Vec<Result<KlineSummaries>>;

                for response in responses {
                    response.iter().map(|r|
                        all_klines.push(
                            Ok(KlineSummaries::AllKlineSummaries(
                                r.iter()
                                    .map(|row| KlineSummary {
                                        open_time: to_i64(&row[0]),
                                        open: to_f64(&row[1]),
                                        high: to_f64(&row[2]),
                                        low: to_f64(&row[3]),
                                        close: to_f64(&row[4]),
                                        volume: to_f64(&row[5]),
                                        close_time: to_i64(&row[6]),
                                        quote_asset_volume: to_f64(&row[7]),
                                        number_of_trades: to_i64(&row[8]),
                                        taker_buy_base_asset_volume: to_f64(&row[9]),
                                        taker_buy_quote_asset_volume: to_f64(&row[10]),
                                    })
                                    .collect(),
                            ))
                        )
                    );
                }

                Ok(all_klines)

            }
        }
    }

    // ...

Error I got

   Compiling binance v0.16.1 (/Users/ihsan/Sync/code/github.com/ihsanturk/binance-rs)
error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:244:60
    |
244 | ...                   open_time: to_i64(&row[0]),
    |                                          ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:245:55
    |
245 | ...                   open: to_f64(&row[1]),
    |                                     ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:246:55
    |
246 | ...                   high: to_f64(&row[2]),
    |                                     ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:247:54
    |
247 | ...                   low: to_f64(&row[3]),
    |                                    ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:248:56
    |
248 | ...                   close: to_f64(&row[4]),
    |                                      ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:249:57
    |
249 | ...                   volume: to_f64(&row[5]),
    |                                       ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:250:61
    |
250 | ...                   close_time: to_i64(&row[6]),
    |                                           ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:251:69
    |
251 | ...                   quote_asset_volume: to_f64(&row[7]),
    |                                                   ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:252:67
    |
252 | ...                   number_of_trades: to_i64(&row[8]),
    |                                                 ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:253:78
    |
253 | ...                   taker_buy_base_asset_volume: to_f64(&row[9]),
    |                                                            ^^^^^^

error[E0608]: cannot index into a value of type `&u8`
   --> src/futures/market.rs:254:79
    |
254 | ...                   taker_buy_quote_asset_volume: to_f64(&row[10]),
    |                                                             ^^^^^^^

error: aborting due to 11 previous errors

For more information about this error, try `rustc --explain E0608`.
error: could not compile `binance`

To learn more, run the command again with --verbose.

[wisespace-io]

@ihsanturk Sorry for the late reply, this crate does not have async support. Have you checked https://github.com/nash-io/openlimits? Not sure if they added Futures API, however they have async support.

[0ihsan]

Thanks for the, I am using Igosuki/binance-rs-async now.

[LeilaMoussa]

@ihsanturk I've tried my hand at binance-rs-async before. Based on my experience, it's too outdated and poorly maintained. You could try maintaining it, but it's really fallen behind with respect to Rust Futures.

[0ihsan]

Thank you @LeilaMoussa. I already ended up switching to python-binance since it was easier to get go for me and supports asynchronous request natively. I thought that I would gain some performance by using Rust for high frequency trading but Python implementation was just right there. So I focused primarily on finishing my project with whatever language (Python in this case) and profiling and optimisation secondarily.

[Swoorup]

@ihsanturk I've tried my hand at binance-rs-async before. Based on my experience, it's too outdated and poorly maintained. You could try maintaining it, but it's really fallen behind with respect to Rust Futures.

Curious what are your preferences now after a year later. I have the same question.