Backfill question

[rayanatrades]

Hi!

I'm trying to understand the complexity of backfilling aggr-server with historical data (from e. g. binance & bybit data files and/or public endpoints).

From what I understood, scripts/coinalize/* might be a good reference, and as I understood it's possible to backfill just minutely trades (which might be the only thing available for majority of exchanges outside of binance and bybit), and resample that into needed timeframes, is that correct?

[Tucsky]

Yeah the coinalize script is just last resort for me when there is no other way to recover that data...

Just enter all the markets you need in the pairs option and the timeframes you need in the influxResampleTo option And of course an influx server running. Also you might want to change the session id used to scrap coinalize data https://github.com/Tucsky/aggr-server/blob/master/scripts/coinalize/utils.js#L42

cookie: 'p_sid=*your coinalize sid here*',

From there you can fetch 1m data, it will resample into your timeframes >1m. C:\Program Files\nodejs\node.exe .\scripts\coinalize\index.js timeframe=1m from=2023-03-22T00:00:00.000Z to=2023-03-23T21:00:00.000Z

[rayanatrades]

Ack. I am trying to repurpose aggr-server as a big candlesticks cache for a project. Provided info is helpful to write own backfill source script. Thanks!

I am also interested in specific usecase for aggr. Can we get in touch in Discord? I have sent a friend request

[codestoned1]

In my experience coinalyze only supported last 10k samples for any time range. There any way to backfill further back?

[adeacetis]

I don't know about Coinalize but it's possible to get the info straight from the exchanges. However, it will require some work because you can get only a thousand datapoint per request but you could go back all the way to 2017 with Binance for instance.