Combining historical data with live data issue (Fix suggestion)

[luki009]

https://github.com/conor19w/Binance-Futures-Trading-Bot/blob/b6b057076f4c27e78803312d9969a81a2c4be56c/Helper.py#L125-L142

In that block of code is small issue. client.futures_historical_klines(symbol, Interval,start_str=start_string) will return historical data but last [-1] candle is current active candle which is never closed. This is causing issue in calculating strategy later because when new last candle is added Close[-2] does not reflect actual previous candle close but reflect current value of candle in time of historical data pull. To fix this do pop before processing data:

hist_data = client.futures_historical_klines(symbol, Interval,start_str=start_string)
hist_data.pop(-1)
for kline in hist_data:
      ........

[conor19w]

Nice one, yeah I noticed the issue before but just ignored it because I wasn't sure if it was happening only on some candles. Edge Case: What If a candle close happens as we download the data, probably best to just assume this is really unlikely.

When combining the historical data I do a comparison to see if the candle is later than the previous, maybe I should check that the candle interval length is uniform with the others instead.Then I could just remove the candle if it isn't uniform, this would solve both issues I guess.

[conor19w]

After thinking further, I think your solution solves the problem entirely good work. Either way if the candle closes or not mid download the websocket stream will have that candle so we are good to throw away the last one.

[luki009]

Exactly as you mentioned. Socket stream has current candle.. Thing is yeah scenario that in historical data you will pull closed candle is really unlikely (I even say impossible) because when candle close - new candle is formed immediately therefore socket stream has previous candle as closed .. Nearly impossible situation is that you will pull historical data in exact time of candle close on miliseconds precision. In that case it would happen that you will remove last whole candle therefore in data will be 1 candle skipped. but as was already concluded that is nearly impossible situation and I believe that new candle is formed by Binance at once with close of candle so there is no chance to get closed candle without new so you always get open candle.