Ingesting real-time streaming data feed (Live) from Interactive Broker instead of .CSV

[developeralgo8888]

@Kismuz ,

Can you give some insights on how best to ingest realtime streaming FOREX data feed from Interactive Broker into btgym instead of .CSV for continuous online training , testing and trading

• Currently i believe we have .CSV Data ==> Btgym (offline training & validation )==> signals/Action ( Buy /Sell/Hold)

what i am trying working on .

With BT , it already has infrastructure in place to get realtime streaming datafeed from IB, so that's not an issue. my issue is how to integrate it with BTGYM infrastructure with minimal changes to Btgym core structure . ( Less work/maintenance means more time to work on improving and testing Algorithms offline )

Interactive Brokers Realtime Data feed => DB for persistence ( Influx/Postgresql/Mongo) ==> Btgym (online training & validation ) ==> signals/Action ( Buy /Sell/Hold) ==> Trade sent to interactive Brokers

[Kismuz]

@developeralgo8888 ,

btgym.datafeed module contains several DataSet classes (data iterators) to construct data input pipeline. Yes, in its current form it is: CSV[source data]-->btgym.Dataset_iterator -->bt.feed -->btgym.strategy btgym.Dataset iterators are self-contained in a sense it hold data-management logic and API (sampling, train|test splitting etc..) and data serving upon requests from decision RL algorithm ( for example RL algorithm could request one validation episode after ten train episodes via parameters of env.reset() methods, and it is exactly btgym.Dataset which provides executive btgym.strategy with relevant data) So in a first glance, it is straightforward to make btgy.Dataset wrapper for any source other than CSV and define all data management logic inside it: bt.stores.IBStore-->btgym.Dataset_wrapper-->btgym.strategy ...but in a case of live data it is a bit more complicated:

• one can see live data stream as stateful one-shot data iterator. This is fine as long as we have pre-trained model we just want to validate/test;
• if we want to get model from scratch, we need a lot of train data repeatedly available as training on live stream is prohibitively expensive;
• moreover, it is evident that for our domain it is necessary to at least adjust already trained model constantly (or at least regularly) as time passes to compensate domain shift via one of meta-learning methods or somehow else;
• it means that, apart from live stream, good data_iterator should maintain subset of [recent] historic data, which can be thought as big external replay memory containing raw (unprocessed) data. Later is important: just storing past experiences in replay memory of RL algorithm is not sufficient because replay memory stores processed experiences, and processing scheme changes as training goes on.
• all these makes implementing live data iteration an interesting topic by itself :) Anyway it should be done as soon as possible. At the moment I'm into implementing multiply assets trading functionality, think live data will be next. Any ideas are appreciated.
• for your reference, there is recently added class btgym.datafeed.casual.BTgymCasualDataDomain imitating live data stream management. It is my first attempt to implement ideas mentioned above. For general overview of data iterators api, see: https://github.com/Kismuz/btgym/blob/master/examples/data_domain_api_intro.ipynb

[developeralgo8888]

@Kismuz , Thanks for that great explanation . I was thinking on having a trained model and just to validate/test it with real-time streaming data to trade. I do understand live online training of streaming data is really massively expensive .

initial thought of a continuous loop:

IB Realtime Streaming data ==> btgym ==> continuously update and validate/test offline Trained model ==> get buy/sell signals ==> send live trade ==> IB

Continuous updating and validating/test offline model will require some rolling window model . Do we do a continuous offline training of model and update or append or rewrite the live trading model every 15 mins, 30 mins or 1 hour depending on our trading timeframe ?

[Kismuz]

@developeralgo8888 ,

Do we do a continuous offline training of model and update or append or rewrite the live trading model every 15 mins, 30 mins or 1 hour depending on our trading timeframe ?

• basically, yes. But with multi-worker algorithms like A3C or A2C it is possible to do better and implement continuous updating of a model via separation worker's tasks: say, one worker is real-time validator while others are running training loop on data available up to present point. Following A3C base update routine, validator can update learned behaviour policy after every rollout or even after single step (and we still got, say, 15 min to retrain model given new data). Similar approach (for off-policy training via exp.replay) is already implemented in recent DeepMind REACTOR algorithm, see paragraph 3.4 of the paper: https://arxiv.org/abs/1704.04651

[developeralgo8888]

@Kismuz , thanks for the great clarification . well appreciated . The REACTOR algorithm is very impressive and i am going through it to see how it can be applied to btgym