search

erdewit / ib_insync

Python sync/async framework for Interactive Brokers API
BSD 2-Clause "Simplified" License
2.75k stars 723 forks source link

higher level functions for requesting historical data #16

Closed josefstr closed 6 years ago

josefstr commented 6 years ago

thanks for setting up another python library for interactive brokers.

Right now it's possible to request historical data on bars and tick data. The main problem is that Interactive Brokers is limiting the look-back for historical data. Here's a list of limitations: http://www.tradingsoftwarelab.com/jtwsdump/table.html So the lowest level on bars to request is seconds and here you can look-back a maximum of 2000 seconds (~30min). But it's possible to request a few month of tick data. You can request a maximum of 1000 ticks for each tick-request. I'm not familiar with python yet, so I just cannot write it on my own yet and make a pull request, but here's the pseudo code:

shevkoplyas commented 6 years ago

Josef, all,

I'd refer to other "pseudo code" for the retrieving historical data from IB API, which I described in this post in May in TWSAPI group: View/Reply Online (#37618) https://groups.io/g/twsapi/message/37618

cheers, Dmitry

On Fri, Sep 8, 2017 at 10:51 AM, josefstr notifications@github.com wrote:

thanks for setting up another python library for interactive brokers.

Right now it's possible to request historical data on bars and tick data. The main problem is that Interactive Brokers is limiting the look-back for historical data. Here's a list of limitations: http://www.tradingsoftwarelab.com/jtwsdump/table.html So the lowest level on bars to request is seconds and here you can look-back a maximum of 2000 seconds (~30min). But it's possible to request a few month of tick data. You can request a maximum of 1000 ticks for each tick-request. I'm not familiar with python yet, so I just cannot write it on my own yet and make a pull request, but here's the pseudo code:

  • make a tick request for a given start date
  • take the most recent timestamp received from this request and set it as the new start date for the second request. Loop until now. You will struggle with some pacing violations and you'll have to wait for them to release again. Best is to save all incoming tick data on a csv file for faster lookup.
  • take the tick data file and build bars out of them in the timeframe you want: 1-sec, 1-min, 22-min, 1hour. IBs data are not the best, so perhaps there is the need to make some winsorizing on them.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/erdewit/ib_insync/issues/16, or mute the thread https://github.com/notifications/unsubscribe-auth/AEC1PR-BWKXFnHk53xScnWnKar7VegaLks5sgVRWgaJpZM4PRScD .

erdewit commented 6 years ago

Josef,

That table must be out of date as I can get 1 min data from 2005 for stocks.

The idea of building bars from historical ticks works in principle, in fact it will take just a couple dozen lines of Python code. It will be very slow for active contracts though as a lot of requests need to be done. Doing it with bars like in Dmitri's post would be faster and has a longer look-back (for 1 min bars at least). Again it would take very little code, basically just a while loop with a reqHistoricalData in it.