Hackzing IB historical data request throttling

[goodboy]

IB has historical bar pacing restraints which are pretty annoying if you're looking to load up on high frequency data (like we do 🏄🏼).

The main limitation for our purposes is,

Making more than 60 requests within any ten minute period.

Which, for high freq data, we'll obviously need to do (eg. getting 1s bars for multiple symbols that have no previously been cached in storage).

Luckily digging into it there appears to be some hacks to work around this which require resetting the client's connection to the historical data servers in IB's farm. This somehow resets the throttle logic (after a few seconds reconnecting) and you can carry on your merry way.

Resources for this:

• https://www.multicharts.com/discussion/viewtopic.php?t=48582
• https://www.multicharts.com/trading-software/index.php/Interactive_Brokers_Pacing_Violation

Sending a key sequence to a given window can be done in X with xdotool: https://faq.i3wm.org/question/2742/send-key-sequence-to-running-application.1.html

Initial tests of this against TWS and GW showed it didn't work until I found this: https://unix.stackexchange.com/a/113694

Using this solution seems to work well on ib-gw:

win="$(xdotool search --name 'IB GATEWAY. ')"

# only seems to work with gw?
xdotool windowactivate --sync $win key 'ctrl+alt+f'

Also, using i3ipc you can retrieve the window ids my easily:

import i3ipc
i3 = i3ipc.Connection()
t = i3.get_tree()

# for tws
t.find_named('Interactive Brokers')[0].window

# for gw
t.find_named('IB')[0].window

So basically we need this logic incorporated with our historical data retrieval so we can avoid holdups when yanking large data sets.

[goodboy]

Just did another test with TWS and still can't get the key inputs to relay through from the top level (maybe there's something in xdotool that can help us here?), but i can make it work if you open the data window and then do:

xdotool windowactivate --sync "$(xdotool search --name 'Connections')" key 'ctrl+alt+f'

but it just means the user has to open that window first before the auto reconnect stuff can work.

update: booyah!

xdotool windowactivate --sync "$(xdotool search --name '<account#>')" mousemove_relative --sync 500 500 click 1 key 'ctrl+alt+f'

seems to work. the trick is making sure a window inside tws is clicked and figuring out that relative px move is pretty hacky.

[goodboy]

Looks like on windows we can use the win32com.client api to do this?

[goodboy]

💥 just put up a working script for i3 in snippets/.

Also made https://github.com/rr-/pyxdotool/issues/#1 to see if we can just use a wrapper in case we need other stuff like this down the road.