Issue with buffer and historical data

[beppe969]

Hello, I have a problem with retrieving historical data, In the example below I want to first get daily data for 30 days, and then minute data for the same days. However, the second set of bars still contains only 30 data points...

What is wrong? I tried to clear the buffer with buf.clear() between the two calls to session.eClientSocket.reqHistoricalData, but it does not work either.

Thanks for any suggestion!

Giuseppe

clear all session = TWS.Session.getInstance(); % create local buffer for historical data events [buf,lh] = TWS.initBufferForEvent(TWS.Events.HISTORICALDATA);
session.eClientSocket.eConnect('127.0.0.1',7496,0); contract = com.ib.client.Contract(); contract.symbol('A') contract.exchange('SMART'); contract.primaryExch('NYSE'); contract.currency('USD'); contract.secType('STK');

% get DAILY data for the last 30 Days: session.eClientSocket.reqHistoricalData(100001,contract,'','30 D','1 day','TRADES',1,1,false,[]); pause(0.5);

toolate = 0; tic while buf.size() == 0 & ~toolate pause(.01); elapsed = toc; toolate = elapsed > 20; end if toolate error('Waited too much!'); end hde = buf.get(); % bars = collection2cell(hde.data); nbars = numel(bars) % this should be equal to 30 data points

%% Now do retrieve minute-by-minute data:
session.eClientSocket.reqHistoricalData(200001, contract,'','30 D','1 min','TRADES',1,1,false,[]); pause(1);

toolate = 0; tic while buf.size() == 0 & ~toolate pause(.05); elapsed = toc; toolate = elapsed > 10; end if toolate error('Waited too much!'); end hdem = buf.get(); barsm = collection2cell(hdem.data); nbarsm = numel(barsm) % this should be much more than 30 data points...

[Despair2000]

Have you tried downloading less 1min bars? 30 days of 1min bars is a lot and there are limits of how much one can download with one request. Try "1 W" instead of "30D" (this works for me) and see if it works. Ahh now I see the problem. You just wait 0.5 seconds (!) for the buffer to fill. Downloading so many bars takes much (!) longer), try 30s :-)

[beppe969]

Hello Deaspair2000, thanks for your answer but I'm afraid it is not a problem of waiting. Actually in my code there is a waiting loop, and in any case the data does not appear even if I wait 50 seconds. Moreover, if you just swap the two requests (i.e., make the '1 min' request BEFORE the '1 day' request then something similar happens, that is the 1 min request is fulfilled correctly, while the daily request is not. So I guess it is something about using the same buffer or something of this sort...

[softwarespartan]

The advice from @Despair2000 is correct. Also not sure if your wait loops are quite right.

At any rate, from your code, it looks like you get the same item from the buffer after both historical data calls. The get method does not remove the item from the queue.

Just make two historical data requests and then look at buf.size(), it should be two.

req1 = buf.get(); buf.remove()
req2 =buf.get(); buf.remove()

which is the same as

req1 = buf.remove()
req2 = buf.remove()

[beppe969]

Thanks for the suggestion! buf.remove() seems to work!

Now I am getting error 322: 322 Error processing request:-'bS' : cause - Only 50 simultaneous API historical data requests allowed.

Once you got the historical data, how to you "close" that data request, so that a new request does not appear to be "simultaneous" ?

Thanks! Giuseppe

[softwarespartan]

That's an IB error for requesting too much data or requesting data too fast.

Please be sure to read the IB API docs

http://interactivebrokers.github.io/tws-api/historical_data.html

In particular, the section on "Historical Data Limitations" and "Pacing Violations"

I'm going to go ahead a close this issue since buffer usage issues resolve. Feel free to reopen if have more questions/issues.