Open jaksiprejak opened 4 years ago
This seems odd. Are you able to reproduce this issue when you make the call directly via blpapi
? Does this issue always happen or is it intermittent?
I haven't had a chance to code it up with the blpapi
, however I ripped out the bbg stuff from tia and ran 2to3
on it. It's been running for over an hour now with no missing ticker issues.
Have you been able to reproduce?
I'm wondering if it is something to do with the timeouts. I get no timeout errors with the tia code (I assume they're handling them internally).
I have been experiencing an odd issue with multithreading and timeouts as well. I have a bunch of threads with their own queues (they're actually actors from pykka). Each one has it's own BCon
object. They're all basically calling ref
on with arguments they get from their queues. They then return the resultant DataFrames with the arguments used for the request.
The ref()
calls are wrapped in a while...try...except
retry loop to handle timeouts. When a timeout occurs, the next successful ref()
seemingly returns data from a request that one of the other threads made - evidenced by the fact that a different set of tickers is returned than the ones requested.
I haven't been able to boil this down to a simple code example to reproduce - it could very well be in my logic (although after a few hours I don't see the problem). I just thought I would mention it in case it would trigger something in your mind to help identify the original issue.
Does this issue always happen or is it intermittent?
The output I posted is from the final iteration of the while loop. It ran successfully 104 times before that.
However I am running a lot of ref
calls in parallel so am seeing it fairly often.
the next successful ref() seemingly returns data from a request that one of the other threads made - evidenced by the fact that a different set of tickers is returned than the ones requested.
Hmm, okay from the sounds of it possibly the default blpapi.EventQueue
for the blpapi.Session
is getting populated with multiple requests. This shouldn't happen across the BCon
instances but for each individual instance there could be some timeout/ backlog issue.
This library doesn't use a seperate blpapi.EventQueue() for each request so this type of in parallel requesting could be problematic and is a use case I haven't really used / tested for. It would definitely make sense to change the blpapi.Session.sendRequest
calls to use their own blpapi.EventQueue
s but I don't think I will get around to making that modification any time soon.
When I repeatedly call for a fields on a list of tickers with
ref
, I occasionally get back an incomplete dataframe i.e. values for some tickers did not get returned.Code sample and debug output below.
I am running with my session connected to local terminal, and with
pdblp
version 0.1.8Debug output