bdib data not consistent with starting time

[gflores87]

I create a query to fetch intraday hourly data for VIX. The first day data is fetch properly, at the 0 minutes mark, but when the next day comes the timestamp shifts to the 15 minutes mark.

This is the actual code I'm running:

import pdblp
con = pdblp.BCon(debug=True, port=8194)
con.start()
VIX = con.bdib('VIX Index', '2017-12-13T14:00:00', '2018-01-24T21:00:00','TRADE',60) 
VIX.index = VIX.index.tz_localize('UTC',).tz_convert('US/Eastern').tz_localize(None)
VIX.head(20)

The output can be verified with

VIX.head(20)

[matthewgilbert]

Could you please remove the VIX.index = VIX.index.tz_localize('UTC',).tz_convert('US/Eastern').tz_localize(None) since this may obfuscate the issue and post the actual screen output printed from debug. Likely better to fetch only two days of data, since this is the minimum it appears is required to reproduce the issue.

[gflores87]

Sure, changing the code to VIX = con.bdib('VIX Index', '2017-12-13T12:00:00', '2017-12-14T21:00:00','TRADE',60) yields:

DEBUG:root:Sending Request:
 IntradayBarRequest = {
    security = "VIX Index"
    eventType = TRADE
    interval = 60
    startDateTime = 2017-12-13T12:00:00
    endDateTime = 2017-12-14T21:00:00
}

DEBUG:root:Message Received:
 IntradayBarResponse = {
    barData = {
        eidData[] = {
        }
        barTickData[] = {
            barTickData = {
                time = 2017-12-13T12:00:00.000
                open = 9.850000
                high = 10.110000
                low = 9.850000
                close = 9.950000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T13:00:00.000
                open = 10.080000
                high = 10.080000
                low = 9.730000
                close = 9.960000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T14:00:00.000
                open = 9.960000
                high = 9.960000
                low = 9.650000
                close = 9.730000
                volume = 0
                numEvents = 175
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T15:00:00.000
                open = 9.720000
                high = 9.860000
                low = 9.680000
                close = 9.790000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T16:00:00.000
                open = 9.790000
                high = 9.980000
                low = 9.790000
                close = 9.850000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T17:00:00.000
                open = 9.880000
                high = 9.950000
                low = 9.820000
                close = 9.870000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T18:00:00.000
                open = 9.870000
                high = 9.970000
                low = 9.870000
                close = 9.880000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T19:00:00.000
                open = 9.880000
                high = 9.890000
                low = 9.650000
                close = 9.840000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T20:00:00.000
                open = 9.850000
                high = 10.160000
                low = 9.840000
                close = 10.140000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-13T21:00:00.000
                open = 10.130000
                high = 10.210000
                low = 10.070000
                close = 10.180000
                volume = 0
                numEvents = 60
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T08:15:00.000
                open = 9.980000
                high = 9.980000
                low = 9.950000
                close = 9.960000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T09:15:00.000
                open = 9.950000
                high = 9.970000
                low = 9.890000
                close = 9.930000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T10:15:00.000
                open = 9.930000
                high = 9.970000
                low = 9.920000
                close = 9.950000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T11:15:00.000
                open = 9.950000
                high = 9.980000
                low = 9.950000
                close = 9.970000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T12:15:00.000
                open = 9.970000
                high = 10.080000
                low = 9.960000
                close = 10.070000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T13:15:00.000
                open = 10.070000
                high = 10.090000
                low = 9.890000
                close = 9.960000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T14:15:00.000
                open = 10.040000
                high = 10.040000
                low = 9.780000
                close = 9.800000
                volume = 0
                numEvents = 174
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T15:15:00.000
                open = 9.790000
                high = 9.980000
                low = 9.780000
                close = 9.970000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T16:15:00.000
                open = 9.950000
                high = 10.110000
                low = 9.930000
                close = 10.050000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T17:15:00.000
                open = 10.050000
                high = 10.140000
                low = 9.980000
                close = 10.140000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T18:15:00.000
                open = 10.160000
                high = 10.260000
                low = 10.100000
                close = 10.190000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T19:15:00.000
                open = 10.190000
                high = 10.540000
                low = 10.110000
                close = 10.210000
                volume = 0
                numEvents = 240
                value = 0.000000
            }
            barTickData = {
                time = 2017-12-14T20:15:00.000
                open = 10.210000
                high = 10.400000
                low = 9.900000
                close = 10.340000
                volume = 0
                numEvents = 180
                value = 0.000000
            }
        }
    }
}

[matthewgilbert]

This appears to be how Bloomberg is returning the data, since the library pdblp just parses the responses and presents the data in a more convenient fashion. e.g.

barTickData = {
    time = 2017-12-14T08:15:00.000
    open = 9.980000
    high = 9.980000
    low = 9.950000
    close = 9.960000
    volume = 0
    numEvents = 240
    value = 0.000000
}

There is possibly some override for changing the default behaviour of IntradayBarRequests discussed in the Developers Guide however I'm less familiar with this Service.

[gflores87]

In the case of BDH the override to address this issue is "RecurDaily=True", how can I use the same override using bdib?

[matthewgilbert]

I would ask using the Bloomberg Terminal help what the corresponding override would be in blpapi, since I am unaware. Alternatively there may be some discussion of it in Developers Guide.

[gflores87]

Let me ask differently. Since bdib doesn't allow overrides that means any solution to my problem involves changing my source code and modify the way bdib works?

[matthewgilbert]

Override was a poor choice of terminology by me above. This may be possible by setting an override or setting a field value. If it is only possible through an override than you are correct that this functionality would not currently be supported by bdib but if you can do this by setting a field value than this would be possible with the elms parameter in bdib. However I am unfamiliar with the keyword you would have to pass if possible.

[gflores87]

One last thing, is it possible to use pdblp to specify a calendar? In regular bloomberg one would do =bdh(ticker,field,startDate,endDate,"CDR=US"). Can this be done with pdblp?

[matthewgilbert]

I would take a look in the old Developer's Guide for this type of thing, for some reason the new guide seems to have removed the schemas. Specifically the section A.2.4 HistoricalDataRequest: Sequence from page 164 onward. It seems like you might want to do something like set elms=[("periodicityAdjustment", "CALENDAR")] and ovrds=[("calendarCodeOverride", "US")] but I haven't tested this.

[gflores87]

Hi,

I made some research and this seems to be the proper sintax:

elms=[("calendarCodeOverride", "EU")])

Thanks for your help!

[matthewgilbert]

Okay great, glad to hear you got things working. I will close this then.

[gflores87]

Hello, One last thing. I made a mistake when posting previously. elms=[("calendarCodeOverride", "EU")]) works for bdh. For bdib I tried your suggestion but got the following error: bdib() got an unexpected keyword argument 'ovrds' So it looks like there's still no way to use a calendar override for intra day queries

[matthewgilbert]

Please post the query you made with the stack trace error message. Also set the debug property in your BCon object to True