Error accessing Null scalar blpapi.element.Element

[Rayzola]

Tried to pull out some historical dividend data through ref() and 'DVD_HIST', but encountered the error "IndexOutOfRangeException".

I did a check on the Bloomberg terminal, there is an empty "payable date" in one of the dividends being pulled out which might cause this issues. Is there a workaround to this "IndexOutOfRangeException" issue?

This codes works perfectly fine

tickers = ['101 HK EQUITY']
flds = ['DVD_HIST']
dates = ['19900430']
ovrds = [ ('DVD_END_DT', '20000430')]
A = con.ref_hist(tickers, flds, dates, date_field='DVD_START_DT', ovrds=ovrds)

But if I change the starting date to include the dividend with the empty "payable date" as follow:

tickers = ['101 HK EQUITY']
flds = ['DVD_HIST']
dates = ['19800430']
ovrds = [ ('DVD_END_DT', '20000430')]
A = con.ref_hist(tickers, flds, dates, date_field='DVD_START_DT', ovrds=ovrds)

I got the error message:

Traceback (most recent call last):

  File "<ipython-input-5-47a18076be67>", line 5, in <module>
    A = con.ref_hist(tickers, flds, dates, date_field='DVD_START_DT', ovrds=ovrds)

  File "C:\Users\otrade\Anaconda3\envs\BloombergAPI\lib\site-packages\pdblp\pdblp.py", line 370, in ref_hist
    timeout=timeout)

  File "C:\Users\otrade\Anaconda3\envs\BloombergAPI\lib\site-packages\pdblp\pdblp.py", line 285, in _parse_ref
    dataj = [ticker, mfld, elm.getValue()]

  File "C:\Users\otrade\Anaconda3\Lib\site-packages\blpapi\element.py", line 505, in getValue
    return valueGetter(self, index)

  File "C:\Users\otrade\Anaconda3\Lib\site-packages\blpapi\element.py", line 436, in getValueAsDatetime
    _ExceptionUtil.raiseOnError(res[0])

  File "C:\Users\otrade\Anaconda3\Lib\site-packages\blpapi\exception.py", line 145, in raiseOnError
    _ExceptionUtil.raiseException(errorCode, description)

  File "C:\Users\otrade\Anaconda3\Lib\site-packages\blpapi\exception.py", line 137, in raiseException
    raise errorClass(description, errorCode)

IndexOutOfRangeException: Attempt to access an empty scalar element 'Payable Date' (0x0005000b)

Thanks

[matthewgilbert]

This looks like a bug, thanks for reporting. As an aside, for your example it's better to use ref than ref_hist since iterative calls to overrides are not necessary.

The issue occurs from attempting to access elm.getValue() of a null blpapi.element.Element. I have not previously seen response types of the form

DVD_HIST = {
    Declared Date = 1986-01-01
    Ex-Date = 1986-04-11
    Record Date = 1986-04-14
    Payable Date = 
    Dividend Amount = 0.280000
    Dividend Frequency = "Semi-Anl"
    Dividend Type = "Interim"
}

which is causing an issue during unpacking.

for elm in field.elements():
    mfld = fld + ":" + str(elm.name())
    dataj = [ticker, mfld, elm.getValue()]
    dataj.extend(corrId)
    data.append(dataj)

Something like this would resolve the issue I believe

for elm in field.elements():
    mfld = fld + ":" + str(elm.name())
    if not elm.isNull():
        val = elm.getValue()
    else:
        val = pd.np.NaN
    dataj = [ticker, mfld, val]
    dataj.extend(corrId)
    data.append(dataj)

Here is a minimal reproducible example of the issue.

import pdblp

con = pdblp.BCon(debug=True)
con.start()
ovrds = [("DVD_START_DT","19860101"), ("DVD_END_DT", "19870101")]
con.ref("101 HK EQUITY", "DVD_HIST", ovrds=ovrds)

DEBUG:root:Sending Request:
 ReferenceDataRequest = {
    securities[] = {
        "101 HK EQUITY"
    }
    fields[] = {
        "DVD_HIST"
    }
    overrides[] = {
        overrides = {
            fieldId = "DVD_START_DT"
            value = "19860101"
        }
        overrides = {
            fieldId = "DVD_END_DT"
            value = "19870101"
        }
    }
}

DEBUG:root:Message Received:
 ReferenceDataResponse = {
    securityData[] = {
        securityData = {
            security = "101 HK EQUITY"
            eidData[] = {
            }
            fieldExceptions[] = {
            }
            sequenceNumber = 0
            fieldData = {
                DVD_HIST[] = {
                    DVD_HIST = {
                        Declared Date = 1986-09-26
                        Ex-Date = 1986-10-31
                        Record Date = 1986-11-01
                        Payable Date = 1986-11-17
                        Dividend Amount = 0.720000
                        Dividend Frequency = "Semi-Anl"
                        Dividend Type = "Final"
                    }
                    DVD_HIST = {
                        Declared Date = 1986-01-01
                        Ex-Date = 1986-04-11
                        Record Date = 1986-04-14
                        Payable Date = 
                        Dividend Amount = 0.280000
                        Dividend Frequency = "Semi-Anl"
                        Dividend Type = "Interim"
                    }
                }
            }
        }
    }
}

[matthewgilbert]

If you update to the newest version of master from Github this should resolve the issue.