Pulling weather forecast and other weather data using the bsrch function

[rcarvalho-ea]

I am using the bsrch function is excel to pull weather data

=BSRCH("comdty:weather","provider=wsi","location=KNYC","model=actuals","target_start_date=2019-01-01","target_end_date=2019-05-10","fields=TEMPERATURE")

It does its job and pulls the data.

However, I am trying to pull the same data using the python api. Assuming that con.bsrch() functions in a similar fashion, I tried doing the same in python but that doesn't work

Even the following code returns an empty data frame

con.bsrch("comdty:weather")

However, the following example returns a list of tickers

con.bsrch("COMDTY:NGFLOW")

I am not sure why. Can someone please help me with this issue

Thanks!

[matthewgilbert]

Unfortunately, as discussed in https://github.com/matthewgilbert/pdblp/issues/14 this service is not documented by Bloomberg, there doesn't appear to be any information in the Bloomberg Developer Guide. Your best bet would be to set con.debug=True to try and make sense of the Bloomberg messages being returned. Unfornately if nothing is returned this has to do with the backend, so your best best would be contacting Bloomberg about this.

The only documentation I have on this is from printing the service, which shows the schema below

 Service //blp/exrsvc
    Operations = {
        OPERATION ExcelGetGridRequest = {
            Request = {
                ELEMENT ExcelGetGridRequest {
                    DESCRIPTION 
                    MIN VALUES 0
                    MAX VALUES 1
                    TYPE ExcelGetGridRequest (SEQUENCE) {
                        DESCRIPTION seqExcelRequest
                        ELEMENT Domain {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 1
                            TYPE STRING
                        }
                        ELEMENT Overrides {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 18446744073709551615
                            TYPE SingleOverride (SEQUENCE)  [] {
                                DESCRIPTION seqSingleOverride
                                ELEMENT name {
                                    DESCRIPTION 
                                    MIN VALUES 0
                                    MAX VALUES 1
                                    TYPE STRING
                                }
                                ELEMENT value {
                                    DESCRIPTION 
                                    MIN VALUES 0
                                    MAX VALUES 1
                                    TYPE STRING
                                }
                            }
                        }
                    }
                } 
            } // End Request
            Responses = {
                ELEMENT GridResponse {
                    DESCRIPTION 
                    MIN VALUES 0
                    MAX VALUES 1
                    TYPE DataRecordSet (SEQUENCE) {
                        DESCRIPTION seqDataRecordSet
                        ELEMENT NumOfFields {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 1
                            TYPE INT32
                        }
                        ELEMENT NumOfRecords {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 1
                            TYPE INT32
                        }
                        ELEMENT ColumnTitles {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 18446744073709551615
                            TYPE STRING [] 
                        }
                        ELEMENT DataRecords {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 18446744073709551615
                            TYPE DataRecord (SEQUENCE)  [] {
                                DESCRIPTION seqDataRecord
                                ELEMENT DataFields {
                                    DESCRIPTION 
                                    MIN VALUES 0
                                    MAX VALUES 18446744073709551615
                                    TYPE DataField (CHOICE)  [] {
                                        DESCRIPTION choiceDataField
                                        ELEMENT IntValue {
                                            DESCRIPTION 
                                            MIN VALUES 0
                                            MAX VALUES 1
                                            TYPE INT32
                                        }
                                        ELEMENT LongValue {
                                            DESCRIPTION 
                                            MIN VALUES 0
                                            MAX VALUES 1
                                            TYPE INT64
                                        }
                                        ELEMENT StringValue {
                                            DESCRIPTION 
                                            MIN VALUES 0
                                            MAX VALUES 1
                                            TYPE STRING
                                        }
                                        ELEMENT FloatValue {
                                            DESCRIPTION 
                                            MIN VALUES 0
                                            MAX VALUES 1
                                            TYPE FLOAT32
                                        }
                                        ELEMENT DoubleValue {
                                            DESCRIPTION 
                                            MIN VALUES 0
                                            MAX VALUES 1
                                            TYPE FLOAT64
                                        }
                                        ELEMENT DateValue {
                                            DESCRIPTION 
                                            MIN VALUES 0
                                            MAX VALUES 1
                                            TYPE DATE
                                        }
                                        ELEMENT TimeValue {
                                            DESCRIPTION 
                                            MIN VALUES 0
                                            MAX VALUES 1
                                            TYPE TIME
                                        }
                                        ELEMENT DateTimeValue {
                                            DESCRIPTION 
                                            MIN VALUES 0
                                            MAX VALUES 1
                                            TYPE DATETIME
                                        }
                                    }
                                }
                            }
                        }
                        ELEMENT ReachMax {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 1
                            TYPE BOOL
                        }
                        ELEMENT Error {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 1
                            TYPE STRING
                        }
                        ELEMENT SequenceNumber {
                            DESCRIPTION 
                            MIN VALUES 0
                            MAX VALUES 1
                            TYPE INT32
                        }
                    }
                } 
            } // End Responses
        } // End OPERATION
    } // End Operations
 } // End Service

[butterflywaltz]

I think the schema is clear. All optional parameters go into the overrides element array.

The requested formula corresponds to below request:

ExcelGetGridRequest = {
    Domain = "COMDTY:Weather"
    Overrides[] = {
        Overrides = {
            name = "provider"
            value = "wsi"
        }
        Overrides = {
            name = "location"
            value = "KNYC"
        }
        Overrides = {
            name = "model"
            value = "actuals"
        }
        Overrides = {
            name = "target_start_date"
            value = "2019-01-01"
        }
        Overrides = {
            name = "target_end_date"
            value = "2019-05-10"
        }
        Overrides = {
            name = "fields"
            value = "TEMPERATURE"
        }
    }
}

The limit in another post can be set in the same way.

And you will get below response:

GridResponse = {
    NumOfFields = 2
    NumOfRecords = 3116
    ColumnTitles[] = {
        "Reported Time", "Temperature (°C)"
    }
    DataRecords[] = {
        DataRecords = {
            DataFields[] = {
                DataFields = {
                    DateTimeValue = 2019-01-01T00:00:00.000+00:00
                }
                DataFields = {
                    DoubleValue = 6.700000
                }
            }
        }
        DataRecords = {
            DataFields[] = {
                DataFields = {
                    DateTimeValue = 2019-01-01T01:00:00.000+00:00
                }
                DataFields = {
                    DoubleValue = 6.700000
                }
            }
        }
...
...
        DataRecords = {
            DataFields[] = {
                DataFields = {
                    DateTimeValue = 2019-05-10T23:00:00.000+00:00
                }
                DataFields = {
                    DoubleValue = 22.200000
                }
            }
        }
    }
    ReachMax = true
    SequenceNumber = 0
}

[rcarvalho-ea]

Thanks for the help!

[lawrencedarby]

Hi can someone please provide an example of the exact code with pdblp to retrieve weather data historical and forecast?

[rcarvalho-ea]

I have modified the original bsrch function so that the dict of overrides and it's values can be passed through it and then reinstalled the library. Here is the modified version

 def bsrch_weather(self, domain, dict_overrides):
        """
        This function uses the Bloomberg API to retrieve 'bsrch' (Bloomberg
        SRCH Data) queries. Returns list of tickers.

        Parameters
        ----------
        domain: string
            A character string with the name of the domain to execute.
            It can be a user defined SRCH screen, commodity screen or
            one of the variety of Bloomberg examples. All domains are in the
            format <domain>:<search_name>. Example "COMDTY:NGFLOW"

        Returns
        -------
        data: pandas.DataFrame
            List of bloomberg tickers from the BSRCH
        """
        logger = _get_logger(self.debug)
        request = self.exrService.createRequest('ExcelGetGridRequest')
        request.set('Domain', domain)

        ors=request.getElement('Overrides')

        for nm in list(dict_overrides.keys()):
            ovrd=ors.appendElement()
            ovrd.setElement('name',nm)
            ovrd.setElement('value',dict_overrides[nm])

        logger.info('Sending Request:\n{}'.format(request))
        self._session.sendRequest(request, identity=self._identity)
        data_time_stamp = []
        data_value=[]
        for msg in self._receive_events(to_dict=False):
            for v in msg.getElement("DataRecords").values():
                for f in v.getElement("DataFields").values():
                    try:
                        data_time_stamp.append(f.getElementAsDatetime("DateTimeValue"))
                    except:
                        data_value.append(f.getElementAsFloat("DoubleValue"))

        return pd.DataFrame({'time_stamp': data_time_stamp,'value':data_value})

Example:

 dict_overrides={"provider":"wsi",
                           "location":loc,
                           "model":model,
                           "target_start_date":start_date,
                           "target_end_date":end_date,
                           "fields":fields,
                           }

con.bsrch_weather("COMDTY:Weather",dict_overrides)

[lawrencedarby]

This is working for me. Thank you kindly !

[loschnauer]

Thanks for sharing!

Might be worth replacing the return statement with something like the below to allow for multiple "fields" to be returned

fields = dict_overrides['fields'].split('|')
fields_value = np.reshape(data_value, (-1, len(fields))).T
data = dict(time_stamp=data_time_stamp, **{k: v for k, v in zip(fields, fields_value)})
return pd.DataFrame(data)

[lawrencedarby]

Hi - I am currently getting an error now with AttributeError: 'BCon' object has no attribute 'bsrch_weather'

I was wondering if anyone knew how this had changed?

[loschnauer]

I think the bsrch_weather function was never part of the bdblp source repo... Presumably you've reinstalled the package and thereby deleted the function from your modified copy? Should work again if you copy the code back into the BCon class of your installation,

[lawrencedarby]

Yea - i had to copy past some mod code. Got if working now !

On Thu, 10 Feb 2022 at 17:39, loschnauer @.***> wrote:

I think the bsrch_weather function was never part of the bdblp source repo... Presumably you've reinstalled the package and thereby deleted the function from your modified copy? Should work again if you copy the code back into the BCon class of your installation,

— Reply to this email directly, view it on GitHub https://github.com/matthewgilbert/pdblp/issues/68#issuecomment-1035140034, or unsubscribe https://github.com/notifications/unsubscribe-auth/APEIUNDAMWGOVJSL4SGDL5DU2PS2LANCNFSM4IPDJCBQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[jonasbo96]

@rcarvalho-ea how can you also extract IntValue from the code you posted?

[jonasbo96]

@matthewgilbert maybe you know if there is a getElementAsInt function or so? As far as I’ve searched I haven’t come across one…

[matthewgilbert]

You could take a look at this way of generically parsing responses

https://github.com/matthewgilbert/pdblp/blob/master/pdblp/pdblp.py#L721

Potentially using the generic .getValue() would work but I haven't looked at this in some time. Barring that you can try looking at the developper docs here

https://data.bloomberglp.com/professional/sites/10/2017/03/BLPAPI-Core-Developer-Guide.pdf