vfilimonov / pydatastream

Python interface to the Refinitiv Datastream (former Thomson Reuters Datastream)
MIT License
71 stars 30 forks source link

Hickup on single day request #31

Closed sdittmar closed 1 year ago

sdittmar commented 2 years ago

If I run get_trading_days() on a SINGLE DAY and one country happens to have a holiday, the code gets a hickup around line 340. I was going to use the function for the current day processing only. I now you intended it as mask for historical data, but it would be great if that "feature" could be added...

You can reproduce the error with the following code:

ds.get_trading_days(['US','UK','FR','SW','IT','NL','BD','BG'], date_from='2022-06-06', date_to='2022-06-06')

Here is the trace:

---------------------------------------------------------------------------
DatastreamException                       Traceback (most recent call last)
Input In [38], in <module>
----> 1 trading_days2 = ds.get_trading_days(['US','UK','FR','SW','IT','NL','BD','BG'], date_from='2022-06-06', date_to='2022-06-06')
      2 trading_days2

File ~\Miniconda3\envs\Eikon\lib\site-packages\pydatastream\pydatastream.py:729, in Datastream.get_trading_days(self, countries, date_from, date_to)
    727     raise DatastreamException(f'Unknowns ISO codes: {", ".join(missing_isos)}')
    728 # By default 0 and NaN are returned, so we add 1
--> 729 res = self.fetch(mnems.MNEM, date_from=date_from, date_to=date_to) + 1
    731 if len(countries) == 1:
    732     return res.iloc[:, 0].to_frame(name=countries[0])

File ~\Miniconda3\envs\Eikon\lib\site-packages\pydatastream\pydatastream.py:455, in Datastream.fetch(self, tickers, fields, date_from, date_to, freq, static, IsExpression, return_metadata, always_multiindex)
    453 raw = self.request(req)
    454 self._last_response_raw = raw
--> 455 data, meta = self.parse_response(raw, return_metadata=True)
    457 if static:
    458     # Static request - drop date from MultiIndex
    459     data = data.reset_index(level=1, drop=True)

File ~\Miniconda3\envs\Eikon\lib\site-packages\pydatastream\pydatastream.py:369, in Datastream.parse_response(self, response, return_metadata)
    358 """ Parse raw JSON response
    359 
    360     If return_metadata is True, then result is tuple (dataframe, metadata),
   (...)
    366     (dataframe, metadata), otherwise each element is a dataframe.
    367 """
    368 if 'DataResponse' in response:  # Single request
--> 369     res, meta = self._parse_one(response['DataResponse'])
    370     self.last_metadata = meta
    371     return (res, meta) if return_metadata else res

File ~\Miniconda3\envs\Eikon\lib\site-packages\pydatastream\pydatastream.py:340, in Datastream._parse_one(self, res)
    338 if v['Type'] == 0:  # Error
    339     if self.raise_on_error:
--> 340         raise DatastreamException(f'"{v["Symbol"]}"("{data_type}"): {value}')
    341     res[data_type][v['Symbol']] = math.nan
    342 elif v['Type'] == 4:  # Date

DatastreamException: "S:VACS"(""): $$ER: 0904,NO DATA AVAILABLE
vfilimonov commented 2 years ago

This is an intended behaviour - if there's an error or unexpected situation, it should not be silently swallowed.

Datastream has an attribute raise_on_error which is set to True by default. If it is set to False, all errors, such as this one, should result in NaN being returned in a corresponding field.

See: https://github.com/vfilimonov/pydatastream#note-2-error-catching