pydata / pandas-datareader

Extract data from a wide range of Internet sources into a pandas DataFrame.
https://pydata.github.io/pandas-datareader/stable/index.html
Other
2.95k stars 681 forks source link

STDERR is used instead of an exception #988

Closed cmpunches closed 9 months ago

cmpunches commented 9 months ago

This issue seems to be pervasive through the library but can be demonstrated with the below:

import pandas_datareader.data as web

fetch = web.get_data_yahoo( "AACI", 1577854800, 1609390800)

It'll produce the following:

1 Failed download:
['AACI']: Exception("%ticker%: Data doesn't exist for startDate = 1577854800, endDate = 1609390800")
[*********************100%%**********************]  1 of 1 completed

The error output is printed to STDERR instead of throwing an exception for the application consuming the library to handle.

This means that the library currently hijacks STDERR for consuming applications to report errors instead of leaving the control over how errors are handled and represented by the consuming application. A library should never write to stdout and stderr unless its functions are specifically to handle writing to those and the consuming application called those functions to create that behaviour.

This also has very big implications for flow control and error detection in consuming logic.

cmpunches commented 9 months ago

This should probably be closed unless pandas_datareader is receptive to doing some workaround things to suppress yfinance output to stderr and check for return sizes and similar to handle its behaviour. This is a bug in yfinance and has been filed over there:

https://github.com/ranaroussi/yfinance/issues/1863