This library uses STDERR instead of throwing exceptions when failures are encountered

[cmpunches]

Describe Design Problem

This issue seems to be pervasive through the library but can be demonstrated with the below (I use pandas_datareader which wraps this library):

import pandas_datareader.data as web

fetch = web.get_data_yahoo( "AACI", 1577854800, 1609390800)

It'll return successfully while producing the following output:

1 Failed download:
['AACI']: Exception("%ticker%: Data doesn't exist for startDate = 1577854800, endDate = 1609390800")
[*********************100%%**********************]  1 of 1 completed

The error output is printed to STDERR instead of yfinance throwing an exception for the consumer of the yfinance library to handle. This breaks error handling and pollutes my application's output.

Essentially this means that yfinance currently hijacks STDERR for consuming applications so that it can report errors instead of throwing an exception with the same information (using exceptions allows consuming applications to determine how failures are handled or represented).

A library should almost never write to stdout and stderr for errors -- this also has very big implications for flow control and error handling in consuming logic-- whether by an application that uses this library or another library that wraps it (such as pandas_datareader which is a very visible example).

It seems to be littered all through the library, but this code specifically exemplifies the issue:

https://github.com/ranaroussi/yfinance/blob/1d31e7ca016768c5a3f820c29049d3aa25132b7d/yfinance/base.py#L1687

I think these might help clarify the reasoning here:

• the errors that libraries throw are sometimes fatal to the logic of the consuming application and so need to be handled in that logic, which will expect an exception from the called function in a library to catch
• In most cases, an application uses stdout+stderr for its own purposes, at its own level of abstraction to talk to the user. STDOUT and STDERR are categorically for the application to talk, not its libraries. For example, this current STDERR behaviour will completely break CURSES-based user interfaces.
• An exception notifies a consuming developer of a problem, output notifies the user. The user is often abstracted very far away from the library code so this is an essential practice to incorporate and it will have a huge impact on other projects to do this. It's good to log errors, but also important to defer that function to whatever the consumer is using.
• An exception can have the same data as what you're currently sending to logger, it just lets the application handle how or whether to show that to the user, or decide if it can even continue whatever its trying to do -- and that application may be only using your library for something low level in one piece of it that isn't related to the core function of the application that the developer will care about but that the user won't.

[cmpunches]

This problem appears to have always been there, but was exacerbated by the introduction of the use of logger:

https://github.com/ranaroussi/yfinance/issues/1378

[dmoklaf]

Ouch 3 points:

• This is true that errors shall be raised as exceptions when thy happen synchronously within a call of the API client (which is the case of the entire yfinance library, no background task doing things on the side)
• I disagree that replacing prints by logs exacerbates this - if the stderr stream needs to be preserved (eg because there is a terminal UI deployed), then configure the logs to not use it
• My previous issue #1378 was aimed at INFO-level and WARNING-level prints, not errors

So logs should be kept (if their content is useful to the library's author or to the library's user, I have no opinion there as I use yfinance sporadically only), but below ERROR-level only, and errors shall be raised as exceptions indeed

[ValueRaider]

Some context on why exceptions were redirected to logger:

• users didn't want failed fetches to abort their analysis loops
• yf.download() parallel download. Suppose one symbol suddenly delisted, that would prevent returning other symbols' good data.

Argument raise_errors was added to history() to optionally restore exception raising.

[dmoklaf]

That’s great @ValueRaider thx

[cmpunches]

I completely forgot to follow up on this:

After reading these responses to what is clarification on a well understood best practice in library behaviour in February, I abandoned this library entirely and migrated to Alpha Vantage and have had no issues since.

I would suggest to any project maintainers that consume this library to consider doing the same or rewriting this project from scratch so that the clone can be used for serious projects without this type of issue, and without this type of reaction to reports of it.

Hijacking STDERR and STDOUT from a consumed library as default behaviour is just completely unacceptable, in a way obvious to pretty much anyone, and the idea that maintainers on a project this visible don't know that isn't inspiring of future confidence sufficient to trust this project in my own code.

I guess this issue can be closed, I just wanted to leave a breadcrumb for any others that need to do more mature things with their code than write a proof of concept.

[ValueRaider]

No need for that immaturity. I simply provided context.