search

JECSand / yahoofinancials

A powerful financial data module used for pulling data from Yahoo Finance. This module can pull fundamental and technical data for stocks, indexes, currencies, cryptos, ETFs, Mutual Funds, U.S. Treasuries, and commodity futures.
https://pypi.python.org/pypi/yahoofinancials
MIT License
896 stars 214 forks source link

Sleeping and retrying for a long time #185

Open valankar opened 3 months ago

valankar commented 3 months ago

I was getting some long hangs in my script and it boiled down to:

https://github.com/JECSand/yahoofinancials/blob/2f2cc6c09e55361a72144cb94f9ba193266407ba/yahoofinancials/data.py#L565

This is retrying a sleep in a loop for 250 times. I'm not sure why 250 was chosen, but could there be an option to disable this or provide a limit to the timeout?

I see there is a timeout option:

https://github.com/JECSand/yahoofinancials/blob/2f2cc6c09e55361a72144cb94f9ba193266407ba/yahoofinancials/yf.py#L67

but it doesn't seem to apply here?

JECSand commented 3 months ago

@valankar Tbh that was sort of a hack to work around the random errors yahoo finance was returning. When I implemented this, it was an effective way of brute forcing a response.

I'm waiting for Yahoo Finance to become more stable before I completely refactor this library. I suspect they are in the middle of a tech stack overhaul similar to 2017's.

If you have a more effective manner to handle this, I'll be more than happy to review and merge in a PR. I'm slammed with work and won't be able to really address anything on my end until April.