search

JECSand / yahoofinancials

A powerful financial data module used for pulling data from Yahoo Finance. This module can pull fundamental and technical data for stocks, indexes, currencies, cryptos, ETFs, Mutual Funds, U.S. Treasuries, and commodity futures.
https://pypi.python.org/pypi/yahoofinancials
MIT License
896 stars 214 forks source link

.get_key_statistics_data() results in a HTTP 401 code #166

Closed bjosun closed 6 months ago

bjosun commented 6 months ago

Using the latest build 1.18 and get this error: key_statistics_data = yahoo_financials.get_key_statistics_data() File "/Users/bjosun/oe_project/github/cv-venv/lib/python3.8/site-packages/yahoofinancials/yf.py", line 106, in get_key_statistics_data return self.get_clean_data(self.get_stock_tech_data('defaultKeyStatistics'), 'defaultKeyStatistics') File "/Users/bj/env/lib/python3.8/site-packages/yahoofinancials/etl.py", line 593, in get_stock_tech_data return self.get_stock_data(statement_type='keystats', tech_type=tech_type) File "/Users/bj/env/lib/python3.8/site-packages/yahoofinancials/etl.py", line 565, in get_stock_data dict_ent = self._create_dict_ent(self.ticker, statement_type, tech_type, report_name, hist_obj) File "/Users/bj/env/lib/python3.8/site-packages/yahoofinancials/etl.py", line 521, in _create_dict_ent re_data = self._get_historical_data(YAHOO_URL, r_map, tech_type, statement_type) File "/Users/bj/env/lib/python3.8/site-packages/yahoofinancials/etl.py", line 248, in _get_historical_data self._request_handler(url, config.get("response_field")) File "/Users/bj/env/lib/python3.8/site-packages/yahoofinancials/etl.py", line 209, in _request_handler raise ManagedException("Server replied with server error code, HTTP " + str(response.status_code) + yahoofinancials.etl.ManagedException: Server replied with server error code, HTTP 401 code while opening the url: https://query2.finance.yahoo.com/v10/finance/quoteSummary/msft?modules=defaultKeyStatistics&formatted=False&lang=en-US&region=US&corsDomain=finance.yahoo.com&crumb=

Do i need to authenticate against the yahoo server?

JECSand commented 6 months ago

@bjosun Are you running a VPN or anything similar? I can't recreate this on my end.

No you do not need to authenticate. It seems the crumb being passed to the url is an empty string.

bjosun commented 6 months ago

@JECSand No i'm not running it on a vpn, running it on a 23.0.0 Darwin Kernel Version 23.0.0 and Python 3.8.0. And the source ip is in Europe. And your right i don't get the crumb with the new url.

daniel-fcm commented 6 months ago

may i add some findings from my side, while having the same issue mentioned above: YahooFinancials(symbol).get_stock_data() --> this leads to an empty result: {'AAPL': {}, 'dataType': ''} YahooFinancials(symbol).get_financial_stmts('quarterly', 'balance') --> leads to a proper result If it would be VPN related, than we should get same response regardless of the function used, or not?

bjosun commented 6 months ago

may i add some findings from my side, while having the same issue mentioned above: YahooFinancials(symbol).get_stock_data() --> this leads to an empty result: {'AAPL': {}, 'dataType': ''} YahooFinancials(symbol).get_financial_stmts('quarterly', 'balance') --> leads to a proper result If it would be VPN related, than we should get same response regardless of the function used, or not?

Yes, same issue here YahooFinancials(symbol).get_financial_stmts('quarterly', 'balance') works as expected.

JECSand commented 6 months ago

I was able to recreate the issue. Going to check to see if the API changed on Yahoo's end.

JECSand commented 6 months ago

may i add some findings from my side, while having the same issue mentioned above: YahooFinancials(symbol).get_stock_data() --> this leads to an empty result: {'AAPL': {}, 'dataType': ''} YahooFinancials(symbol).get_financial_stmts('quarterly', 'balance') --> leads to a proper result If it would be VPN related, than we should get same response regardless of the function used, or not?

get_stock_data() is a method primarily used by other public methods and returns as an empty response when executed without parameters. This is due to how the yahoo finance url is constructed (based on the function inputs).

In short, when you call get_stock_data like this: yf.get_stock_data(statement_type='profile', tech_type='assetProfile', report_name='assetProfile')

The URL yf will execute will resemble this: https://query1.finance.yahoo.com/v10/finance/quoteSummary/c?modules=assetProfile&formatted=False&lang=en-US&region=US&corsDomain=finance.yahoo.com

However if you call the method without params, i.e. yf.get_stock_data()

The URL generated looks like this: https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/c

Since the url has no parameters, yahoo's API returns no data. This method being public is a vestige from the days when we used web scraping to grab the data. Back then executing this function with no parameters would return data.

I can either make this method private or give it parameter defaults that return useful data.

On a different note I have a fix for yahoo_financials.get_key_statistics_data almost ready.

JECSand commented 6 months ago

Just released v1.19 which includes a fix for this issue.

daniel-fcm commented 6 months ago

i can confirm - it is working. thanks for taking the time on it.

bjosun commented 6 months ago

Thanks for taking the time on it. Downloaded the latest version, managed to get the get_key_statistic_data once but now I get the same error: yahoofinancials.etl.ManagedException: Server replied with server error code, HTTP 401 code while opening the url: https://query1.finance.yahoo.com/v10/finance/quoteSummary/msft?modules=defaultKeyStatistics&formatted=False&lang=en-US&region=US&corsDomain=finance.yahoo.com