the service is not working

[jirisarri10]

Describe the bug when you do this (last version 2.3.2) import yahooquery

ticker=yahooquery.Ticker('AAPL') ticker.summary_detail {'error': 'HTTP 404 Not Found. Please try again'}

Version python 3.11.5

I'm very grateful for your work and dedication to this impeccable project Thank very much, Best Regards, Javier Irisarri

[RudyNL]

Same here. So its probably a problem caused by Yahoo. It started about Oct 19, 22:00 CET. So the situation is unchanged for about 16 hours.

[RudyNL]

yfinance has the same problem .info returns 404 on all tickers

[james-stevens]

Occasionally this has been a regional thing so thought I'd mention I'm in the UK and I've been seeing this issue since about 21:00 yesterday :(

$ cat yq.py
#! /usr/bin/python3

import json

from yahooquery import Ticker

tickers = Ticker(["TSLA","AAPL"])
print(json.dumps(tickers.price,indent=3))

$ ./yq.py
{
   "error": "HTTP 404 Not Found.  Please try again"
}

[0zd3m1r]

Same here (Germany).

[AlessandroGianfelici]

It's an inssue on Yahoo itself. Apparently the API is not working and they're trying to fix it:

https://query2.finance.yahoo.com/v6/finance/quoteSummary/ENI.MI

[james-stevens]

yfinance has the same problem .info returns 404 on all tickers

according to that thread, using a proxy/VPN via the USA works

[jirisarri10]

Thank you for your email. I am still receiving the same message as last Friday. I am making the query from Spain, and the Yahoo service is not working. If I switch to a United States IP, the service works without any issues. Sending warm regards. Javier Irisarri

l dom, 22-10-2023 a las 02:01 -0700, galashour escribió:

think issue got 'resolved' (experienced the issues since Friday, but as of a few minutes ago (Sunday, Oct 22, 2023) seems to work) as it used to (without any change on my side).

[galashour]

yes- earlier today it worked for a few minutes, and then stopped working again (non US IP), thus I 'deleted' my previous message that turned out to be premature .. ).

[caliskanuzay]

It is down, same here in Turkey. I have a question as a beginner: does changing my chrome IP address change my IP address for python API calls into yahooquery too? Or are they different things?

[kallavikas]

I have java code , which is working and yahooquery in python is not working seems something we need to add in python library. Will share the updates once I figure it out.

[fredrik-corneliusson]

I posted this observation in yfinance discussion but it is probably relevant for yahooquery as well.

[ms82494]

I posted this observation in yfinance discussion but it is probably relevant for yahooquery as well.

Thanks for the suggestion, @fredrik-corneliusson . I am actually in the US, but as of yesterday I also got 404 errors. Your workaround suggestion continues to work for me (for now).

[0zd3m1r]

I posted this observation in yfinance discussion but it is probably relevant for yahooquery as well.

you described works for me for the moment.

[jfmatth]

odd, some tickers work others don't :(

[iamjaymaxx]

Australia-based responses interesting. No modules working at all with Aussie-based VPN. Switched to LA and single modules were working yesterday, but this morning not working. Moved to NY VPN and works at single module level but neither of all_modules() nor get_modules() works - 404 error persists. Not sure how to interpret this other than think it may be IP-related on yahoo end as suggested above. Weird though.

Oh, meant to add, that streamlit app works just fine no matter what the VPN connection.

[Sciguymjm]

https://github.com/ranaroussi/yfinance/pull/1657 <- possible fix

[iamjaymaxx]

ranaroussi/yfinance#1657 <- possible fix

Thx for tip. I sort of get the fix and how it might work. That said, can anyone add how the fix can be incorporated into yahooquery standard python code calls. Solution seems to be based on making http calls directly rather than using the yq modules. Given I'm not familiar with the http approach, it doesn't directly address the yq issues being faced. If there's a way then it would save a lot of reworking of code ...

[Sciguymjm]

https://query2.finance.yahoo.com/v7/finance/quote gives you this: https://bit.ly/yahoo-finance-api-feedback

[iamjaymaxx]

Yes, I've seen that. What you are effectively suggesting is that yahooquery is now 'dead' or have i misunderstood?
I guess the question then arises for @dpguthrie about whether that is the case or not. But TBH if one can get around it via http python response coding for quote module then I'd be surprised if the author can't find a solution that covers the full range of yq modules.

[yalcindavid]

Hopefully i develope from scratch the webscrapping first time i use it is down ! Fucking dead !

[RudyNL]

Reading https://bit.ly/yahoo-finance-api-feedback makes me wondering why access by US IP-addresses is still working.

We’re sorry for the inconvenience, but API-level access to Yahoo Finance quotes data has been disabled.

Is it a mistake by Yahoo forgetting to block USA IP-adresses?

I have installed Proton VPN on my Linux computer and opened a free Proton VPN account. By Proton VPN, location USA, I can use yahooquery from the Netherlands. On a daily basis I am downloading rather extended information of about 1000 selected stocks and ETF's from European and North-American stock exchanges. Everything is running fine.

As I understood, yfinance is a kind of webcrawler interpreting webpages. They should not have problems with the policy change of Yahoo. I expect that yfinance will overcome the problems.

[iamjaymaxx]

ranaroussi/yfinance#1657 <- possible fix

This is helpful as I said but only covers data in quote response. What would code in particular the urls and params etc look like for other data eg financials, key-statistics etc that isn't included in quote responses.?

[RudyNL]

I am wondering where this document comes from and when its dated https://bit.ly/yahoo-finance-api-feedback Notice that there is already a paid Premium version of the Yahoo API. It looks like the document is written before the Premium version was introduced. I can't find the text of the document elsewhere on the internet. I am having the feeling that it is an old forgotten document at Google Docs. Can anyone find the date of the document?

A question. Is there a Premium user and does the Premium user also have problems? I don't think Yahoo would like to lose non US Premium users.

Has anyone considered to use a proxy server? Documentation: https://yahooquery.dpguthrie.com/guide/keyword_arguments/#proxies

[RudyNL]

A workaround for Linux users

Simply add this line to your /etc/hosts file

69.147.92.11 query2.finance.yahoo.com

This means its no problem of Yahoo, but a network problem. The resolving of the hostname into an IP address has problems. This can be related to the regional servers of Yahoo and the regional dependency of the resolving of hostnames. This would explain the location dependency of the problem.

Thanks to @0zd3m1r in .info returns 404 on all tickers

[jirisarri10]

Wow. I am going to try this. But i can not try right now You are a genius. Thank you very much.

El jue, 26 oct 2023 14:19, RudyNL @.***> escribió:

A workaround for Linux users

Simply add this line to your /etc/hosts file

69.147.92.11 query2.finance.yahoo.com

This means its no problem of Yahoo, but a network problem. The resolving of the hostname into an IP address has problems. This can be related to the regional servers of Yahoo and the regional dependency of the resolving of hostnames. This would explain the location dependency of the problem.

Thanks to @0zd3m1r https://github.com/0zd3m1r in .info returns 404 on all tickers https://github.com/ranaroussi/yfinance/issues/1729

— Reply to this email directly, view it on GitHub https://github.com/dpguthrie/yahooquery/issues/224#issuecomment-1781010141, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIWJGU274QTN4COZA3UD5Z3YBJIOBAVCNFSM6AAAAAA6IUSUTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBRGAYTAMJUGE . You are receiving this because you authored the thread.Message ID: @.***>

[lucas03]

This means its no problem of Yahoo, but a network problem.

it might just mean that there is a old server where it still works, but might stop working in the future

[yalcindavid]

Until that yahoo resolve the problem you can use google colab -> IDE of google to code on python it's notebook where you can code in python with no worries about library and if you want to install yahoo query at the top of your code on collab write

!pip install yahooquery

And enjoy!

[dpguthrie]

2.3.3 is now live in PyPi. Give it a try. I imagine that there will still be some edge cases that this doesn't solve for, but was hoping to get the majority of you unblocked.

[ms82494]

I'm getting the following error with 2.3.3 (this is on a MacBook Air M1, chrome and webdriver version are both 118):

In [1]: import os

In [2]: import yahooquery as yq

In [3]: YUSER = os.getenv('YUSER')

In [4]: YPASS = os.getenv('YPASS')

In [5]: ticker = 'TSLA'

In [6]: yqConn = yq.Ticker(ticker, username=YUSER, password=YPASS)
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[6], line 1
----> 1 yqConn = yq.Ticker(ticker, username=YUSER, password=YPASS)

File ~/opt/anaconda3/lib/python3.9/site-packages/yahooquery/ticker.py:90, in Ticker.__init__(self, symbols, **kwargs)
     89 def __init__(self, symbols, **kwargs):
---> 90     super(Ticker, self).__init__(**kwargs)
     91     self.symbols = symbols
     92     self.invalid_symbols = None

File ~/opt/anaconda3/lib/python3.9/site-packages/yahooquery/base.py:934, in _YahooFinance.__init__(self, **kwargs)
    932 password = os.getenv("YF_PASSWORD") or kwargs.get("password")
    933 if username and password:
--> 934     self.login(username, password)

File ~/opt/anaconda3/lib/python3.9/site-packages/yahooquery/base.py:984, in _YahooFinance.login(self, username, password)
    983 def login(self, username, password):
--> 984     ys = YahooSelenium(username=username, password=password)
    985     d = ys.yahoo_login()
    986     try:

File ~/opt/anaconda3/lib/python3.9/site-packages/yahooquery/login.py:31, in YahooSelenium.__init__(self, username, password)
     28 chrome_options.add_argument("--ignore-certificate-errors")
     29 chrome_options.add_argument("--ignore-ssl-errors")
     30 self.driver = webdriver.Chrome(
---> 31     service=ChromeService(ChromeDriverManager().install()),
     32     options=chrome_options,
     33 )

File ~/opt/anaconda3/lib/python3.9/site-packages/webdriver_manager/chrome.py:39, in ChromeDriverManager.install(self)
     38 def install(self) -> str:
---> 39     driver_path = self._get_driver_binary_path(self.driver)
     40     if all(test_os not in driver_path for test_os in ["mac_arm64", "mac_x64"]):
     41         os.chmod(driver_path, 0o755)

File ~/opt/anaconda3/lib/python3.9/site-packages/webdriver_manager/core/manager.py:33, in DriverManager._get_driver_binary_path(self, driver)
     30 if binary_path:
     31     return binary_path
---> 33 file = self._download_manager.download_file(driver.get_driver_download_url())
     34 binary_path = self._cache_manager.save_file_to_cache(driver, file)
     35 return binary_path

File ~/opt/anaconda3/lib/python3.9/site-packages/webdriver_manager/drivers/chrome.py:59, in ChromeDriver.get_driver_download_url(self)
     56 if os_type in ["mac_64", "mac64_m1", "mac_arm64"]:
     57     os_type = "mac-arm64"
---> 59 modern_version_url = self.get_url_for_version_and_platform(driver_version_to_download, os_type)
     60 log(f"Modern chrome version {modern_version_url}")
     61 return modern_version_url

File ~/opt/anaconda3/lib/python3.9/site-packages/webdriver_manager/drivers/chrome.py:94, in ChromeDriver.get_url_for_version_and_platform(self, browser_version, platform)
     91             if d["platform"] == platform:
     92                 return d["url"]
---> 94 raise Exception(f"No such driver version {browser_version} for {platform}")

Exception: No such driver version 118.0.5993.117 for mac-arm64

In [7]: 

[RudyNL]

@dpguthrie Still the same problems with version 2.3.3. With the line 69.147.92.11 query2.finance.yahoo.com added to the /etc/hosts file its running fine. Conclusion, version 2.3.3 is good, but it isn't solving the problem.

[galashour]

I tried (non US IP) version 2.3.3 with a few tickers, and it seems to work (just had time to experiment with a few tickers though).

[RudyNL]

@dpguthrie There was a problem with my libraries. From Europe I got version 2.3.3 running without problems. Only I got a bunch of warnings from deprecated stuff. Tomorrow I will try to get rid of the warnings.

[dpguthrie]

Ya, let me know what warnings you're getting. We can prolly get something released pretty quickly to fix this. I knew that this wouldn't be a 100% fix but may unblock some people and we can iterate from here.

[galashour]

@dpguthrie There was a problem with my libraries. From Europe I got version 2.3.3 running without problems. Only I got a bunch of warnings from deprecated stuff. Tomorrow I will try to get rid of the warnings.

hmm, I got the same warnings but they seemed to be related to pandas, so I figured I probably need to prune my code better to use iloc in some cases that pandas in previous versions were more forgiving, and possibly the upgrade to 2.3.3 also implied a more updated version of pandas.

[RudyNL]

@dpguthrie The upgrade of yahooquery also implied an upgrade of pandas. pandas is generating warnings for deprecated stuff. The function fillnashould be replaced by bfill or ffill. I have also the iloc problem @galashour mentioned. This is a problem of my program and not of yahooquery.

There is another problem. Now yahooquery isn't so stable anymore. By repeating the running of the same program I am getting different results. The result of the functions price, quote_type, summary_detail , asset_profile is sometimes the string 'Invalid Cookie'. I am also sometimes catching exceptions of the type KeyError for the value AAPL or any other stock in case of history.

[Marcel2059]

Not sure if this helps, but 404 error goes away when I select New York from my VPN. If another USA location is used then it does not work.

[dpguthrie]

This feels a bit like playing whack-a-mole - the problems that someone experiences will be different based on where they are and (I'm guessing) how often they're hitting the APIs. While this unblocks some people in that it sets both cookies and a crumb on the request, it also serves as a way to identify those requests for YF and makes it easy to block them and then return either "Invalid Cookie", 404s, invalid data like 'For input string: "-9882999.99999999"', or something else.

I'm sure there's a better way to do all of this, so if you have a suggestion, I'm all ears.

[Marcel2059]

Yes, You are correct about "the problems that someone experiences will be different based on where they are and (I'm guessing) how often they're hitting the APIs." After a while New York was generating the error. After tying vpn connections to other cities and countries, I eventually found another one that works. Don't have any suggestions, but I do wonder why yh offers and api, but discourage its use. They could just remove it. But they have not done so. So I assume there must some benefits for them.

[ms82494]

I was able to fix the Exception: No such driver version 118.0.5993.117 for mac-arm64 error by upgrading webdriver-manager to version 4.0.1 using the command pip install --upgrade webdriver-manager. The upgrade to yahooquery 2.3.3 seems to impose a requirement for webdriver-manager <4.0.0 which didn't work for me. From pip output during yahooquery install:

  Obtaining dependency information for webdriver-manager<4.0.0,>=3.8.6 from https://files.pythonhosted.org/packages/7d/47/a719c25b777275d96e24c4f7f3612db05de4d077400fb945bffef58819c7/webdriver_manager-3.9.1-py2.py3-none-any.whl.metadata
  Downloading webdriver_manager-3.9.1-py2.py3-none-any.whl.metadata (11 kB)

I believe the actually required version of webdriver-manager is 4.0.1. With that installed, yahooquery 2.3.3 works for me without adding extra lines to the hosts file. However, while supplying a username and password to Ticker provides access to the "premium" modules it seems to precludes access to the "non-premium" modules. Here's an example:

In [2]: import os

In [3]: import yahooquery as yq

In [4]: YUSER, YPASS = operator.itemgetter('YUSER', 'YPASS')(os.environ)

In [5]: ticker = 'AAPL'

In [6]: yqConn = yq.Ticker(ticker, username=YUSER, password=YPASS)

In [7]: yqConn.p_income_statement(frequency='q', trailing=False)
Out[7]: 
         asOfDate periodType currencyCode  ...  TotalRevenue  TotalUnusualItems  TotalUnusualItemsExcludingGoodwill
symbol                                     ...                                                                     
AAPL   1985-09-30         3M          USD  ...  4.097000e+08                NaN                                 NaN
AAPL   1985-12-31         3M          USD  ...  5.339000e+08                NaN                                 NaN
AAPL   1986-03-31         3M          USD  ...  4.089000e+08                NaN                                 NaN
AAPL   1986-06-30         3M          USD  ...  4.483000e+08                NaN                                 NaN
AAPL   1986-09-30         3M          USD  ...  5.108000e+08                NaN                                 NaN
...           ...        ...          ...  ...           ...                ...                                 ...
AAPL   2022-06-30         3M          USD  ...  8.295900e+10                NaN                                 NaN
AAPL   2022-09-30         3M          USD  ...  9.014600e+10                NaN                                 NaN
AAPL   2022-12-31         3M          USD  ...  1.171540e+11                NaN                                 NaN
AAPL   2023-03-31         3M          USD  ...  9.483600e+10                NaN                                 NaN
AAPL   2023-06-30         3M          USD  ...  8.179700e+10                NaN                                 NaN

[152 rows x 54 columns]

In [8]: yqConn.price
Out[8]: {'AAPL': 'Invalid Crumb'}

In [9]: yqConn = yq.Ticker(ticker)

In [10]: yqConn.price
Out[10]: 
{'AAPL': {'maxAge': 1,
  'preMarketSource': 'FREE_REALTIME',
  'postMarketChangePercent': -0.00118954,
  'postMarketChange': -0.200104,
  'postMarketTime': '2023-10-27 16:59:49',
  'postMarketPrice': 168.02,
  'postMarketSource': 'FREE_REALTIME',
  'regularMarketChangePercent': 0.007969332,
  'regularMarketChange': 1.3300018,
  'regularMarketTime': '2023-10-27 13:00:01',
  'priceHint': 2,
  'regularMarketPrice': 168.22,
  'regularMarketDayHigh': 168.96,
  'regularMarketDayLow': 166.84,
  'regularMarketVolume': 58499129,
  'regularMarketPreviousClose': 166.89,
  'regularMarketSource': 'FREE_REALTIME',
  'regularMarketOpen': 166.91,
  'exchange': 'NMS',
  'exchangeName': 'NasdaqGS',
  'exchangeDataDelayedBy': 0,
  'marketState': 'CLOSED',
  'quoteType': 'EQUITY',
  'symbol': 'AAPL',
  'underlyingSymbol': None,
  'shortName': 'Apple Inc.',
  'longName': 'Apple Inc.',
  'currency': 'USD',
  'quoteSourceName': 'Nasdaq Real Time Price',
  'currencySymbol': '$',
  'fromCurrency': None,
  'toCurrency': None,
  'lastMarket': None,
  'marketCap': 2629985107968}}

In [11]: 

[jirisarri10]

Hello Doug: I dey hope say you dey okay. First of all, I wan thank you for the effort and dedication wey you put into this amazing and valuable project. I just wan let you know say I run into this problem:

requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://query2.finance.yahoo.com/v6/finance/quoteSummary/AMP.MC?modules=financialData&modules=quoteType&modules=defaultKeyStatistics&modules=assetProfile&modules=summaryDetail&ssl=true

Thanks, my guy, and we dey in touch. One love Javier

[RudyNL]

There is also a change in the treatment of capitals in names. Capitalization of names is optional and the name with capitalization is now returning in the result.

>>> Ticker('bol.pa').summary_detail
{'bol.pa': {'maxAge': 1, 'priceHint': 2

or

>>> Ticker('BOL.PA').summary_detail
{'BOL.PA': {'maxAge': 1, 'priceHint': 2

This is a recent improvement of Yahoo. But sometimes its going wrong:

>>> Ticker('BG.VI').summary_detail
{'BG.VI': 'Invalid Cookie'}
>>> Ticker('bg.vi').summary_detail
{'bg.vi': {'maxAge': 1, 'priceHint': 2,

Or the other way round

>>> Ticker('ia.pa').summary_detail
{'ia.pa': 'Invalid Cookie'}
>>> Ticker('AI.PA').summary_detail
{'AI.PA': {'maxAge': 1, 'priceHint': 2,

What is happening here? Is Yahoo converting the capitalization? Should we in case of errors try the other capitalization. Is this behavior stable? I really don´t know.

[jfmatth]

2.3.3 fixed it for me.

[fredrik-corneliusson]

Version 2.3.3 seems to have fixed the issue for me as well. However, I noticed a slight change in the data provided by Yahoo when the asset_profile for a ticker is missing. Previously, it seemed to return None, but now it returns a string. For example: {'GBTC': 'No fundamentals data found for any of the summaryTypes=assetProfile'} I can't be completely sure that it used to return None, but that’s what I recall. After adjusting my script to account for this change, everything appears to be working well with the new version.

Thank you so much for your fantastic work with this library.

[jeremyjpj0916]

2.3.3 fixed it for me.

Same, fixed for my scripting purposes as well. I am based in US.

[james-stevens]

69.147.92.11 query2.finance.yahoo.com

For me, this has worked for a few days, but overnight last night this stopped being a solution for me :(

[chfiii]

2.3.3 is working for me without the hosts addition. I'm in CT, USA

[RudyNL]

Slowly its getting better here. I am using version 2.3.3. I am adapting my personal software to get more reliability.

• catching the exception KeyError
• detecting the result string Invalid Cookie
• detecting result strings of the form No fundamentals data found for any of the summaryTypes=summaryDetail where the last part summaryTypes=summaryDetail is dependent on the function
• same for balance data with the result string Balance Sheet data unavailable where Balance Sheet is dependent on the function
• using capitalized key names only like AAPL
• strict type checking on the results like

  result=yq.Ticker(stock).summary_detail[stock]
  if isinstance(result,dict):

There are still rare complete drop outs of stocks (approximately 1:100); which stocks are affected is random and time dependent and it happens more with incourant stocks. My personal software is 4 times retrying to obtain detected missing data. At this moment the service level of my own software, except for the drop outs, is at the same level as before the problems started.

[james-stevens]

I just upgraded to 2.3.3 and its works for me (UK), without the host addition - I update my prices with a cron job, so if it fails every so often it doesn't matter too much

[caliskanuzay]

Excuse me if this is a very dumb question, but I am a beginner and I don't know the code in order to upgrade to 2.3.3. In python console, "pip install yahooquery" gives me an older version. How do I upgrade to 2.3.3? Thanks a lot for help!

On Wed, Nov 1, 2023, 17:54 James Stevens @.***> wrote:

I just upgraded to 2.3.3 and its works for me (UK) - I update my prices with a cron job, so if it fails every so often it doesn't matter too much

— Reply to this email directly, view it on GitHub https://github.com/dpguthrie/yahooquery/issues/224#issuecomment-1789099767, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7BAGQ35DCIP367H3Z2L3GTYCJPB5AVCNFSM6AAAAAA6IUSUTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGA4TSNZWG4 . You are receiving this because you commented.Message ID: @.***>

[chfiii]

It’s :

pip install -U

From: caliskanuzay @.> Sent: Wednesday, November 1, 2023 4:12 PM To: dpguthrie/yahooquery @.> Cc: chfiii @.>; Comment @.> Subject: Re: [dpguthrie/yahooquery] the service is not working (Issue #224)

Excuse me if this is a very dumb question, but I am a beginner and I don't know the code in order to upgrade to 2.3.3. In python console, "pip install yahooquery" gives me an older version. How do I upgrade to 2.3.3? Thanks a lot for help!

On Wed, Nov 1, 2023, 17:54 James Stevens @.***> wrote:

I just upgraded to 2.3.3 and its works for me (UK) - I update my prices with a cron job, so if it fails every so often it doesn't matter too much

— Reply to this email directly, view it on GitHub https://github.com/dpguthrie/yahooquery/issues/224#issuecomment-1789099767, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7BAGQ35DCIP367H3Z2L3GTYCJPB5AVCNFSM6AAAAAA6IUSUTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGA4TSNZWG4 . You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/dpguthrie/yahooquery/issues/224#issuecomment-1789606244 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACX2GPZVUWVBOGDYHCIQ2MDYCKUJNAVCNFSM6AAAAAA6IUSUTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGYYDMMRUGQ . You are receiving this because you commented.Message ID: @.***>