fc.yahoo.com not responding

[helphp]

fc.yahoo.com not responding for me. It's used in the setup_session_with_cookies_andcrumb() function in utils/__init_\.py before doing anything else. I temporarily changed line 176 to use the URL https://finance.yahoo.com/quote/TSLA/ instead and it seems to be working for now.

[jirisarri10]

Hello. When I realized the following: import yahooquery ticker=yahooquery.Ticker('ORCL') ticker.price {'ORCL': 'Invalid cookie'} Thank you for your great work!! Javier

[RudyNL]

Same problems in the Netherlands. I had the same results as @jirisarri10 at Decemeber 5 0:30 CET. A VPN to simulate the US location doesn't work. Finally made the change as proposed by @helphp

    try:
        response = session.get('https://finance.yahoo.com/quote/TSLA/')
#        response = session.get('https://fc.yahoo.com')
    except Exception:

in ......../site-packages/yahooquery/utils/__init__.py. My own program is running again but has missing data problems. The test of @jirisarri10 has the same outcome {'ORCL': 'Invalid cookie'}. Its my bedtime, tomorrow I will look at it again.

[0zd3m1r]

yes, yahooquery is currently not working.

; <<>> DiG 9.16.1-Ubuntu <<>> fc.yahoo.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11630
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;fc.yahoo.com.                  IN      A

;; ANSWER SECTION:
fc.yahoo.com.           185     IN      CNAME   sdarlasplitroute.gapx.yahoodns.net.
sdarlasplitroute.gapx.yahoodns.net. 60 IN A     87.248.119.251
sdarlasplitroute.gapx.yahoodns.net. 60 IN A     87.248.119.252

[RudyNL]

The Yahoo servers seem to work again and there are now problems again. My program did run correctly from December 5 6:15 till 8:15 CET in the Netherlands. And now around 8:45 CET I am having the same issues as yesterday. It looks like Yahoo is working on it.

[ibart]

It looks like Yahoo is working on it.

They are working on it since october...

[chfiii]

The patch to line 177 in utils/init.py worked for me today. Thanks for the tip HELPHP!

[RudyNL]

Is the patch really working? For me it isn't doing anything. I am doing the @jirisarri10 test and with or without the patch the result is the same: {'ORCL': 'Invalid cookie'} ticker.history() Is working with and without the patch.

[newspapermann]

not working for me either

[nathanramoscfa]

Same thing happening to me. When I run:

import yahooquery
ticker=yahooquery.Ticker('ORCL')
ticker.price

I get this:

WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='fc.yahoo.com', port=443): Read timed out. (read timeout=5)")': /
WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='fc.yahoo.com', port=443): Read timed out. (read timeout=5)")': /
WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='fc.yahoo.com', port=443): Read timed out. (read timeout=5)")': /
WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='fc.yahoo.com', port=443): Read timed out. (read timeout=5)")': /
WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='fc.yahoo.com', port=443): Read timed out. (read timeout=5)")': /
{'ORCL': 'Invalid Cookie'}

[james-stevens]

Stopped working for me about 19:05 4th Dec UTC - I'm UK based using v2.3.3

$ ./yq.py
{
   "TSLA": "Invalid Cookie",
   "AAPL": "Invalid Cookie"
}
$ cat yq.py
#! /usr/bin/python3

import json

from yahooquery import Ticker

tickers = Ticker(["TSLA","AAPL"])
print(json.dumps(tickers.price,indent=3))

[nathanramoscfa]

I went to fc.yahoo.com in my browser and got this message:

I tried yfinance library to see if maybe that library was having the same problems but it worked with no problems:

import yfinance as yf
orcl = yf.Ticker("ORCL").info
orcl

{'address1': '2300 Oracle Way',
 'city': 'Austin',
 'state': 'TX',
 'zip': '78741',
 'country': 'United States',
 'phone': '737-867-1000',
 'website': 'https://www.oracle.com/',
 'industry': 'Software - Infrastructure',
 'industryKey': 'software-infrastructure',
 'industryDisp': 'Software - Infrastructure',
 'sector': 'Technology',
 'sectorKey': 'technology',
 'sectorDisp': 'Technology',
...

[chfiii]

The patch that HELPHP suggested to change line 177 in yahooquery/utils/init.py from: "https://fc.yahoo.com/" to "https://finance.yahoo.com/quote/TSLA" works fine for me. (in CT, USA, yahooquery version 2.3.3)

I'm using both the price and history modules.

[ibart]

I went to fc.yahoo.com in my browser and got this message:

I tried yfinance library to see if maybe that library was having the same problems but it worked with no problems:

import yfinance as yf
orcl = yf.Ticker("ORCL").info
orcl

{'address1': '2300 Oracle Way',
 'city': 'Austin',
 'state': 'TX',
 'zip': '78741',
 'country': 'United States',
 'phone': '737-867-1000',
 'website': 'https://www.oracle.com/',
 'industry': 'Software - Infrastructure',
 'industryKey': 'software-infrastructure',
 'industryDisp': 'Software - Infrastructure',
 'sector': 'Technology',
 'sectorKey': 'technology',
 'sectorDisp': 'Technology',
...

it used to has... I think they solved with cookie & crumb.

This nightmare shows us that we need to find a more stable source of data. I would like to have this, with a subscription, by Yahoo Finance. Why the hell Yahoo is not understanding this? -_-

[RudyNL]

I am testing at December 5 18:40 CET in the Netherlands. With and without patch it isn't working. But with and without the patch its now working with a VPN into the USA.

My conclusion is that the best thing you can do is to wait. Yahoo is continuously changing the circumstances. Just notice that early this morning I could run a program retrieving data of about 1000 equities within two hours without any problems.

[SC8198]

Can confirm stopped working Dec 4th around 12:30 pm ~ MST

looking again and when I print the output of the tickers i get <yahooquery.ticker.Ticker object at 0x7ff6ff68b950>

[SC8198]

The patch that HELPHP suggested to change line 177 in yahooquery/utils/init.py from: "https://fc.yahoo.com/" to "https://finance.yahoo.com/quote/TSLA" works fine for me. (in CT, USA, yahooquery version 2.3.3)

I'm using both the price and history modules.

May I ask how you patched this? did you clone the repo and install from a directory? Just trying to get my script working thanks.

[nathanramoscfa]

fc.yahoo.com not responding for me. It's used in the setup_session_with_cookies_and_crumb() function in utils/init.py before doing anything else. I temporarily changed line 176 to use the URL https://finance.yahoo.com/quote/TSLA/ instead and it seems to be working for now.

Won't this only pull in data for Tesla (ticker: TSLA)?

[chfiii]

The 'approved' way would be to clone the repo. I hacked quickly by doing a "pip show yahooquery" to get the directory and then just replaced the line.

As for the question about won't is just get Tesla data - no. This call is just to establish the cookie and crumb with yahoo so the session can be setup. I initially thought the same ;)

[itzmk21]

Can confirm, I get the same errors, 5th Dec, 08:11pm GMT(UTC+0), UK based.

[fredrik-corneliusson]

Using the suggested fix by changing URL on line 176 just makes it return "invalid cookie" in the data response.

import yahooquery as yq
tq = yq.Ticker('TSLA')
print(tq.earnings)

Output: {'TSLA': 'Invalid Cookie'}

[quikksilver1]

At least you guys are getting responses back. I am calling the Ticker() class and then the history class and it just stalls out. I am doing this in a vacuum so I know my code is not interfering. Is it possible for them to block someone?

[ibart]

At least you guys are getting responses back. I am calling the Ticker() class and then the history class and it just stalls out. I am doing this in a vacuum so I know my code is not interfering. Is it possible for them to block someone?

you are not banned. In case, you can just try a VPN. The response is the same for all of us:

HTTPSConnectionPool(host='fc.yahoo.com', port=443): Max retries exceeded with url: / (Caused by ReadTimeoutError("HTTPSConnectionPool(host='fc.yahoo.com', port=443): Read timed out. (read timeout=5)"))

Even changing the timeout, the problem persists. I think we are having the cookie and crumbs problem yfinance used to have 1 month ago.

[no0be]

Using the suggested fix by changing URL on line 176 just makes it return "invalid cookie" in the data response.

import yahooquery as yq
tq = yq.Ticker('TSLA')
print(tq.earnings)

Output: {'TSLA': 'Invalid Cookie'}

I confirm I observe the same behavior.

import json
from yahooquery import Ticker
tickers = Ticker(["TSLA"])
print(json.dumps(tickers.price,indent=3))

Output:

{
   "TSLA": "Invalid Cookie"
}

[nathanramoscfa]

I forked yahooquery and made the suggested change to utils/__init__.py:

I can confirm that it is now working as of this morning.

Edit: Actually, I see missing data scattered throughout my dataframes. Still a problem, but now I see yahooquery has been updated, so gonna try the the new upgrade.

[no0be]

I forked yahooquery and made the suggested change to utils/__init__.py:

I can confirm that it is now working as of this morning.

Do you need to do something to rebuild the package or something similar after the change? I'm running inside a venv build with pipenv and just changed the file as you did in .venv/lib/python3.9/site-packages/yahooquery/utils/__init__.py but it still does not work.

[fredrik-corneliusson]

@nathanramoscfa , could you provide a example of the code that was solved by the suggested solution? I suspect this "solution" just hides and/or solves a subset of the API-calls in yahooquery. Is there integration test suite for yahoo query that tests against the live yahoo API we can use to verify what is working and what is broken?

[fredrik-corneliusson]

@nathanramoscfa , could you provide a example of the code that was solved by the suggested solution? I suspect this "solution" just hides and/or solves a subset of the API-calls in yahooquery. Is there integration test suite for yahoo query that tests against the live yahoo API we can use to verify what is working and what is broken?

I just did a quick check, the fix "solves" that the ticker.history() hangs for a while before returning data. Probably due to waiting for fc.yahoo.com to time out. But ticker.history() returns data with and without the "solution", just much slower. Earnings (and I suspect many other request) still returns invalid_cookie. This is probably due to the response.cookies is empty (RequestsCookieJar[]) for the suggested URL "solution": response = session.get('https://finance.yahoo.com/quote/TSLA/').

Test code:

import yahooquery as yq
tq = yq.Ticker('TSLA')
print(tq.history())
print(tq.earnings)

[RudyNL]

@fredrik-corneliusson I am using a private program analyzing about 1000 equities based on yahooquery. You can interpret this program as a stress test. It has been running this morning and yesterday evening after closure of Wall-Street using a VPN (Proton VPN on Linux) to simulate my location in the USA. I don't need the patch and there are no problems. The only difference with normally running is that the data retrieval is about a factor two slower. From the Netherlands without VPN only history() is working. All other methods are leading to error messages.

My conclusion is that the results are very location dependent. The current situation is for me stable for about one and a half day. Before that, the results were also very time dependent.

[fredrik-corneliusson]

Thanks for the info, see my comment I made just before yours for some more details (we posted almost at the same time).

My conclusion is that the results are very location dependent.

This seems to be very similar to the previous issue we had a couple of months ago. My guess is that yahoo rolls out changes to their infrastructure gradually in different regions. If that is correct it is probably just a matter of time before the VPN solution breaks as well.

[pkonduri]

Does anyone know of a long term fix here? Or will yahoo just make this an untenable option?

[dpguthrie]

My guess here is that at some point another issue will be introduced from yahoo's side. It will continue to be like playing whack-a-mole.

With that said, I did just release 2.3.4. Hopefully that helps some of you (tested in a few different areas and was able to get results back from each endpoint...so 🤷 )

[jirisarri10]

Hi Doug: I upgraded yahooquery 2.3.5, however i obtained the same error. I am in Spain. Successfully installed yahooquery-2.3.5  javier  (e) pyenv  ~  python Python 3.11.6 (main, Nov 14 2023, 09:36:21) [GCC 13.2.1 20230801] on linux Type "help", "copyright", "credits" or "license" for more information.

import yahooquery ORCL=yahooquery.Ticker('ORCL') ORCL.price {'ORCL': 'Invalid Cookie'}

As always, thank you very much, and appreciate your commendable work

[RudyNL]

Same here with version 2.3.5. In the Netherlands I still need a VPN to the USA.

[fredrik-corneliusson]

Thanks Doug for your relentless and splendid work! 2.3.5 got it working again from Amazon EC2 instance in US-West region, but not from EU-Stockholm region. So probably region sensitive issue.

[dpguthrie]

For those of you not in the U.S., when you navigate to yahoo finance, what is the site that you end up going to? Is it "https://es.finance.yahoo.com", "https://de.finance.yahoo.com", "https://uk.finance.yahoo.com"...something else? I'm wondering if you pass in a country from here to one where you're located or close by when initializing the Ticker if that helps at all.

import yahooquery as yq

t = Ticker('aapl', country='spain')
t.price

[fredrik-corneliusson]

For those of you not in the U.S., when you navigate to yahoo finance, what is the site that you end up going to?

At least from Sweden I cant see any other URL than the plain finance.yahoo.com and similar. Maybe they control what server is accessed through DNS, last time there was a similar issue in yfinance replacing the URL in the code with U.S.-resolved IP address worked as a temporary solution.

[nathanramoscfa]

I can confirm that the new 2.3.5 update fixed the problem. I earlier said that the previous quick fix of changing the URL to https://finance.yahoo.com/quote/TSLA/ worked, but actually, the data had lots of NaNs. Now, the data populates.

[ibart]

My guess here is that at some point another issue will be introduced from yahoo's side. It will continue to be like playing whack-a-mole.

With that said, I did just release 2.3.4. Hopefully that helps some of you (tested in a few different areas and was able to get results back from each endpoint...so 🤷 )

I agree about the whack - a - mole thing. The strangest thing is understanding what are they doing. Are they banning the use of their deprecated api or not? if yes, why not doing once and forever? if not, why not selling the service? I would like to pay not having this mess daily in my script...

I am looking forward another service BUT I don't like the idea of changing my python script :(

[RudyNL]

I am having the same problems with version 2.3.5 with VPN into the USA. There is a lot of missing data. In my program all equities are checked for missing or wrong data data and in a second and third pass I am trying to retrieve the data again. This normally helps against random problems, but now the missing data is structural.

Version 2.3.5 in Europe doesn't work

>>> import yahooquery
>>> ticker=yahooquery.Ticker('ORCL')
>>> ticker.price
{'ORCL': 'Invalid Cookie'}

For production I am using version 2.3.3 with VPN into the USA and the patch response = session.get('https://finance.yahoo.com') instead of response = session.get('https://fc.yahoo.com') The extension /quote/TSLA/ is superfluous. This is running slow (about a factor two) but fine.

Using the url https://de.finance.yahoo.com instead of https://finance.yahoo.com in the web-browser doesn't seem to make much difference. Only the news feed is in German and partly the information presented is translated into German. In my program I am cycling the location country='spain' hoping to alternate the used Yahoo servers. I never had any benefit of this process.

[jirisarri10]

Hi Doug:

i obtained the same result put the parameter "spain".

As always, Gracias!! Sincerely, Javier

[dpguthrie]

Thanks for testing the country parameter. I was trying to use that as a way to set the host name when obtaining cookies/crumb. Doesn't appear to do anything, so I'll probably remove that piece of code then.

[kensthilaire]

@dpguthrie, I have updated to 2.3.5 and verified that all is well again for my application. I am in the eastern US. Thank you once again for the quick turn on the refactoring to whack this latest mole.

And, thank you for adjusting the poetry file to allow python 3.8. Saved me from having my own version that relaxed the python version.

Ken

[jirisarri10]

Hi Doug: You welcome!! If you need any additional proof, don't hesitate to ask me. Gracias!!! Javier

[jirisarri10]

i debugged the code in Spain for if it helps:

[jirisarri10]

i forget this: When i am over the browser (Firefox) obtain this Token:

However the code shows a error 501:

Thanks

[SC8198]

The 'approved' way would be to clone the repo. I hacked quickly by doing a "pip show yahooquery" to get the directory and then just replaced the line.

As for the question about won't is just get Tesla data - no. This call is just to establish the cookie and crumb with yahoo so the session can be setup. I initially thought the same ;)

THANK YOU!!!

[RudyNL]

There are two related issues at yfinance

1. ReadTimeoutError on fc.yahoo.com
2. Connection refused

[james-stevens]

Sadly, v2.3.5 doesn't solve the problem for me (UK, so prob cos of Brexit) :sob:

...
Successfully installed yahooquery-2.3.5

$ ./yq.py
{
   "TSLA": "Invalid Cookie",
   "AAPL": "Invalid Cookie"
}

$ cat yq.py
#! /usr/bin/python3

import json

from yahooquery import Ticker

tickers = Ticker(["TSLA","AAPL"])
print(json.dumps(tickers.price,indent=3))

For production I am using version 2.3.3 with VPN into the USA and the patch

Considering the number of times its failed outside the US, but not inside, its almost worth renting a VPS in the US to act as a proxy - would def be cheaper than any of the subscription services.

I wouldn't mind paying for a service, the problem I came up against was I couldn't find anything between "100 queries/ month - free" and "10M queries per month $1000". I'm currently running at about 2700/day or about 85K/month - four times an hour for each stock I own.

[ibart]

import yahooquery as yq

t = yq.Ticker('aapl')
t.price 

2.3.6, from Italy it fails: Failed to obtain crumb. Ability to retrieve data will be significantly limited. BUT with a VPN from U.S.A. is working.

[dpguthrie]

Ya, I'm guessing they're just blocking requests outside of the U.S. for some reason. I'll keep trying to figure out a solution to this but at least there's a workaround with VPN/proxies