search

iprak / yahoofinance

Home Assistant component which allows you to get stock updates from Yahoo finance.
MIT License
81 stars 11 forks source link

Yahoo Finance stops working #136

Open seg-on opened 3 weeks ago

seg-on commented 3 weeks ago

Hi, It seems that after updating HA to 2024.10.4, Yahoo Finance stops working. I've tried rebooting HA, rebooting PC but it doesn't help.

iprak commented 3 weeks ago

I need more info.

You can also enable debug level logging to capture more data

  logs:
    custom_components.yahoofinance: debug
seg-on commented 3 weeks ago

Hi, you mean this?

2024-10-29 10:20:23.636 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:20:23.637 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:20:53.640 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:20:53.641 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:21:23.645 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:21:23.645 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:21:53.649 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:21:53.649 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:22:23.654 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:22:23.654 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:22:53.659 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:22:53.660 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:23:23.662 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:23:23.663 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:23:53.665 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:23:53.665 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:24:23.669 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting crumb. 
2024-10-29 10:24:23.669 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds
2024-10-29 10:24:38.671 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Navigating to base Yahoo page
iprak commented 3 weeks ago

Yes, this has happened when the Yahoo website returns different data. Usually due to GDPR consent. What is your location and the url of the Yahoo website (if opened in a browser) ?

seg-on commented 3 weeks ago

Yes, I'm from Slovakia (part of EU). I don't understand what you mean by Yahoo website URL? When I type finance.yahoo.com I get https://finance.yahoo.com/.

iprak commented 3 weeks ago

This is most likely due to something specific in Yahoo content for that region. Seem other folks from EU region have encountered this (https://github.com/iprak/yahoofinance/issues/125). The extension tries to automatically accept the GDPR consent and maybe that is not working. Not sure at this point. This should not be related to 2024.10.4 release.

seg-on commented 3 weeks ago

It is strange, but now at 18:11 it has started to work. I'll be watching that.

vellanix commented 3 weeks ago

Also here (Italy) doesnt work from yesterday afternoon.

Same error: Unable to get crumb

rummepa commented 3 weeks ago

same thing here in Belgium:

Logger: custom_components.yahoofinance Source: custom_components/yahoofinance/init.py:241 integration: Yahoo Finance (documentation, issues) First occurred: October 29, 2024 at 21:03:19 (3155 occurrences) Last logged: 10:17:09 Unable to get crumb, re-trying in 15 seconds

jan-jeroen commented 3 weeks ago

Same here in the Netherlands since last weekend. Unable to get crumb.

iprak commented 3 weeks ago

The release 1.2.11 should help.

jan-jeroen commented 3 weeks ago

The release 1.2.11 should help.

Confirmed! Many thanks

seg-on commented 3 weeks ago

I'm sad. I updated to this version and after that it stopped working :(

iprak commented 3 weeks ago

Can you share debug log?

seg-on commented 3 weeks ago

here you are:

2024-10-31 20:39:45.002 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds
2024-10-31 20:40:15.002 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting data from https://query1.finance.yahoo.com/v7/finance/quote?symbols=EURUSD=X,GC=F,ES=F,MNQ=F,^VIX,CL=F,TTF=F,^GSPC,^NDX,SXR8.DE,BTC-USD,BTC-EUR,XMR-EUR&crumb=FKbLQWwD20B
2024-10-31 20:40:15.002 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds
2024-10-31 20:40:45.002 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting data from https://query1.finance.yahoo.com/v7/finance/quote?symbols=EURUSD=X,GC=F,ES=F,MNQ=F,^VIX,CL=F,TTF=F,^GSPC,^NDX,SXR8.DE,BTC-USD,BTC-EUR,XMR-EUR&crumb=FKbLQWwD20B
2024-10-31 20:40:45.003 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds
2024-10-31 20:41:15.002 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting data from https://query1.finance.yahoo.com/v7/finance/quote?symbols=EURUSD=X,GC=F,ES=F,MNQ=F,^VIX,CL=F,TTF=F,^GSPC,^NDX,SXR8.DE,BTC-USD,BTC-EUR,XMR-EUR&crumb=FKbLQWwD20B
2024-10-31 20:41:15.002 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds
2024-10-31 20:41:45.002 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting data from https://query1.finance.yahoo.com/v7/finance/quote?symbols=EURUSD=X,GC=F,ES=F,MNQ=F,^VIX,CL=F,TTF=F,^GSPC,^NDX,SXR8.DE,BTC-USD,BTC-EUR,XMR-EUR&crumb=FKbLQWwD20B
2024-10-31 20:41:45.002 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds
2024-10-31 20:42:15.002 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting data from https://query1.finance.yahoo.com/v7/finance/quote?symbols=EURUSD=X,GC=F,ES=F,MNQ=F,^VIX,CL=F,TTF=F,^GSPC,^NDX,SXR8.DE,BTC-USD,BTC-EUR,XMR-EUR&crumb=FKbLQWwD20B
2024-10-31 20:42:15.003 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds
2024-10-31 20:42:45.002 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting data from https://query1.finance.yahoo.com/v7/finance/quote?symbols=EURUSD=X,GC=F,ES=F,MNQ=F,^VIX,CL=F,TTF=F,^GSPC,^NDX,SXR8.DE,BTC-USD,BTC-EUR,XMR-EUR&crumb=FKbLQWwD20B
2024-10-31 20:42:45.003 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds
2024-10-31 20:42:47.202 ERROR (MainThread) [custom_components.weatherdotcom.coordinator] Error fetching Weather.com data: TimeoutError()
2024-10-31 20:43:05.001 DEBUG (MainThread) [custom_components.yahoofinance.coordinator] Requesting data from 'https://query1.finance.yahoo.com/v7/finance/quote?symbols=EURUSD=X,GC=F,ES=F,MNQ=F,^VIX,CL=F,TTF=F,^GSPC,^NDX,SXR8.DE,BTC-USD,BTC-EUR,XMR-EUR&crumb=FKbLQWwD20B'
iprak commented 3 weeks ago

@seg-on I think you have some other problem. The log indicates similar failure for weatherdotcom.coordinator data fetch.

Everything seems to check out from my end when using VPN to fake European source.

jan-jeroen commented 2 weeks ago

My problem is back since yesterday, the new version worked for about a day.

2024-11-01 14:04:18.872 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds 2024-11-01 14:04:33.966 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds 2024-11-01 14:04:49.072 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds

iprak commented 2 weeks ago

I am not hopeful of addressing this issue. I am not able to replicate this over VPN. But I have pushed out another release with more logging.

jan-jeroen commented 2 weeks ago

Installed your new release. This is what I get:

2024-11-02 10:01:41.962 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Crumb request responded with status=404 2024-11-02 10:01:41.963 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds 2024-11-02 10:01:57.137 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Crumb request responded with status=404 2024-11-02 10:01:57.138 WARNING (MainThread) [custom_components.yahoofinance] Unable to get crumb, re-trying in 15 seconds

seg-on commented 2 weeks ago

Yahoo Finance ver: 1.2.12

2024-11-02 13:03:30.003 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Timed out getting data from https://query1.finance.yahoo.com/v7/finance/quote?symbols=EURUSD=X,GC=F,ES=F,MNQ=F,^VIX,CL=F,TTF=F,^GSPC,^NDX,SXR8.DE,BTC-USD,BTC-EUR,XMR-EUR&crumb=N1QwNF/tiD5
2024-11-02 13:03:30.004 WARNING (MainThread) [custom_components.yahoofinance.coordinator] Error obtaining data, retrying in 20 seconds
iprak commented 2 weeks ago

I have attempted one more fix. It has been trouble some to even replicate this issue which could be due to how my VPN connection is directing traffic or perhaps I am getting the same remote address.

Sorry but I won't be able to look into this. One can however fork the project and implement/support this elsewhere.

It was not my intention for the integration to be completely usable outside US. It would be difficult to continuously bypass GDPS restrictions to account for changes. This will also definitely make it not acceptable as a core integration in HomeAssistant which is what I eventually want it to be.

seg-on commented 2 weeks ago

Hi @iprak, I found out that API v6 is working for me (I am using a AppScript in Google Sheets that use this API to get stock data). Can you look if there is the big difference between V7 you are using and V6? If it is not possible to port to V6? https://query2.finance.yahoo.com/v6/finance/options/ https://query2.finance.yahoo.com/v6/finance/options/aapl

iprak commented 2 weeks ago

The Yahoo end points are undocumented, this integration is based on my trial an error. v7 url is what is used when accessing finance.yahoo.com from US location, so I cannot tell how is v6 different. One would think that v7 is newer.

I get 404 when using v6 Received status 404 (Not Found HTTP 404 Not Found) for https://query1.finance.yahoo.com/v6/finance/quote?symbols=ISTNX,GBPUSD=X,IDFCBANK.BO,BABA,^SSMI,EMIM.L,AAPL,USDINR=X,ADYEN.AS&crumb=hLovZgIrG10

Something like this happened in May 2023 (#96) where v6 version was successful temporarily. You can locally change to use v6 but I won't be able to make that part of the integration.

nexusis7 commented 2 days ago

This has stopped working this mornin.

2024-11-19 10:03:30.129 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Received status 401 (Unauthorized User is unable to access this feature - https://bit.ly/yahoo-finance-api-feedback

Says API has been disabled

website gives the following message

{"finance":{"result":null,"error":{"code":"Unauthorized","description":"User is unable to access this feature - https://bit.ly/yahoo-finance-api-feedback"}}}

applegeeks commented 2 days ago

This has stopped working this mornin.

2024-11-19 10:03:30.129 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Received status 401 (Unauthorized User is unable to access this feature - https://bit.ly/yahoo-finance-api-feedback

Says API has been disabled

website gives the following message

{"finance":{"result":null,"error":{"code":"Unauthorized","description":"User is unable to access this feature - https://bit.ly/yahoo-finance-api-feedback"}}}

seems to be up and running again.

nexusis7 commented 2 days ago

yes can confirm it’s working again

iprak commented 2 days ago

This seems to be more of a Yahoo end point issue. What country is this?

nexusis7 commented 2 days ago

I’m in Canada.

iprak commented 2 days ago

I will give it a test using VPN.

applegeeks commented 2 days ago

I’m in Canada.

Australia here.

vPotusFlotus commented 1 day ago

This has stopped working this mornin.

2024-11-19 10:03:30.129 ERROR (MainThread) [custom_components.yahoofinance.coordinator] Received status 401 (Unauthorized User is unable to access this feature - https://bit.ly/yahoo-finance-api-feedback

Says API has been disabled

website gives the following message

{"finance":{"result":null,"error":{"code":"Unauthorized","description":"User is unable to access this feature - https://bit.ly/yahoo-finance-api-feedback"}}}

I receive the same error as stated above (from Belgium). Seems that API Access to Yahoo Finance Quotes might be disabled? image

Seems they potentially want to make it a paid service:

image

At least that's what the URL states from the Error Message: https://bit.ly/yahoo-finance-api-feedback