achannarasappa / ticker

Terminal stock ticker with live updates and position tracking
GNU General Public License v3.0
4.94k stars 264 forks source link

Nothing displayed - Yahoo API breaking contract change #247

Closed clauseggers closed 1 year ago

clauseggers commented 1 year ago

When I open ticker I get no stocks listed. Same as in #246 so perhaps they are related. I’m on version 4.5.6. I tried opening without the config file, but got same result. I can run something like curl https://query1.finance.yahoo.com/v6/finance/quote\?symbols\=AAPL,GOOG,F | jq and get good output.

Running on MacOS 13.3.1 (a), installed with Ticker was installed with Homebrew.

hchr commented 1 year ago

v7 strikes again it seems curl https://query1.finance.yahoo.com/v7/finance/quote?symbols=tsla {"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Cookie"}}}13:14

MattyRad commented 1 year ago

Rebuilding ticker after search/replacing v7 with v6 seems to fix the issue.

Is there interest in making the yahoo API version configurable?

ChangheeOh commented 1 year ago

Same issue. It seems to be related to macOS latest update.

sarcasticadmin commented 1 year ago

Its broken for me on linux too. No recent OS updates have been applied for me:

$ ticker -v
ticker version v4.5.5

I suspect its a similar outage to a few weeks ago: https://github.com/achannarasappa/ticker/issues/246

achannarasappa commented 1 year ago

Thanks for all the reports! I've created a new release to use the v6 Yahoo API which should fix it but I'll leave this open for a bit to confirm

ajgringo619 commented 1 year ago

Working as advertised here - thanks for the quick fix!

achannarasappa commented 1 year ago

Try with the latest version 4.5.10

Longer term some kind of abstraction layer should help mitigate these sorts of issues in the future.

clauseggers commented 1 year ago

Did Yahoo just shut off the v6 API?

v6

curl https://query1.finance.yahoo.com/v6/finance/quote\?symbols\=AAPL,GOOG,F | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    91  100    91    0     0   1489      0 --:--:-- --:--:-- --:--:--  1625
{
  "finance": {
    "result": null,
    "error": {
      "code": "Not Found",
      "description": "HTTP 404 Not Found"
    }
  }
}

v7

curl https://query1.finance.yahoo.com/v7/finance/quote\?symbols\=AAPL,GOOG,F | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    90  100    90    0     0   1722      0 --:--:-- --:--:-- --:--:--  1914
{
  "finance": {
    "result": null,
    "error": {
      "code": "Unauthorized",
      "description": "Invalid Cookie"
    }
  }
}
vvidic commented 1 year ago

Probably the same problem described here:

https://stackoverflow.com/questions/76065035/yahoo-finance-v7-api-now-requiring-cookies-python

achannarasappa commented 1 year ago

Thanks for reporting this - I can replicate the issue but there is no simple workaround as before so it may take some time to fix this issue.

@vvidic thanks for sharing the above link. I was not able fix the issue by following the steps listed in the link but I'll keep searching to see if someone else has found a workaround.

achannarasappa commented 1 year ago

I've made an update that should restore functionality with v4.5.13. I'll close this issue for now but please re-open if this happens again.


For those interested, there are three differences with the v7 Yahoo API in the latest breaking change:

  1. API calls must have the A3 cookie set a. Sessions expire after about 6 hours b. Cookies can be retrieved by visiting the Yahoo Finance homepage (example)
  2. API calls must have the URL query parameter crumb set a. "crumbs" can be retrieved by requesting GET https://query2.finance.yahoo.com/v1/test/getcrumb (example)
  3. API contract for the response from GET v7/finance/quote has changed for several fields to include both raw and formatted values (example)
clauseggers commented 1 year ago

Thank you @achannarasappa but I can’t get past executing the app. When I press return I just get the cursor moved to the next line, where it is then stuck seemingly doing nothing. In some terminal emulators it blinks a few times before going solid.

hchr commented 1 year ago

Thank you @achannarasappa but I can’t get past executing the app. When I press return I just get the cursor moved to the next line, where it is then stuck seemingly doing nothing. In some terminal emulators it blinks a few times before going solid.

I get the same here on Arch ticker -v ticker version v4.5.13

Added: It is connected and the data is flowing in: Sent Recived
ticker eno1 7.831 38.404 KB/sec

works for some it seems. Maybe OS related?

jfbourdeau commented 1 year ago

Tks for the last update. Everything is working again. JF MTL Canada

achannarasappa commented 1 year ago

Looks like there's still an issue but it does not impact everyone. It could be region or machine specific so for those that have issue, would you mind sharing country and machine details to help pinpoint the issue?

clauseggers commented 1 year ago

I’m on MacOS latest, installed ticker with Homebrew, and I’m located in the Netherlands.

097115 commented 1 year ago

Ubuntu 22.04.2, downloaded the binary from the Releases page, another EU country. Same symptoms as described above.

hchr commented 1 year ago

System: Kernel: 6.3.4-arch1-1 Desktop: KDE Plasma v: 5.27.5 Distro: EndeavourOS CPU: model: AMD Ryzen 9 5900HX with Radeon Graphics

Denmark

ktpx commented 1 year ago

I implemented the same approach in my own app and it works now. However it does not work with "finance.yahoo.com" in the initial request to get the cookie, I have to use "fc.yahoo.com".

eriknh commented 1 year ago

I just updated via the snap updater on Ubuntu server 22.04 to ticker version 4.5.13 and the app works as expected again. However I did not set any cookies. How is it working without me setting a cookie or specifying the CRUMB value?

Also, can I set these somehow as a variable upon calling the "ticker" command? How would I input my own cookie variables otherwise?

Thank you for the update @achannarasappa and thank you for the explanations.

ktpx commented 1 year ago

You don't have to set anything, the code does it.

Cubuss commented 1 year ago

i get nothing using snap ticker version 4.5.13

ticker -w GME it just hangs still no output

eriknh commented 1 year ago

You don't have to set anything, the code does it.

Thanks @ktpx! Great to know the code does this for us. @achannarasappa amazing development on that CRUMB/COOKIE effort. Would love to know if any of these Yahoo Finance variables gets added to the config file should Yahoo start getting finicky about access.

achannarasappa commented 1 year ago

I was able to replicate the issue reported by EU users. On finance.yahoo.com, there is a prompt related to EU data privacy regulations that the user must accept before using the Yahoo Finance. Without accepting this prompt, the required cookie (A3) is not set. ticker v4.5.13 does not support this today. Look out for an update here soon that adds support.

ktpx commented 1 year ago

I was able to replicate the issue reported by EU users. On finance.yahoo.com, there is a prompt related to EU data privacy regulations that the user must accept before using the Yahoo Finance. Without accepting this prompt, the required cookie (A3) is not set. ticker v4.5.13 does not support this today. Look out for an update here soon that adds support.

Thats probably why fc.yahoo.com works here (im in eu), and not finance.yahoo.com.

achannarasappa commented 1 year ago

v4.5.14 has just been released which fixes the quote retrieval issue for EU users. Closing this issue for now but will reopen if there's still any unresolved problems.

Thanks all for providing details and reports that helped understand the issue better!