Intermittent HTTP 401 returned from Yahoo API

[centic9]

I am calling the API at a frequency of 5 minutes for around 10 stocks, which worked fine for quite some time.

Since a few days, the Yahoo API still works, but sometimes fails with HTTP 401.

There have previously been issues with some cookies which lead to this error, e.g. https://github.com/sstrickx/yahoofinance-api/issues/130

Is this a re-occurrence of this?

If the API now really would need authentication via a Yahoo-ID and OAuth it would fail always, not intermittently, so seems at least some similar issue.

Can I investigate this some more? Logging? Debugging?

Code:

Stock stock = YahooFinance.get("ORCL");

Exception

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 401 for URL: https://query1.finance.yahoo.com/v7/finance/quote?symbols=ORCL
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1974)
        at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1969)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1968)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250)
        at yahoofinance.quotes.query1v7.QuotesRequest.getResult(QuotesRequest.java:74)
        at yahoofinance.YahooFinance.getQuotes(YahooFinance.java:381)
        at yahoofinance.YahooFinance.get(YahooFinance.java:98)
        at yahoofinance.YahooFinance.get(YahooFinance.java:82)
        ...
Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://query1.finance.yahoo.com/v7/finance/quote?symbols=ORCL
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520)
        at java.base/[java.net](http://java.net/).HttpURLConnection.getResponseCode(HttpURLConnection.java:527)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:334)
        at yahoofinance.util.RedirectableRequest.openConnection(RedirectableRequest.java:54)
        at yahoofinance.util.RedirectableRequest.openConnection(RedirectableRequest.java:34)
        at yahoofinance.quotes.query1v7.QuotesRequest.getResult(QuotesRequest.java:72)
        ... 8 more

[centic9]

When I tried to load https://query1.finance.yahoo.com/v7/finance/quote?symbols=ORCL in the Browser, it mostly works, but sometimes I get

{"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Crumb"}}}

So it seems this may be some problem at Yahoo itself.

[emidesy]

Hi! the same error to me, code":"Unauthorized"

Something change on Yahoo?

[lumirodri]

@centic9 @emidesy Same here. Apparently you need to pass the crumb as an argument now?

If you go to https://query1.finance.yahoo.com/v1/test/getcrumb on a browser, you should get a string result. If you pass it to the URL, you should be able to get the info like so:

https://query1.finance.yahoo.com/v7/finance/quote?symbols=ORCL&crumb=<YOUR_CRUMB_RESULT>

However, this is not working in my Android App. I've debugged the app a little and Cookie and Crumb are returning empty because the CrumbManager is not matching the following patterns that are used:

Pattern patternPostForm = Pattern.compile("action=\"/consent\"");
Pattern patternInput = Pattern.compile("(<input type=\"hidden\" name=\")(.*?)(\" value=\")(.*?)(\">)");

I'm assuming something changed related to how the API is accessed and the content returned from HISTQUOTES2_SCRAPE_URL also changed.

[code-monkey-101]

Thank you @lumism! You are right, adding the crumb to the URL fixed it for me indeed on PC.

The pattern doesn't match for me either but for some reason I can get the crumb without the cookie.

[JoseOcampo02]

Thank you @lumism! You are right, adding the crumb to the URL fixed it for me indeed on PC.

The pattern doesn't match for me either but for some reason I can get the crumb without the cookie.

I have the api as a dependency in my Maven project. How can I modify the QuotesRequest class as you have above? I do not know where to find or how to modify the files? I am using Eclipse IDE

[code-monkey-101]

I have the api as a dependency in my Maven project. How can I modify the QuotesRequest class as you have above? I do not know where to find or how to modify the files? I am using Eclipse IDE

Just download the source from github, unpack it into a subdirectory and add it to the build path

[playinlab]

I have the same problem:

Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://query1.finance.yahoo.com/v7/finance/quote?symbols=..... at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1913) ~[na:na] at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509) ~[na:na] at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) ~[na:na] at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:329) ~[na:na] at yahoofinance.util.RedirectableRequest.openConnection(RedirectableRequest.java:54) ~[YahooFinanceAPI-3.17.0.jar:na] at yahoofinance.util.RedirectableRequest.openConnection(RedirectableRequest.java:34) ~[YahooFinanceAPI-3.17.0.jar:na] at yahoofinance.quotes.query1v7.QuotesRequest.getResult(QuotesRequest.java:72) ~[YahooFinanceAPI-3.17.0.jar:na]

[tanaysomani]

Download source , open in IDE do code change and build yahoofinance jar in local add jar manually to your project and also add jackson-databind dependancy to your pom

[playinlab]

I added params.put("crumb", CrumbManager.getCrumb()); I still got the same error. "crumb=" is added to URL. Should not this crumb parameter be empty?

java.io.IOException: Server returned HTTP response code: 401 for URL: https://query1.finance.yahoo.com/v7/finance/quote?symbols=CSOL-EUR%2CSOL-USD&crumb= at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:na] at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:na] at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[na:na] at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1974) ~[na:na] at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1969) ~[na:na] at java.base/java.security.AccessController.doPrivileged(Native Method) ~[na:na] at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1968) ~[na:na] at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536) ~[na:na] at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) ~[na:na] at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250) ~[na:na] at yahoofinance.quotes.query1v7.QuotesRequest.getResult(QuotesRequest.java:76) ~[YahooFinanceAPI-3.18.0-SNAPSHOT.jar:na] at yahoofinance.YahooFinance.getQuotes(YahooFinance.java:381) ~[YahooFinanceAPI-3.18.0-SNAPSHOT.jar:na] at yahoofinance.YahooFinance.get(YahooFinance.java:220) ~[YahooFinanceAPI-3.18.0-SNAPSHOT.jar:na] at yahoofinance.YahooFinance.get(YahooFinance.java:199) ~[YahooFinanceAPI-3.18.0-SNAPSHOT.jar:na]

[code-monkey-101]

I added params.put("crumb", CrumbManager.getCrumb()); I still got the same error. "crumb=" is added to URL. Should not this crumb parameter be empty?

It could depend on the country that you are in whether or not you need the cookie. I'm in the US.

[playinlab]

The problem is that crumb is not generated in the function.

If I manually use URL https://query1.finance.yahoo.com/v1/test/getcrumb in browser, than I got a crumb. The code below does not return crumb. It is always empty.

    URL crumbRequest = new URL(YahooFinance.HISTQUOTES2_CRUMB_URL);
    RedirectableRequest redirectableCrumbRequest = new RedirectableRequest(crumbRequest, 5);
    redirectableCrumbRequest.setConnectTimeout(YahooFinance.CONNECTION_TIMEOUT);
    redirectableCrumbRequest.setReadTimeout(YahooFinance.CONNECTION_TIMEOUT);

    Map<String, String> requestProperties = new HashMap<String, String>();
    requestProperties.put("Cookie", cookie);
    log.info(cookie);

    URLConnection crumbConnection = redirectableCrumbRequest.openConnection(requestProperties);
    InputStreamReader is = new InputStreamReader(crumbConnection.getInputStream());
    BufferedReader br = new BufferedReader(is);        
    String crumbResult = br.readLine();
    log.info("crumbResult", crumbResult);

[playinlab]

I tried also with chatGPT code and crumb is not returned

   URL url = new URL("https://query1.finance.yahoo.com/v1/test/getcrumb");
   HttpURLConnection con = (HttpURLConnection) url.openConnection();
   con.setRequestMethod("GET");
   con.setRequestProperty("User-Agent", "Mozilla/5.0");

   BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
   String inputLine;
   StringBuilder response = new StringBuilder();
   while ((inputLine = in.readLine()) != null) {
      response.append(inputLine);
   }
   in.close();

   String crumb = response.toString().replaceAll("\"", "");
   System.out.println("Crumb value is: " + crumb);

[code-monkey-101]

Don't know what else to say. Works for me (cookie is an empty string, though).

[costamarco]

use https://query1.finance.yahoo.com/v6/finance/quote?symbols=...

it works for me....v7 -> v6

[playinlab]

Hi,

I tried to use windscribe vpn. I found out that it did not work from Europe(Vienna, Oslo, Zagreb) but it worked from Tokyo and US. Then I tried also Frankfurt Germany and it worked. After that also other EU cities worked (Vienna, Oslo, Zagreb).

Change in QuotesRequest works for me. String crumb = CrumbManager.getCrumb(); params.put("crumb", crumb);

Thanks

[centic9]

use https://query1.finance.yahoo.com/v6/finance/quote?symbols=...

it works for me....v7 -> v6

This means the following may be a workaround in the client-code without changes to the yahoofinance-api code itself

System.setProperty("yahoofinance.baseurl.quotesquery1v7", "https://query1.finance.yahoo.com/v6/finance/quote");

At least it seems to make it work for me.

[lumirodri]

The problem is that crumb is not generated in the function.

If I manually use URL https://query1.finance.yahoo.com/v1/test/getcrumb in browser, than I got a crumb. The code below does not return crumb. It is always empty.

    URL crumbRequest = new URL(YahooFinance.HISTQUOTES2_CRUMB_URL);
    RedirectableRequest redirectableCrumbRequest = new RedirectableRequest(crumbRequest, 5);
    redirectableCrumbRequest.setConnectTimeout(YahooFinance.CONNECTION_TIMEOUT);
    redirectableCrumbRequest.setReadTimeout(YahooFinance.CONNECTION_TIMEOUT);

    Map<String, String> requestProperties = new HashMap<String, String>();
    requestProperties.put("Cookie", cookie);
    log.info(cookie);

    URLConnection crumbConnection = redirectableCrumbRequest.openConnection(requestProperties);
    InputStreamReader is = new InputStreamReader(crumbConnection.getInputStream());
    BufferedReader br = new BufferedReader(is);        
    String crumbResult = br.readLine();
    log.info("crumbResult", crumbResult);

I have the same problem - Crumb is indeed generated on the browser, but not in the app where the same URL is used. Like I've mentioned before, the patterns are not matched against the HTML the app receives, so scraping the crumb and cookie isn't working for me also.

use https://query1.finance.yahoo.com/v6/finance/quote?symbols=... it works for me....v7 -> v6

This means the following may be a workaround in the client-code without changes to the yahoofinance-api code itself

System.setProperty("yahoofinance.baseurl.quotesquery1v7", "https://query1.finance.yahoo.com/v6/finance/quote");

At least it seems to make it work for me.

Changing the URL does seem to work. Apparently that one hasn't changed how it's accessed and is working exactly as it was before (for now, anyway!)

[peternees]

For me, the crumb is not null and adding it as a url parameter is sufficient. Don't need to go back to v6.

[lumirodri]

For me, the crumb is not null and adding it as a url parameter is sufficient. Don't need to go back to v6.

I'm assuming that the content from which the crumb is extracted is regionalized then. I'm from Europe and the content from the base URL scrape does not get any pattern matches. Let's hope v6 doesn't suffer the same fate!

[oauthtester01]

Igot the same error today , I am using the API to get the ticker quote , how to ia add the crumb to my call ?

stock = YahooFinance.get(stockArray[i]);

[tal26]

It's not the first time something like this happens. It usually takes a couple of days for them to fix their backend

[kadirbekov]

changing url worked for me too. thanks. System.setProperty("yahoofinance.baseurl.quotesquery1v7", "https://query1.finance.yahoo.com/v6/finance/quote");

[oauthtester01]

thanks that worked

[centic9]

It seems the "v6" API was now shutdown by Yahoo, likely because everyone switched over from "v7"

[IvanDuhov]

@centic9 is there any fix available for the v7 of the API?

[vw98075]

It likely is the case. I get the "404" error for the v6.

{
    "finance": {
        "result": null,
        "error": {
            "code": "Not Found",
            "description": "HTTP 404 Not Found"
        }
    }
}

[bufayadexiaotudou]

please help me `C:\Users\l2026.jdks\corretto-17.0.7\bin\java.exe "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2023.1\lib\idea_rt.jar=49815:C:\Program Files\JetBrains\IntelliJ IDEA 2023.1\bin" -Dfile.encoding=UTF-8 -classpath F:\java_items\StockAnalysis\target\classes;C:\Users\l2026.m2\repository\org\springframework\boot\spring-boot-starter-web\3.0.6\spring-boot-starter-web-3.0.6.jar;C:\Users\l2026.m2\repository\org\springframework\boot\spring-boot-starter\3.0.6\spring-boot-starter-3.0.6.jar;C:\Users\l2026.m2\repository\org\springframework\boot\spring-boot-starter-logging\3.0.6\spring-boot-starter-logging-3.0.6.jar;C:\Users\l2026.m2\repository\ch\qos\logback\logback-classic\1.4.7\logback-classic-1.4.7.jar;C:\Users\l2026.m2\repository\ch\qos\logback\logback-core\1.4.7\logback-core-1.4.7.jar;C:\Users\l2026.m2\repository\org\apache\logging\log4j\log4j-to-slf4j\2.19.0\log4j-to-slf4j-2.19.0.jar;C:\Users\l2026.m2\repository\org\apache\logging\log4j\log4j-api\2.19.0\log4j-api-2.19.0.jar;C:\Users\l2026.m2\repository\org\slf4j\jul-to-slf4j\2.0.7\jul-to-slf4j-2.0.7.jar;C:\Users\l2026.m2\repository\jakarta\annotation\jakarta.annotation-api\2.1.1\jakarta.annotation-api-2.1.1.jar;C:\Users\l2026.m2\repository\org\yaml\snakeyaml\1.33\snakeyaml-1.33.jar;C:\Users\l2026.m2\repository\org\springframework\boot\spring-boot-starter-json\3.0.6\spring-boot-starter-json-3.0.6.jar;C:\Users\l2026.m2\repository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.14.2\jackson-datatype-jdk8-2.14.2.jar;C:\Users\l2026.m2\repository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.14.2\jackson-datatype-jsr310-2.14.2.jar;C:\Users\l2026.m2\repository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.14.2\jackson-module-parameter-names-2.14.2.jar;C:\Users\l2026.m2\repository\org\springframework\boot\spring-boot-starter-tomcat\3.0.6\spring-boot-starter-tomcat-3.0.6.jar;C:\Users\l2026.m2\repository\org\apache\tomcat\embed\tomcat-embed-core\10.1.8\tomcat-embed-core-10.1.8.jar;C:\Users\l2026.m2\repository\org\apache\tomcat\embed\tomcat-embed-el\10.1.8\tomcat-embed-el-10.1.8.jar;C:\Users\l2026.m2\repository\org\apache\tomcat\embed\tomcat-embed-websocket\10.1.8\tomcat-embed-websocket-10.1.8.jar;C:\Users\l2026.m2\repository\org\springframework\spring-web\6.0.8\spring-web-6.0.8.jar;C:\Users\l2026.m2\repository\org\springframework\spring-beans\6.0.8\spring-beans-6.0.8.jar;C:\Users\l2026.m2\repository\io\micrometer\micrometer-observation\1.10.6\micrometer-observation-1.10.6.jar;C:\Users\l2026.m2\repository\io\micrometer\micrometer-commons\1.10.6\micrometer-commons-1.10.6.jar;C:\Users\l2026.m2\repository\org\springframework\spring-webmvc\6.0.8\spring-webmvc-6.0.8.jar;C:\Users\l2026.m2\repository\org\springframework\spring-aop\6.0.8\spring-aop-6.0.8.jar;C:\Users\l2026.m2\repository\org\springframework\spring-context\6.0.8\spring-context-6.0.8.jar;C:\Users\l2026.m2\repository\org\springframework\spring-expression\6.0.8\spring-expression-6.0.8.jar;C:\Users\l2026.m2\repository\org\springframework\spring-core\6.0.8\spring-core-6.0.8.jar;C:\Users\l2026.m2\repository\org\springframework\spring-jcl\6.0.8\spring-jcl-6.0.8.jar;C:\Users\l2026.m2\repository\org\apache\commons\commons-csv\1.9.0\commons-csv-1.9.0.jar;C:\Users\l2026.m2\repository\com\opencsv\opencsv\5.4\opencsv-5.4.jar;C:\Users\l2026.m2\repository\org\apache\commons\commons-lang3\3.12.0\commons-lang3-3.12.0.jar;C:\Users\l2026.m2\repository\org\apache\commons\commons-text\1.9\commons-text-1.9.jar;C:\Users\l2026.m2\repository\commons-beanutils\commons-beanutils\1.9.4\commons-beanutils-1.9.4.jar;C:\Users\l2026.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\l2026.m2\repository\commons-collections\commons-collections\3.2.2\commons-collections-3.2.2.jar;C:\Users\l2026.m2\repository\org\apache\commons\commons-collections4\4.4\commons-collections4-4.4.jar;C:\Users\l2026.m2\repository\org\springframework\boot\spring-boot-devtools\3.0.6\spring-boot-devtools-3.0.6.jar;C:\Users\l2026.m2\repository\org\springframework\boot\spring-boot\3.0.6\spring-boot-3.0.6.jar;C:\Users\l2026.m2\repository\org\springframework\boot\spring-boot-autoconfigure\3.0.6\spring-boot-autoconfigure-3.0.6.jar;C:\Users\l2026.m2\repository\org\datavec\datavec-api\1.0.0-beta7\datavec-api-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\commons-io\commons-io\2.5\commons-io-2.5.jar;C:\Users\l2026.m2\repository\commons-codec\commons-codec\1.15\commons-codec-1.15.jar;C:\Users\l2026.m2\repository\org\slf4j\slf4j-api\2.0.7\slf4j-api-2.0.7.jar;C:\Users\l2026.m2\repository\joda-time\joda-time\2.2\joda-time-2.2.jar;C:\Users\l2026.m2\repository\org\nd4j\jackson\1.0.0-beta7\jackson-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\org\freemarker\freemarker\2.3.32\freemarker-2.3.32.jar;C:\Users\l2026.m2\repository\org\nd4j\nd4j-common\1.0.0-beta7\nd4j-common-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\org\nd4j\guava\1.0.0-beta7\guava-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\org\apache\commons\commons-math3\3.5\commons-math3-3.5.jar;C:\Users\l2026.m2\repository\org\apache\commons\commons-compress\1.18\commons-compress-1.18.jar;C:\Users\l2026.m2\repository\org\nd4j\nd4j-api\1.0.0-beta7\nd4j-api-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\com\jakewharton\byteunits\byteunits\0.9.1\byteunits-0.9.1.jar;C:\Users\l2026.m2\repository\com\google\flatbuffers\flatbuffers-java\1.10.0\flatbuffers-java-1.10.0.jar;C:\Users\l2026.m2\repository\org\nd4j\protobuf\1.0.0-beta7\protobuf-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\commons-net\commons-net\3.1\commons-net-3.1.jar;C:\Users\l2026.m2\repository\net\ericaro\neoitertools\1.0.0\neoitertools-1.0.0.jar;C:\Users\l2026.m2\repository\org\bytedeco\javacpp\1.5.3\javacpp-1.5.3.jar;C:\Users\l2026.m2\repository\com\clearspring\analytics\stream\2.9.8\stream-2.9.8.jar;C:\Users\l2026.m2\repository\net\sf\opencsv\opencsv\2.3\opencsv-2.3.jar;C:\Users\l2026.m2\repository\com\tdunning\t-digest\3.2\t-digest-3.2.jar;C:\Users\l2026.m2\repository\it\unimi\dsi\fastutil\6.5.7\fastutil-6.5.7.jar;C:\Users\l2026.m2\repository\org\deeplearning4j\deeplearning4j-nn\1.0.0-beta7\deeplearning4j-nn-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\org\deeplearning4j\deeplearning4j-utility-iterators\1.0.0-beta7\deeplearning4j-utility-iterators-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\org\lucee\oswego-concurrent\1.3.4\oswego-concurrent-1.3.4.jar;C:\Users\l2026.m2\repository\org\deeplearning4j\deeplearning4j-common\1.0.0-beta7\deeplearning4j-common-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\org\nd4j\nd4j-native-api\1.0.0-beta7\nd4j-native-api-1.0.0-beta7.jar;C:\Users\l2026.m2\repository\com\google\code\gson\gson\2.9.1\gson-2.9.1.jar;C:\Users\l2026.m2\repository\com\github\oshi\oshi-core\3.4.2\oshi-core-3.4.2.jar;C:\Users\l2026.m2\repository\net\java\dev\jna\jna-platform\4.3.0\jna-platform-4.3.0.jar;C:\Users\l2026.m2\repository\net\java\dev\jna\jna\4.3.0\jna-4.3.0.jar;C:\Users\l2026.m2\repository\org\threeten\threetenbp\1.3.3\threetenbp-1.3.3.jar;C:\Users\l2026.m2\repository\com\fasterxml\jackson\dataformat\jackson-dataformat-csv\2.10.5\jackson-dataformat-csv-2.10.5.jar;C:\Users\l2026.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.14.2\jackson-annotations-2.14.2.jar;C:\Users\l2026.m2\repository\com\fasterxml\jackson\core\jackson-core\2.14.2\jackson-core-2.14.2.jar;C:\Users\l2026.m2\repository\com\yahoofinance-api\YahooFinanceAPI\3.17.0\YahooFinanceAPI-3.17.0.jar;C:\Users\l2026.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.13.0\jackson-databind-2.13.0.jar com.example.stockanalysis.StockCrawler.StockCrawlerService 11:37:08.723 [main] INFO yahoofinance.quotes.query1v7.QuotesRequest -- Sending request: https://query1.finance.yahoo.com/v7/finance/quote?symbols=TSLA java.io.IOException: Server returned HTTP response code: 401 for URL: https://query1.finance.yahoo.com/v7/finance/quote?symbols=TSLA at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:2051) at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:2046) at java.base/java.security.AccessController.doPrivileged(AccessController.java:569) at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:2045) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1609) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224) at yahoofinance.quotes.query1v7.QuotesRequest.getResult(QuotesRequest.java:74) at yahoofinance.YahooFinance.getQuotes(YahooFinance.java:381) at yahoofinance.YahooFinance.get(YahooFinance.java:98) at com.example.stockanalysis.StockCrawler.StockCrawlerService.crawlStock(StockCrawlerService.java:53) at com.example.stockanalysis.StockCrawler.StockCrawlerService.crawlStock(StockCrawlerService.java:46) at com.example.stockanalysis.StockCrawler.StockCrawlerService.main(StockCrawlerService.java:36) Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://query1.finance.yahoo.com/v7/finance/quote?symbols=TSLA at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:2000) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308) at yahoofinance.util.RedirectableRequest.openConnection(RedirectableRequest.java:54) at yahoofinance.util.RedirectableRequest.openConnection(RedirectableRequest.java:34) at yahoofinance.quotes.query1v7.QuotesRequest.getResult(QuotesRequest.java:72) ... 5 more Crawl result for stock TSLA for the past 30 days: false

进程已结束,退出代码0 `

[vw98075]

@bufayadexiaotudou Not sure whether you read through this post or not. The URL no longer functions.

[Scrounger]

It seems that here is a solution to get the crumb for e.g. EU countries, to use the v7 api again. But it's in JavaScript in i don't understand it in total. Perhaps someone can help to transfer this?

issue: https://github.com/gadicc/node-yahoo-finance2/issues/633

code of fix: https://github.com/gadicc/node-yahoo-finance2/commit/a4f90f1b400f772cb851a2e4b38f471ffece2213

[lumirodri]

It seems the "v6" API was now shutdown by Yahoo, likely because everyone switched over from "v7"

I think you're spot on. If this means we have to use v7, the whole part on getting the crumbs and scraping will have to be redone, since it's pretty much obsolete now.

[luismqoliveira]

Isn't this valid?

[oauthtester01]

I am getting this new error in my java spring boot application. java.io.FileNotFoundException: https://query1.finance.yahoo.com/v6/finance/quote?symbols=ITOT at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1963) at java.base/sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1958) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1957) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1525) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:245) at yahoofinance.quotes.query1v7.QuotesRequest.getResult(QuotesRequest.java:74) at yahoofinance.YahooFinance.getQuotes(YahooFinance.java:381) at yahoofinance.YahooFinance.get(YahooFinance.java:98) at yahoofinance.YahooFinance.get(YahooFinance.java:82) at com.kripal.stocktracker.MainController.index(MainController.java:100) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:655) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:764) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:895) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1732) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:834)

[kallavikas]

Take reference from this code which which is working change the existing source code and create a jar , now application will work back to normal

`import org.junit.Test; import yahoofinance.Stock; import yahoofinance.YahooFinance; import yahoofinance.histquotes.Interval; import yahoofinance.histquotes2.CrumbManager;

import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.math.BigDecimal; import java.net.HttpURLConnection; import java.net.URL; import java.util.Calendar; import java.util.Map;

import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue;

/*

• @author Stijn Strickx */ public class QuoteRequestFlowTest1 {

  @Test public void quoteRefreshTest() throws IOException { String cookie = getCookie (); String crumb = getCrumb ( cookie ); System.setProperty("yahoofinance.crumb",crumb ); System.setProperty("yahoofinance.cookie",cookie); Stock stock = YahooFinance.get("TSLA"); // System.out.println (getStockData ( cookie,crumb )); System.out.println (stock.getQuote ().getPrice ());

  Calendar from = Calendar.getInstance();
  Calendar to = Calendar.getInstance();
  from.add(Calendar.MONTH, -1); // from 5 years ago
  
  Stock google = YahooFinance.get("GOOG", from, to, Interval.WEEKLY);
  System.out.println (google.getQuote ().getPrice ());

  }

  public String getCookie() { String cookie = null; try { URL url = new URL("https://fc.yahoo.com"); HttpURLConnection connection = (HttpURLConnection) url.openConnection();

      // Get the cookie from the response headers
       cookie = connection.getHeaderField("Set-Cookie");
  
      if (cookie != null) {
          // Print the cookie
          System.out.println("Cookie: " + cookie);
      }
  
      //get crumb
      System.out.println (getCrumb ( cookie ));
  
      // Disconnect the connection
      connection.disconnect();
  } catch (Exception e) {
      e.printStackTrace();
  }
  return  cookie;

  }

  public static String getCrumb(String cookie){ StringBuilder response = new StringBuilder(); try { URL url = new URL("https://query2.finance.yahoo.com/v1/test/getcrumb"); HttpURLConnection connection = (HttpURLConnection) url.openConnection();

      // Set the cookie
      connection.setRequestProperty("Cookie", cookie);
  
      // Make the HTTP request
      connection.setRequestMethod("GET");
  
      // Read the response content
      BufferedReader reader = new BufferedReader(new InputStreamReader (connection.getInputStream()));
      String line;
      while ((line = reader.readLine()) != null) {
          response.append(line);
      }
      reader.close();
  
      // Process the response content as needed
      System.out.println("Response: " + response.toString());
  
      // Disconnect the connection
      connection.disconnect();
  } catch (Exception e) {
      e.printStackTrace();
  }
  return  response.toString ();

  }

  public static String getStockData(String stockName,String cookie,String crumb){ StringBuilder response = new StringBuilder(); try { URL url = new URL("https://query2.finance.yahoo.com/v7/finance/quote?symbols="+stockName+"&crumb="+crumb); HttpURLConnection connection = (HttpURLConnection) url.openConnection();

      // Set the cookie
      connection.setRequestProperty("Cookie", cookie);
  
      // Make the HTTP request
      connection.setRequestMethod("GET");
  
      // Read the response content
      BufferedReader reader = new BufferedReader(new InputStreamReader (connection.getInputStream()));
      String line;
      while ((line = reader.readLine()) != null) {
          response.append(line);
      }
      reader.close();
  
      // Process the response content as needed
      System.out.println("Response: " + response.toString());
  
      // Disconnect the connection
      connection.disconnect();
  } catch (Exception e) {
      e.printStackTrace();
  }
  return  response.toString ();

  } } `

[lumirodri]

@kallavikas

I've adapted the code to my use case, and I can confirm this solutions works (for now). Let's hope it stays that way this time.

[jschlade]

You need to use the v7 quotes query URL as the v6 one has been disabled.

Looking at the the 3.17.0 code base you need to change the getResult() method in the abstract class QuotesRequest in package yahoofinance.quotes.query1v7 and append the crumb to the URL.

    String url = YahooFinance.QUOTES_QUERY1V7_BASE_URL + "?" + Utils.getURLParameters(params);
    if (!CrumbManager.getCrumb().isEmpty()) {
        url = url + "&crumb=" + CrumbManager.getCrumb();
    }

[luismqoliveira]

You need to use the v7 quotes query URL as the v6 one has been disabled.

Looking at the the 3.17.0 code base you need to change the getResult() method in the abstract class QuotesRequest in package yahoofinance.quotes.query1v7 and append the crumb to the URL.

    String url = YahooFinance.QUOTES_QUERY1V7_BASE_URL + "?" + Utils.getURLParameters(params);
    if (!CrumbManager.getCrumb().isEmpty()) {
        url = url + "&crumb=" + CrumbManager.getCrumb();
    }

I tried by it gives me "" the CrumbManager.getCrumb() call... I think it's the cookie, the cookie gives "" too...

[jschlade]

Hi @luismqoliveira

I'm in the US and there is no cookie value either. Not sure if having no cookie value effects quote history, etc.?

But okay some questions:

1) What country are you trying this from? 2) From a browser, what do you see from doing:

• Go to https://finance.yahoo.com/ first and then try:
• https://query1.finance.yahoo.com/v1/test/getcrumb

Do you see a crumb value? If not this will be harder to fix.

Thanks

[adsharpe]

I have the api as a dependency in my Maven project. How can I modify the QuotesRequest class as you have above? I do not know where to find or how to modify the files? I am using Eclipse IDE

Just download the source from github, unpack it into a subdirectory and add it to the build path

Is there a reason that this still hasn't been added to the repo?

[joshualj]

To address the 401, I have tried to add the attached project to my build class path, however I'm now getting a new error: Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/annotation/JsonMerge

In the POM file (both the original pom and libsrc pom), I have set the jackson-databind dependency to the latest version. Any recommendations?

I have the api as a dependency in my Maven project. How can I modify the QuotesRequest class as you have above? I do not know where to find or how to modify the files? I am using Eclipse IDE

Just download the source from github, unpack it into a subdirectory and add it to the build path

Is there a reason that this still hasn't been added to the repo?

[joshualj]

@code-monkey-101 where do the following packages come from?

com.ib.client com.ib.contracts com.ib.controller

I am trying to integrate the project you mentioned into my own project, while maintaining a valid project structure.

I have the api as a dependency in my Maven project. How can I modify the QuotesRequest class as you have above? I do not know where to find or how to modify the files? I am using Eclipse IDE

Just download the source from github, unpack it into a subdirectory and add it to the build path

[code-monkey-101]

@code-monkey-101 where do the following packages come from?

com.ib.client com.ib.contracts com.ib.controller

That's the Interactive Brokers API. You don't need that.

[joshualj]

Thank you for the project structure recommendation, @code-monkey-101 !

I also do not see the following 3 classes in the screenshot --

Stock.java, Utils.java, YahooFinance.java

-- however they are in the project on GitHub.

If I include those within the yahoofinance package, then YahooFinance.java conflicts with the YahooFinance.class imported by Maven.

Would you advise which location the above three classes should go? Thank you again

[StephensCode]

I downgraded my Java to 1.8 and it works for me. This isn't a solution but it should be noted.

[bnsd55]

Worked for me https://github.com/sstrickx/yahoofinance-api/pull/210#pullrequestreview-2022020295

[sameershrestha333]

Igot the same error today , I am using the API to get the ticker quote , how to ia add the crumb to my call ?

stock = YahooFinance.get(stockArray[i]);

Hi am also facing same issue, have found any solution?