yql / yql-tables

YQL is extensible using a table definition. This repository will hold community contributed definitions.
http://developer.yahoo.com/yql
724 stars 416 forks source link

Yahoo.finance.quotes ((400) Bad Request) #489

Open darrylr02124 opened 7 years ago

darrylr02124 commented 7 years ago

The query below has reliably returned data from the yahoo.finance.quotes table for years until late last week (August 24 or 25, 2017):

https://query.yahooapis.com/v1/public/yql?q=select * from yahoo.finance.quotes where symbol in ('AEO','BAC')&diagnostics=true&env=store://datatables.org/alltableswithkeys

Although it still works reliably in the YQL console, the query now fails two thirds of the time with the error "No definition found for Table yahoo.finance.quotes" when run manually from the browser, or half of the time with the error "(400) Bad Request" when called programmatically. Otherwise, it returns the expected results, which suggests that there have been no syntax changes.

From the intermittent nature of the problem, it appears that one or more servers in what I'm assuming is a load balancing cluster is offline. Is anyone (e.g. at datatables.org) working on a fix for this problem?

maxgalbu commented 7 years ago

Happens to me too, for table yahoo.finance.xchange

darrylr02124 commented 7 years ago

Thanks, maxgalbu. I emailed the yahooapis.com domain technical contact (at domainadmin@yahoo-inc.com) this morning. Perhaps this will prompt someone to investigate it. My posts to GitHub and the Yahoo Finance help forum have been ineffective.

maxgalbu commented 7 years ago

I've tweeted to @yql twitter account too. https://twitter.com/ilgalbu/status/903563809067077632

maxgalbu commented 7 years ago

In the meantime, I've gone directly to the source of the YQL table, by looking at the "diagnostic" key take from the console, in my case:

"diagnostics": {
   "redirect": [
    {
     "from": "/ec?url=http%3a%2f%2fwww.datatables.org%2fyahoo%2ffinance%2fyahoo.finance.xchange.xml&t=1504273122&ttl=60&sig=uppUBXB3j0jMMxa4WM_y.Q--~D",
     "status": "307",
     "content": "http://www.datatables.org/yahoo/finance/yahoo.finance.xchange.xml"
    },
    {
     "from": "/ec?url=http%3a%2f%2fdownload.finance.yahoo.com%2fd%2fquotes.csv%3fs%3dEURCHF%3dX%26f%3dsnl1d1t1ab&t=1504273122&ttl=60&sig=M0pQTVl77dl69H97NywzJA--~D",
     "status": "307",
     "content": "http://download.finance.yahoo.com/d/quotes.csv?s=EURCHF=X&f=snl1d1t1ab"
    },
    {
     "from": "/ec?url=http%3a%2f%2fdownload.finance.yahoo.com%2fd%2fquotes.csv%3fs%3dGBPCHF%3dX%26f%3dsnl1d1t1ab&t=1504273122&ttl=60&sig=9_K4NyuyLPALCNOFoXHEKA--~D",
     "status": "307",
     "content": "http://download.finance.yahoo.com/d/quotes.csv?s=GBPCHF=X&f=snl1d1t1ab"
    }
   ],

So the source is the xml at http://www.datatables.org/yahoo/finance/yahoo.finance.xchange.xml which in turn uses http://download.finance.yahoo.com/d/quotes.csv?s=[pairs]&f=snl1d1t1ab to get exchange rates. :white_check_mark:

darrylr02124 commented 7 years ago

Max, thanks for the workaround. It's good to know that we can get the quotes in raw csv format if worse comes to worse. In my case, the URL turns out to be http://download.finance.yahoo.com/d/quotes.csv?f=aa2bb2b3b4cc1c3c4c6c8dd1d2ee1e7e8e9ghjkg1g3g4g5g6ii5j1j3j4j5j6k1k2k4k5ll1l2l3mm2m3m4m5m6m7m8nn4opp1p2p5p6qrr1r2r5r6r7ss1s7t1t7t8vv1v7ww1w4xy&s=[pairs]

darrylr02124 commented 7 years ago

At the risk of jinxing myself, I'd like to report that this issue was resolved yesterday, September 5, 2017 (for quotes at least).

maxgalbu commented 7 years ago

I'll stick to the csv. One less layer that could fail

joshuakoh7 commented 7 years ago

Apologies for bumping this but I seem to be facing the same error. Command works great in YQL console but 400 error (No definition found for Table yahoo.finance.quotes) via the URL.

https://query.yahooapis.com/v1/public/yql?q=env%20%27store://datatables.org/alltableswithkeys%27;%20select%20*%20from%20yahoo.finance.quotes%20where%20symbol=%22aapl%22;

maxgalbu commented 7 years ago

Ok yahoo blocked all requests to those files:

http://download.finance.yahoo.com/d/quotes.csv?s=%5Bpairs%5D&f=snl1d1t1ab

It has come to our attention that this service is being used in violation of the Yahoo Terms of Service. As such, the service is being discontinued. For all future markets and equities data research, please refer to finance.yahoo.com

They don't work through YQL either.