Quotes now result in server errors

drusepth commented 7 years ago

I'm not sure if Yahoo just changed their API, but logic that would previously retrieve quotes over a period of time is now returning server errors from Yahoo.

Example code:

irb(main):055:0> data = StockQuote::Stock.quote('GOOG', Time.now - 1.year, Time.now)
=> [
#<StockQuote::Stock:0x0055a50117b410 @response_code=200, @symbol="GOOG", @_html_="<HEAD>">, 
#<StockQuote::Stock:0x0055a50115eb30 @response_code=200, @symbol="GOOG", @_html_="<TITLE>Error</TITLE>">, 
#<StockQuote::Stock:0x0055a501152c18 @response_code=200, @symbol="GOOG", @_html_="</HEAD>">, 
#<StockQuote::Stock:0x0055a50113a398 @response_code=404, @no_data_message="Query returns no valid data">, 
#<StockQuote::Stock:0x0055a50113a230 @response_code=200, @symbol="GOOG", @_html_="<BODY BGCOLOR=\"white\" FGCOLOR=\"black\">">, 
#<StockQuote::Stock:0x0055a501129a48 @response_code=200, @symbol="GOOG", @_html_="<!-- status code : 404 -->">, 
#<StockQuote::Stock:0x0055a501101098 @response_code=200, @symbol="GOOG", @_html_="<!-- Error: GET -->">, 
#<StockQuote::Stock:0x0055a5010f9370 @response_code=200, @symbol="GOOG", @_html_="<!-- host machine: media-router6.prod.media.bf1.yahoo.com -->">, 
#<StockQuote::Stock:0x0055a5010edc28 @response_code=200, @symbol="GOOG", @_html_="<!-- timestamp: 1496865667.000 -->">, 
#<StockQuote::Stock:0x0055a5010da600 @response_code=200, @symbol="GOOG", @_html_="<!-- url: https://ichart.finance.yahoo.com/table.csv?a=5&b=7&e=7&g=d&c=2016&d=5&f=2017&s=GOOG-->">, 
#<StockQuote::Stock:0x0055a5010bdb68 @response_code=200, @symbol="GOOG", @_html_="<H1>Error</H1>">, 
#<StockQuote::Stock:0x0055a5010b17a0 @response_code=200, @symbol="GOOG", @_html_="<HR>">, 
#<StockQuote::Stock:0x0055a50109d228 @response_code=404, @no_data_message="Query returns no valid data">, 
#<StockQuote::Stock:0x0055a50109d138 @response_code=200, @symbol="GOOG", @_html_="<FONT FACE=\"Helvetica,Arial\"><B>">, 
#<StockQuote::Stock:0x0055a50108cd88 @response_code=200, @symbol="GOOG", @_html_="Description: Could not process this \"GET\" request.">, 
#<StockQuote::Stock:0x0055a501078630 @response_code=200, @symbol="GOOG", @_html_="</B></FONT>">, 
#<StockQuote::Stock:0x0055a501046ea0 @response_code=200, @symbol="GOOG", @_html_="<HR>">, 
#<StockQuote::Stock:0x0055a5010266f0 @response_code=200, @symbol="GOOG", @_html_="</BODY>">
]

.quote is returning an array of HTML error lines (instead of actual stock/quote data), which look like this when concatenated together:

irb(main):054:0> puts data.map { |s| s.instance_variable_get(:@_html_)}
<HEAD>
<TITLE>Error</TITLE>
</HEAD>

<BODY BGCOLOR="white" FGCOLOR="black">
<!-- status code : 404 -->
<!-- Error: GET -->
<!-- host machine: media-router46.prod.media.bf1.yahoo.com -->
<!-- timestamp: 1496865436.000 -->
<!-- url: https://ichart.finance.yahoo.com/table.csv?a=5&b=7&e=7&g=d&c=2016&d=5&f=2017&s=GOOG-->
<H1>Error</H1>
<HR>

<FONT FACE="Helvetica,Arial"><B>
Description: Could not process this "GET" request.
</B></FONT>
<HR>
</BODY>

The URL being requested )https://ichart.finance.yahoo.com/table.csv?a=5&b=7&e=7&g=d&c=2016&d=5&f=2017&s=GOOG) does indeed result in a 404 for me. Is this the correct URL to be fetching from? The same error appears when adjusting the date range (1 day, 30 days, and 90 days) from 1 year as well, and also has the same error when trying other ticker symbols also.

tyrauber commented 7 years ago

Confirmed. Unfortunately, it appears that yahoo deprecated this functionality 5/16/17. https://forums.yahoo.net/t5/Yahoo-Finance-help/Is-Yahoo-Finance-API-broken/td-p/250503

elperez commented 7 years ago

Does this means we cannot get historical data? I was getting this result - "Failed to open TCP connection to itable.finance.yahoo.com:80 (getaddrinfo: nodename nor servname provided, or not known)"

drusepth commented 7 years ago

Looks like the source it pulls from is completely (and intentionally, probably forever) down.

I've switched to the market_beat gem (which fetches from Google, I believe) for now and it works with a very similar interface. Fairly easy to swap out.

Thanks for all the awesome work though, @tyrauber.

tyrauber commented 7 years ago

Sorry folks, I dropped the ball on this one. I intended to develop a new approach to historical quotes (most likely switching back to the Google finance API), but failed to find the time. Given that any change of this magnitude would result in breaking changes in functionality, I think the better approach now is to raise an error notifying people that this functionality is no longer available in the structure they expected.

Master updated and version bumped to 1.4.0. Sorry for the troubles.

tyrauber / stock_quote

Quotes now result in server errors #38