tq_get(financials) not working with some stocks?

[phlsmk]

Hi, I have a list of 13 stocks, all listed in the US, and some but not all have issues getting financial statements (though none have issues getting stock prices)?

blah<-read.csv("tickers.csv",header=FALSE,colClasses=c("character"))

ffs <- tq_get(blah, get = "financials",complete_cases = TRUE) Warning messages: 1: x = 'COP', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files Removing COP. 2: x = 'APC', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files Removing APC. 3: x = 'MRO', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files Removing MRO. 4: x = 'CRC', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files Removing CRC. 5: x = 'EOG', get = 'financials': Error in download.file(paste(google.fin, Symbol, sep = ""), quiet = TRUE, : cannot download all files Removing EOG.

The full list of stocks is COP,APC,APA,HES,MRO,CLR,WLL,PXD,CRC,DVN,EOG,OAS,OXY,LPI

Thanks again

[mdancho84]

Try updating to then latest CRAN version of quantmod. Here's my test case that worked.

install.packages("quantmod")
library(tidyquant)

stock_list <- c("COP","APC","APA","HES","MRO","CLR","WLL","PXD","CRC","DVN","EOG","OAS","OXY","LPI")

stock_prices <- stock_list %>%
    tq_get()
stock_prices

stock_financials <- stock_list %>%
    tq_get(get = "financials")
stock_financials

[phlsmk]

Strangely I now get a different error (below)

Warning messages:
1: x = 'COP', get = 'financials': Error in thead[x]:thead[x + 1]: NA/NaN argument
 Removing COP. 
2: x = 'APC', get = 'financials': Error in thead[x]:thead[x + 1]: NA/NaN argument
 Removing APC. 
3: x = 'MRO', get = 'financials': Error in thead[x]:thead[x + 1]: NA/NaN argument
 Removing MRO. 
4: x = 'CRC', get = 'financials': Error in thead[x]:thead[x + 1]: NA/NaN argument
 Removing CRC. 
5: x = 'EOG', get = 'financials': Error in thead[x]:thead[x + 1]: NA/NaN argument
 Removing EOG. 

Here is my setup, all packages are up to date

> version
               _                           
platform       x86_64-apple-darwin15.6.0   
arch           x86_64                      
os             darwin15.6.0                
system         x86_64, darwin15.6.0        
status                                     
major          3                           
minor          4.0                         
year           2017                        
month          04                          
day            21                          
svn rev        72570                       
language       R                           
version.string R version 3.4.0 (2017-04-21)
nickname       You Stupid Darkness         

quantmod             "quantmod"             "/Library/Frameworks/R.framework/Versions/3.4/Resources/library" "0.4-9"      NA           

[mdancho84]

This is odd. Try this:

> quantmod::getFin("COP")
[1] "COP.f"

Then this:

> COP.f$IS
$Q
                                                 2017-03-31 2016-12-31 2016-09-30 2016-06-30 2016-03-31
Revenue                                             7497.00    6958.00    6513.00    5533.00    5143.00
Other Revenue, Total                                     NA         NA         NA         NA         NA
Total Revenue                                       7497.00    6958.00    6513.00    5533.00    5143.00
Cost of Revenue, Total                              4452.00    4587.00    4338.00    3317.00    3578.00
Gross Profit                                        3045.00    2371.00    2175.00    2216.00    1565.00
Selling/General/Admin. Expenses, Total               483.00     464.00     472.00     476.00     475.00
Research & Development                               551.00     343.00     457.00     610.00     505.00
Depreciation/Amortization                           1979.00    2061.00    2425.00    2329.00    2247.00
Interest Expense(Income) - Net Operating                 NA         NA         NA         NA         NA
...

Let me know your output and if it matches above.

[phlsmk]

Odd indeed, I get the error below.

> quantmod::getFin("COP")
Error in thead[x]:thead[x + 1] : NA/NaN argument

After looking at the quantmod code to see what URL is being loaded for the financials, it seems like https://www.google.com/finance?fstype=ii&q=COP does not necessarily give the data -- I get an error in my usual web browser, which seems to be the same error that the download.file call receives.

If I load that URL via a proxy it comes up fine. Meanwhile if I use a ticker "NYSE:COP" instead of just "COP" (https://www.google.com/finance?fstype=ii&q=NYSE:COP) it works in either case. Perhaps Google changes the default behaviour or assumed exchanges based on location (I'm in the UK?).

Will look some more, but perhaps a workaround would have to be including the exchange in all cases.

[mdancho84]

We are looking at using a different data source for financial statements that will likely correct the issue you are seeing, but it may be a bit before this is implemented. We can keep this open although this seems to be related to the underlying quantmod function and/or your web browser setup. If you're having success with the second URL, you might try this:

> tidyquant::tq_get("NYSE:COP", get = "financials")
# A tibble: 3 x 3
   type             annual            quarter
* <chr>             <list>             <list>
1    BS <tibble [168 x 4]> <tibble [210 x 4]>
2    CF  <tibble [76 x 4]>  <tibble [95 x 4]>
3    IS <tibble [196 x 4]> <tibble [245 x 4]>

[jar1karp]

Would it be possible to make tq_get more fault-tolerant?

Currently, if you download e.g. the DOW constituents by

constituents <- tq_index('DOW')

and then try to fetch financials by

financials <- tq_get(constituents$symbol, get='financials')

the code stop due to error in tq_get_map

Error in mutate_impl(.data, dots) : 
  Evaluation error: Evaluation error: Argument 2 must be length 0, not 2..
Called from: mutate_impl(.data, dots)

The reason is that DWDP which is a new component of DOW doesn't have financial data yet. I'm currently doing the following to work around this:

financials <- do.call(bind_rows, lapply(constituents$symbol, function(x) {
  dat <- NULL
  tryCatch(dat <- tq_get(x, get='financials') %>% mutate(symbol=x), error=function(e) {
    message('failed to get data for ', x)
  })
  dat
}))

It seems that downloading key.ratios just gives a warning, it would be convenient if financials were handled similarly.

[mdancho84]

We will take a look at improving the fault tolerance.