search

business-science / tidyquant

Bringing financial analysis to the tidyverse
https://business-science.github.io/tidyquant/
Other
840 stars 175 forks source link

Key Ratios / Object Not Found #93

Closed ghost closed 6 years ago

ghost commented 6 years ago

Thanks for all of your efforts.

Console log below... running version 0.5.4

library(tidyquant)

aapl_key_ratios <- tq_get("AAPL", get = "key.ratios") Parsed with column specification: cols( Growth Profitability and Financial Ratios for Apple Inc = col_character() ) additional arguments ignored in warning() Warning messages: 1: In rbind(names(probs), probs_f) : number of columns of result is not a multiple of vector length (arg 1) 2: x = 'AAPL', get = 'key.ratios': Error in value[3L]: object 'key_ratios' not found

ibm_key_ratios <- tq_get("IBM", get = "key.ratios") Parsed with column specification: cols( Growth Profitability and Financial Ratios for International Business Machines Corp = col_character() ) additional arguments ignored in warning() Warning messages: 1: In rbind(names(probs), probs_f) : number of columns of result is not a multiple of vector length (arg 1) 2: x = 'IBM', get = 'key.ratios': Error in value[3L]: object 'key_ratios' not found

mdancho84 commented 6 years ago

Looks to be working for me. Let me know if you are still having the issue. 

> tq_get("AAPL", get = "key.ratios") %>% unnest()
# A tibble: 890 x 6
   section    sub.section group category        date        value
   <chr>      <chr>       <dbl> <chr>           <date>      <dbl>
 1 Financials Financials   1.00 Revenue USD Mil 2008-09-01  37491
 2 Financials Financials   1.00 Revenue USD Mil 2009-09-01  42905
 3 Financials Financials   1.00 Revenue USD Mil 2010-09-01  65225
 4 Financials Financials   1.00 Revenue USD Mil 2011-09-01 108249
 5 Financials Financials   1.00 Revenue USD Mil 2012-09-01 156508
 6 Financials Financials   1.00 Revenue USD Mil 2013-09-01 170910
 7 Financials Financials   1.00 Revenue USD Mil 2014-09-01 182795
 8 Financials Financials   1.00 Revenue USD Mil 2015-09-01 233715
 9 Financials Financials   1.00 Revenue USD Mil 2016-09-01 215639
10 Financials Financials   1.00 Revenue USD Mil 2017-09-01 229234
# ... with 880 more rows
ghost commented 6 years ago

Thanks for your response. I'm trying on my machine and still getting the error...

tq_get("AAPL", get = "key.ratios") %>% unnest() Parsed with column specification: cols( Growth Profitability and Financial Ratios for Apple Inc = colcharacter() ) additional arguments ignored in warning() Error in UseMethod("unnest") : no applicable method for 'unnest_' applied to an object of class "logical" In addition: Warning messages: 1: In rbind(names(probs), probs_f) : number of columns of result is not a multiple of vector length (arg 1) 2: x = 'AAPL', get = 'key.ratios': Error in value[3L]: object 'key_ratios' not found

mdancho84 commented 6 years ago

I'm sorry to hear this, but unfortunately, I'm having difficulty replicating your issue. It may be something local to your configuration. Trying restarting R, updating packages, and rerunning the example.

DavisVaughan commented 6 years ago

Can you try two things to help us out?

1) Run the code, then immediately run traceback(). Hopefully you get something useful. Just copy and paste what you get from that.

2) Run the code, then run devtools::session_info(). Paste that too so we can see what you are running.

ghost commented 6 years ago

Thanks @DavisVaughan - here is the full console....

library(tidyquant)
Loading required package: lubridate

Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

    date

Loading required package: PerformanceAnalytics
Loading required package: xts
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Package PerformanceAnalytics (1.4.3541) loaded.
Copyright (c) 2004-2014 Peter Carl and Brian G. Peterson, GPL-2 | GPL-3
http://r-forge.r-project.org/projects/returnanalytics/

Attaching package: ‘PerformanceAnalytics’

The following object is masked from ‘package:graphics’:

    legend

Loading required package: quantmod
Loading required package: TTR
Version 0.4-0 included new data defaults. See ?getSymbols.
Learn from a quantmod author: https://www.datacamp.com/courses/importing-and-managing-financial-data-in-r
Loading required package: tidyverse
-- Attaching packages --------------------------------------- tidyverse 1.2.1 --
v ggplot2 2.2.1     v purrr   0.2.4
v tibble  1.4.2     v dplyr   0.7.4
v tidyr   0.8.0     v stringr 1.3.0
v readr   1.1.1     v forcats 0.3.0
-- Conflicts ------------------------------------------ tidyverse_conflicts() --
x lubridate::as.difftime() masks base::as.difftime()
x lubridate::date()        masks base::date()
x dplyr::filter()          masks stats::filter()
x dplyr::first()           masks xts::first()
x lubridate::intersect()   masks base::intersect()
x dplyr::lag()             masks stats::lag()
x dplyr::last()            masks xts::last()
x lubridate::setdiff()     masks base::setdiff()
x lubridate::union()       masks base::union()
> tq_get("AAPL", get = "key.ratios") %>% unnest()
Parsed with column specification:
cols(
  `Growth Profitability and Financial Ratios for Apple Inc` = col_character()
)
additional arguments ignored in warning()
Error in UseMethod("unnest_") : 
  no applicable method for 'unnest_' applied to an object of class "logical"
In addition: Warning messages:
1: In rbind(names(probs), probs_f) :
  number of columns of result is not a multiple of vector length (arg 1)
2: x = 'AAPL', get = 'key.ratios': Error in value[[3L]](cond): object 'key_ratios' not found

> traceback()
11: unnest_(data, unnest_cols = unnest_cols, .drop = .drop, .id = .id, 
        .sep = .sep, .preserve = .preserve)
10: unnest.default(.)
9: unnest(.)
8: function_list[[k]](value)
7: withVisible(function_list[[k]](value))
6: freduce(value, `_function_list`)
5: `_fseq`(`_lhs`)
4: eval(quote(`_fseq`(`_lhs`)), env, env)
3: eval(quote(`_fseq`(`_lhs`)), env, env)
2: withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
1: tq_get("AAPL", get = "key.ratios") %>% unnest()
> devtools::session_info()
Session info -----------------------------------------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.4.3 (2017-11-30)
 system   x86_64, mingw32             
 ui       RStudio (1.1.383)           
 language (EN)                        
 collate  English_United States.1252  
 tz       America/New_York            
 date     2018-02-22                  

Packages ---------------------------------------------------------------------------------------------------------------------------------------------
 package              * version  date       source        
 assertthat             0.2.0    2017-04-11 CRAN (R 3.4.3)
 base                 * 3.4.3    2017-12-06 local         
 bindr                  0.1      2016-11-13 CRAN (R 3.4.3)
 bindrcpp               0.2      2017-06-17 CRAN (R 3.4.3)
 broom                  0.4.3    2017-11-20 CRAN (R 3.4.3)
 cellranger             1.1.0    2016-07-27 CRAN (R 3.4.3)
 cli                    1.0.0    2017-11-05 CRAN (R 3.4.3)
 colorspace             1.3-2    2016-12-14 CRAN (R 3.4.3)
 compiler               3.4.3    2017-12-06 local         
 crayon                 1.3.4    2017-09-16 CRAN (R 3.4.3)
 curl                   3.1      2017-12-12 CRAN (R 3.4.3)
 datasets             * 3.4.3    2017-12-06 local         
 devtools               1.13.5   2018-02-18 CRAN (R 3.4.3)
 digest                 0.6.15   2018-01-28 CRAN (R 3.4.3)
 dplyr                * 0.7.4    2017-09-28 CRAN (R 3.4.3)
 forcats              * 0.3.0    2018-02-19 CRAN (R 3.4.3)
 foreign                0.8-69   2017-06-22 CRAN (R 3.4.3)
 ggplot2              * 2.2.1    2016-12-30 CRAN (R 3.4.3)
 glue                   1.2.0    2017-10-29 CRAN (R 3.4.3)
 graphics             * 3.4.3    2017-12-06 local         
 grDevices            * 3.4.3    2017-12-06 local         
 grid                   3.4.3    2017-12-06 local         
 gtable                 0.2.0    2016-02-26 CRAN (R 3.4.3)
 haven                  1.1.1    2018-01-18 CRAN (R 3.4.3)
 hms                    0.4.1    2018-01-24 CRAN (R 3.4.3)
 httr                   1.3.1    2017-08-20 CRAN (R 3.4.3)
 jsonlite               1.5      2017-06-01 CRAN (R 3.4.3)
 lattice                0.20-35  2017-03-25 CRAN (R 3.4.3)
 lazyeval               0.2.1    2017-10-29 CRAN (R 3.4.3)
 lubridate            * 1.7.2    2018-02-06 CRAN (R 3.4.3)
 magrittr               1.5      2014-11-22 CRAN (R 3.4.3)
 memoise                1.1.0    2017-04-21 CRAN (R 3.4.3)
 methods              * 3.4.3    2017-12-06 local         
 mnormt                 1.5-5    2016-10-15 CRAN (R 3.4.1)
 modelr                 0.1.1    2017-07-24 CRAN (R 3.4.3)
 munsell                0.4.3    2016-02-13 CRAN (R 3.4.3)
 nlme                   3.1-131  2017-02-06 CRAN (R 3.4.3)
 parallel               3.4.3    2017-12-06 local         
 PerformanceAnalytics * 1.4.3541 2014-09-16 CRAN (R 3.4.3)
 pillar                 1.1.0    2018-01-14 CRAN (R 3.4.3)
 pkgconfig              2.0.1    2017-03-21 CRAN (R 3.4.3)
 plyr                   1.8.4    2016-06-08 CRAN (R 3.4.3)
 psych                  1.7.8    2017-09-09 CRAN (R 3.4.3)
 purrr                * 0.2.4    2017-10-18 CRAN (R 3.4.3)
 Quandl                 2.8.0    2016-04-23 CRAN (R 3.4.3)
 quantmod             * 0.4-12   2017-12-10 CRAN (R 3.4.3)
 R6                     2.2.2    2017-06-17 CRAN (R 3.4.3)
 Rcpp                   0.12.15  2018-01-20 CRAN (R 3.4.3)
 readr                * 1.1.1    2017-05-16 CRAN (R 3.4.3)
 readxl                 1.0.0    2017-04-18 CRAN (R 3.4.3)
 reshape2               1.4.3    2017-12-11 CRAN (R 3.4.3)
 rlang                  0.1.6    2017-12-21 CRAN (R 3.4.3)
 rstudioapi             0.7      2017-09-07 CRAN (R 3.4.3)
 rvest                  0.3.2    2016-06-17 CRAN (R 3.4.3)
 scales                 0.5.0    2017-08-24 CRAN (R 3.4.3)
 stats                * 3.4.3    2017-12-06 local         
 stringi                1.1.6    2017-11-17 CRAN (R 3.4.2)
 stringr              * 1.3.0    2018-02-19 CRAN (R 3.4.3)
 tibble               * 1.4.2    2018-01-22 CRAN (R 3.4.3)
 tidyquant            * 0.5.4    2018-02-19 CRAN (R 3.4.3)
 tidyr                * 0.8.0    2018-01-29 CRAN (R 3.4.3)
 tidyverse            * 1.2.1    2017-11-14 CRAN (R 3.4.3)
 tools                  3.4.3    2017-12-06 local         
 TTR                  * 0.23-3   2018-01-24 CRAN (R 3.4.3)
 utf8                   1.1.3    2018-01-03 CRAN (R 3.4.3)
 utils                * 3.4.3    2017-12-06 local         
 withr                  2.1.1    2017-12-19 CRAN (R 3.4.3)
 xml2                   1.2.0    2018-01-24 CRAN (R 3.4.3)
 xts                  * 0.10-1   2017-12-20 CRAN (R 3.4.3)
 yaml                   2.1.16   2017-12-12 CRAN (R 3.4.3)
 zoo                  * 1.8-1    2018-01-08 CRAN (R 3.4.3)
DavisVaughan commented 6 years ago

What does this give you?

library(magrittr)

url <- c("http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:AAPL&region=usa&culture=en-US&cur=&order=asc",
"http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNYS:AAPL&region=usa&culture=en-US&cur=&order=asc",
"http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XASE:AAPL&region=usa&culture=en-US&cur=&order=asc")

for(i in 1:3) {
    text <- httr::RETRY("GET", url[i], times = 5) %>%
        httr::content()

    if(!is.null(text)) {

        # Test to see if file returned is just a message containing "We're sorry"
        text_test <- text %>%
            xml2::as_list() %>%
            unlist() %>%
            stringr::str_detect("^We.re sorry")

        # If text does not contain "We're sorry" message, break
        if (!text_test) {
            break
        }
    }
}

text
ghost commented 6 years ago

Thanks again for your response. Provided the console below. Still striking out unfortunately.

library(magrittr)

Attaching package: ‘magrittr’

The following object is masked from ‘package:purrr’:

    set_names

The following object is masked from ‘package:tidyr’:

    extract

> url <- c("http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:AAPL&region=usa&culture=en-US&cur=&order=asc",
+         .... [TRUNCATED] 

> for(i in 1:3) {
+   text <- httr::RETRY("GET", url[i], times = 5) %>%
+     httr::content()
+   
+   if(!is.null(text)) {
+     
+     # Test to see .... [TRUNCATED] 
Parsed with column specification:
cols(
  `Growth Profitability and Financial Ratios for Apple Inc` = col_character()
)
Warning: 93 parsing failures.
row # A tibble: 5 x 5 col     row col   expected  actual     file         expected   <int> <chr> <chr>     <chr>      <chr>        actual 1     2 NA    1 columns 12 columns <raw vector> file 2     3 NA    1 columns 12 columns <raw vector> row 3     4 NA    1 columns 12 columns <raw vector> col 4     5 NA    1 columns 12 columns <raw vector> expected 5     6 NA    1 columns 12 columns <raw vector>
... ................................. ... ............................................... ........ ............................................................................................................................................................................ ...... ............................. [... truncated]
Error in UseMethod("as_list") : 
  no applicable method for 'as_list' applied to an object of class "c('tbl_df', 'tbl', 'data.frame')"
In addition: Warning message:
In rbind(names(probs), probs_f) :
  number of columns of result is not a multiple of vector length (arg 1)

Am able to load the text object and view this tibble, but not quite there. image

DavisVaughan commented 6 years ago

This is good actually. This is NOT what I get, so we have isolated the problem. I'm curious as to what this gives you.

resp <- httr::RETRY("GET", url = "http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&callback=?&t=XNAS:AAPL&region=usa&culture=en-US&cur=&order=asc", times = 5)

resp

resp$headers[["Content-Type"]]

httr::content(resp)

I would really love if you could run this as a reprex to get as much info as we can out of it.

1) Run install.packages("reprex") 2) Copy the code I pasted above into a new R script in RStudio 3) Highlight all of the code while it is sitting in the new R script and copy it (Command + C) on Mac 4) Run reprex::reprex() in the R console. This should run the code and present you with the HTML output over in the viewer. That fancy output will now be on your clipboard (meaning you can paste it with Command+V) 5) Come back to GitHub, open a new comment, and paste the output. It should be properly formatted so all you have to do is submit the comment.

ghost commented 6 years ago

For some reason my clipboard is not cooperating, but hopefully this offers some clues...

`

resp <- httr::RETRY("GET", url = "http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&amp;callback=?&amp;t=XNAS:AAPL&amp;region=usa&amp;culture=en-US&amp;cur=&amp;order=asc&quot;, times = 5)


resp


> Response [http://financials.morningstar.com/finan/ajax/exportKR2CSV.html?&amp;callback=?&amp;t=XNAS:AAPL&amp;region=usa&amp;culture=en-US&amp;cur=&amp;order=asc]


>   Date: 2018-02-22 21:11


>   Status: 200


>   Content-Type: text/csv;charset=UTF-8


>   Size: 7.9 kB


> <U+FEFF>Growth Profitability and Financial Ratios for Apple Inc


> Financials


> ,2008-09,2009-09,2010-09,2011-09,2012-09,2013-09,2014-09,2015-09,2016-09...


> Revenue USD Mil,"37,491","42,905","65,225","108,249","156,508","170,910"...


> Gross Margin %,35.2,40.1,39.4,40.5,43.9,37.6,38.6,40.1,39.1,38.5,38.4


> Operating Income USD Mil,"8,327","11,740","18,385","33,790","55,241","48...


> Operating Margin %,22.2,27.4,28.2,31.2,35.3,28.7,28.7,30.5,27.8,26.8,26.9


> Net Income USD Mil,"6,119","8,235","14,013","25,922","41,733","37,037","...


> Earnings Per Share USD,0.97,1.30,2.16,3.95,6.31,5.68,6.45,9.22,8.31,9.21...


> Dividends USD,,,,,0.38,1.63,1.81,1.98,2.18,2.40,2.46


> ...


resp$headers[["Content-Type"]]


> [1] "text/csv;charset=UTF-8"


httr::content(resp)


> Parsed with column specification:


> cols(


>   Growth Profitability and Financial Ratios for Apple Inc = col_character()


> )


> Warning in rbind(names(probs), probs_f): number of columns of result is not


> a multiple of vector length (arg 1)


> Warning: 93 parsing failures.


> row # A tibble: 5 x 5 col     row col   expected  actual     file         expected   <int> <chr> <chr>     <chr>      <chr>        actual 1     2 <NA>  1 columns 12 columns <raw vector> file 2     3 <NA>  1 columns 12 columns <raw vector> row 3     4 <NA>  1 columns 12 columns <raw vector> col 4     5 <NA>  1 columns 12 columns <raw vector> expected 5     6 <NA>  1 columns 12 columns <raw vector>


> ... ................. ... ............................................... ........ ............................................... ...... ............................................... .... ............................................... ... ............................................... ... ............................................... ........ ...............................................


> See problems(...) for more details.


> # A tibble: 103 x 1


>    Growth Profitability and Financial Ratios for Apple Inc


>    <chr>


>  1 Financials


>  2 <NA>


>  3 Revenue USD Mil


>  4 Gross Margin %


>  5 Operating Income USD Mil


>  6 Operating Margin %


>  7 Net Income USD Mil


>  8 Earnings Per Share USD


>  9 Dividends USD


> 10 Payout Ratio % *


> # ... with 93 more rows

`

Also provided this in the console... `reprex(input = "reprex.R") Unable to put result on the clipboard. How to get it:

ghost commented 6 years ago

Any updates here or should I give it the old uninstall/reinstall + restart + clear my internet cache? =P

DavisVaughan commented 6 years ago

Try this for me, I'm optimistic.

resp <- httr::RETRY("GET", url = "http://financials.morningstar.com/ajax/exportKR2CSV.html?t=XNAS:AAPL", times = 5)
httr::content(resp, type = "text/html", encoding = "UTF-8")
ghost commented 6 years ago

That code ran successfully...

resp <- httr::RETRY("GET", url = "http://financials.morningstar.com/ajax/exportKR2CSV.html?t=XNAS:AAPL", times = 5)
> httr::content(resp, type = "text/html", encoding = "UTF-8")
{xml_document}
<html>
[1] <body><p>Growth Profitability and Financial Ratios for Apple Inc\nFinancials\n,2008-09,2009-09,2010-09,2011-09,2012-09,2013-09,2014-09,2015-09,2 ...
DavisVaughan commented 6 years ago

Great. Your "response" was coming back as "text/csv" but we needed it as "text/html" to be parsed correctly. Not sure why yours did that, but we can force it using type = "text/html". I'll fix this in the dev version of tidyquant.

DavisVaughan commented 6 years ago

@johnohsner Can you try installing the dev version? Hopefully youre able to.

devtools::install_github("business-science/tidyquant")`
ghost commented 6 years ago

We're in business Davis. Resolved. thanks for your efforts.