search

joshuaulrich / quantmod

Quantitative Financial Modelling Framework
http://www.quantmod.com/
GNU General Public License v3.0
798 stars 219 forks source link

getSymbols unable to import #332

Closed stakerauo closed 3 years ago

stakerauo commented 3 years ago

Description

I have created a vector with all my tickers in, and I've created a for loop that gives me the symbols that I want when I set from = "2021-01-01". However, when I set from "2020-02-01" it tells me it's unable to import. I've looked at Yahoo Finance and the price data is available.

Expected behavior

I hope for getSymbols to return stock price data from up to two years back, and probably rid of the following error.

Warning: AEGA.OL download failed; trying again.
 Error in getSymbols.yahoo(Symbols = "AEGA.OL", env = <environment>, verbose = FALSE,  : 
  Unable to import “AEGA.OL”.
argument "conn" is missing, with no default

Minimal, reproducible example

sym.vd = c("AEGA.OL", "AKAST.OL", "AMSC.OL", "AUSS.OL",
           "AGAS.OL", "AWDR.OL", "ALNG.OL", "AXXIS.OL")
#Total of 71 tickers.

Stocks = lapply(sym.vd, function(sym) {
  dailyReturn(na.locf(getSymbols(sym, from = "2020-02-01", auto.assign = FALSE)))
})

Session Info

R version 3.6.2 (2019-12-12)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19041)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

other attached packages:
[1] quantmod_0.4.18 TTR_0.23-6      xts_0.12-0      zoo_1.8-8      
[5] expsmooth_2.3   fma_2.4         forecast_8.12   ggplot2_3.3.3  
[9] fpp2_2.4       

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.4.6      urca_1.3-0        compiler_3.6.2   
 [4] pillar_1.4.4      tseries_0.10-48   tools_3.6.2      
 [7] nlme_3.1-142      lattice_0.20-38   lifecycle_1.0.0  
[10] tibble_3.0.1      gtable_0.3.0      pkgconfig_2.0.3  
[13] rlang_0.4.10      cli_2.0.2         DBI_1.1.1        
[16] rstudioapi_0.11   curl_4.3          parallel_3.6.2   
[19] withr_2.4.1       dplyr_1.0.5       generics_0.0.2   
[22] vctrs_0.3.6       nnet_7.3-12       lmtest_0.9-37    
[25] grid_3.6.2        tidyselect_1.1.0  glue_1.4.1       
[28] R6_2.4.1          fansi_0.4.1       purrr_0.3.4      
[31] magrittr_1.5      scales_1.1.1      ellipsis_0.3.1   
[34] assertthat_0.2.1  timeDate_3043.102 colorspace_1.4-1 
[37] fracdiff_1.5-1    quadprog_1.5-8    munsell_0.5.0    
[40] crayon_1.3.4

Final note

I appreciate any comment that could lead me in the right direction. I've googled for solutions, and looked around anywhere, but I can't seem to find anything similar. Thank you!

joshuaulrich commented 3 years ago

Your example works for me. Does it work for you? Maybe you only get the error if you have more tickers?

joshuaulrich commented 3 years ago

I'm going to close this since I can't replicate and this doesn't seem to be an issue with the package. That said, I'll still try to help if you're still having issues.

stakerauo commented 3 years ago

Hey, I was just about to reply, but looks like I was just a minute too late. The issue prevails. I've since tried updating my R console, seeing that I was using 3.6.2, and quantmod was made for 3.6.3. I'm now using 4.0.5.

I've shortened the ticker list to four tickers, and it works as it should now(it didn't at 3.6.2), but as you suggested, the full list is what's causing the issue. I'll attach the full vector below, if you want to give it a shot at replicating. I'm thinking maybe it could be the size of the data set that is too large for RStudio to handle, but I don't know how to approach the issue.

sym.vd = c("AEGA.OL", "AKAST.OL", "AMSC.OL", "AUSS.OL", "AGAS.OL", "AWDR.OL", "ALNG.OL", "AXXIS.OL", "B2H.OL", "BONHR.OL", "BOR.OL", "BORR.OL", "BWLPG.OL", "DNO.OL", "DOF.OL", "EAM.OL", "EIOF.OL", "ELE.OL", "ELK.OL", "EPIC.OL", "EQNR.OL", "RISH.OL", "GRONG.OL", "HAVI.OL", "HBC.OL", "HSPG.OL", "IDEX.OL", "INSR.OL", "JAREN.OL", "JIN.OL", "KOMP.OL", "KOA.OL", "NAPA.OL", "NEXT.OL", "NANOV.OL", "NORTH.OL", "OCY.OL", "ODL.OL", "ODF.OL", "OLT.OL", "PGS.OL", "PHLY.OL", "PLCS.OL", "PRS.OL", "QEC.OL", "REACH.OL", "RECSI.OL", "SDSD.OL", "SADG.OL", "SBX.OL", "SHLF.OL", "SIOFF.OL", "SBVG.OL", "NONG.OL", "SNOR.OL", "SOAG.OL", "MING.OL", "SVEG.OL", "SOR.OL", "MORG.OL", "SNI.OL", "STB.OL", "SUBC.OL", "SUNSB.OL", "TGS.OL", "VVL.OL", "WAWI.OL", "WILS.OL", "WWI.OL", "XXL.OL", "YAR.OL")

joshuaulrich commented 3 years ago

Hey, I was just about to reply, but looks like I was just a minute too late.

:)

The issue prevails.

Weird, I still can't replicate. This works for me. Can you run with verbose = TRUE in the getSymbols() call? That may give me a better idea where the issue is. The output of traceback() after the error would be helpful too.

sym.vd = c("AEGA.OL", "AKAST.OL", "AMSC.OL", "AUSS.OL", "AGAS.OL", "AWDR.OL",
  "ALNG.OL", "AXXIS.OL", "B2H.OL", "BONHR.OL", "BOR.OL", "BORR.OL", "BWLPG.OL",
  "DNO.OL", "DOF.OL", "EAM.OL", "EIOF.OL", "ELE.OL", "ELK.OL", "EPIC.OL",
  "EQNR.OL", "RISH.OL", "GRONG.OL", "HAVI.OL", "HBC.OL", "HSPG.OL", "IDEX.OL",
  "INSR.OL", "JAREN.OL", "JIN.OL", "KOMP.OL", "KOA.OL", "NAPA.OL", "NEXT.OL",
  "NANOV.OL", "NORTH.OL", "OCY.OL", "ODL.OL", "ODF.OL", "OLT.OL", "PGS.OL",
  "PHLY.OL", "PLCS.OL", "PRS.OL", "QEC.OL", "REACH.OL", "RECSI.OL", "SDSD.OL",
  "SADG.OL", "SBX.OL", "SHLF.OL", "SIOFF.OL", "SBVG.OL", "NONG.OL", "SNOR.OL",
  "SOAG.OL", "MING.OL", "SVEG.OL", "SOR.OL", "MORG.OL", "SNI.OL", "STB.OL",
  "SUBC.OL", "SUNSB.OL", "TGS.OL", "VVL.OL", "WAWI.OL", "WILS.OL", "WWI.OL",
  "XXL.OL", "YAR.OL")

Stocks = lapply(sym.vd, function(sym) {
  dailyReturn(na.locf(getSymbols(sym, from = "2020-02-01", auto.assign = FALSE)))
})
stakerauo commented 3 years ago

Sure thing! Ran it with Verbose = TRUE:

downloading  AEGA.OL .....

Warning: AEGA.OL download failed; trying again.
 Error in getSymbols.yahoo(Symbols = "AEGA.OL", env = <environment>, verbose = TRUE,  : 
  Unable to import “AEGA.OL”.
argument "conn" is missing, with no default`

and then from traceback()

11: stop(msg)
10: getSymbols.yahoo(Symbols = "AEGA.OL", env = <environment>, verbose = TRUE, 
        warnings = TRUE, auto.assign = FALSE, from = "2020-07-01", 
        .has1sym. = TRUE)
9: do.call(paste("getSymbols.", symbol.source, sep = ""), list(Symbols = current.symbols, 
       env = env, verbose = verbose, warnings = warnings, auto.assign = auto.assign, 
       ..., .has1sym. = .has1sym.))
8: getSymbols(sym, verbose = TRUE, from = "2020-07-01", auto.assign = FALSE)
7: na.locf(getSymbols(sym, verbose = TRUE, from = "2020-07-01", 
       auto.assign = FALSE))
6: is.xts(x)
5: try.xts(x)
4: periodReturn(x, "daily", subset, type, leading, ...)
3: dailyReturn(na.locf(getSymbols(sym, verbose = TRUE, from = "2020-07-01", 
       auto.assign = FALSE))) at #2
2: FUN(X[[i]], ...)
1: lapply(sym.vd, function(sym) {
       dailyReturn(na.locf(getSymbols(sym, verbose = TRUE, from = "2020-07-01", 
           auto.assign = FALSE)))
   })
stakerauo commented 3 years ago

If you're still looking in the error; I've noticed at the first attempt, I'm also getting this error:

Error in curl::curl_fetch_memory(cu, handle = h) : 
  transfer closed with outstanding read data remaining

with the following traceback:

13.
curl::curl_fetch_memory(cu, handle = h) 
12.
new.session() 
11.
.getHandle(curl.options) 
10.
getSymbols.yahoo(Symbols = "AEGA.OL", env = <environment>, verbose = FALSE, 
    warnings = TRUE, auto.assign = FALSE, from = "2020-02-01", 
    .has1sym. = TRUE) 
9.
do.call(paste("getSymbols.", symbol.source, sep = ""), list(Symbols = current.symbols, 
    env = env, verbose = verbose, warnings = warnings, auto.assign = auto.assign, 
    ..., .has1sym. = .has1sym.)) 
8.
getSymbols(sym, from = "2020-02-01", auto.assign = FALSE) 
7.
na.locf(getSymbols(sym, from = "2020-02-01", auto.assign = FALSE)) 
6.
is.xts(x) 
5.
try.xts(x) 
4.
periodReturn(x, "daily", subset, type, leading, ...) 
3.
dailyReturn(na.locf(getSymbols(sym, from = "2020-02-01", auto.assign = FALSE))) 
2.
FUN(X[[i]], ...) 
1.
lapply(sym.vd, function(sym) {
    dailyReturn(na.locf(getSymbols(sym, from = "2020-02-01", 
        auto.assign = FALSE)))
})
stakerauo commented 3 years ago

I was finally able to resolve the issue. Turns out, there was a problem with the RTools software. I reinstalled RTools, then I ran the following line of code, as recommended on the cran.rstudio.com:

writeLines('PATH="${RTOOLS40_HOME}\\usr\\bin;${PATH}"', con = "~/.Renviron")
Sys.which("make")
install.packages("jsonlite", type = "source")

The problem must have been somewhere along those lines. I recognize the argument con = was mentioned in the error as "missing with no default", so my guess is I was missing the .Renviron file.

Thanks a lot, Joshua. I really appreciate that you tried to help me.

joshuaulrich commented 3 years ago

Thanks a lot, Joshua. I really appreciate that you tried to help me.

You're welcome! I'm glad you were able to figure it out.

downloading  AEGA.OL .....

Warning: AEGA.OL download failed; trying again.
 Error in getSymbols.yahoo(Symbols = "AEGA.OL", env = <environment>, verbose = TRUE,  : 
  Unable to import “AEGA.OL”.
argument "conn" is missing, with no default`

Also, I fixed this in d4c6ec56054b9fec4c7bba69366a8e23fceb9ed8. Now it gives a better error message.