Get error in getSymbols("7203.T", src="yahooj")

[lobo-az]

Description

Call getSymbols("7203.T", src="yahooj") is raised XML error.

I tried use debug function.

debug(getSymbols.yahooj)
...
debug: URL <- paste(yahoo.URL, "?code=", Symbols.name, "&sm=", from.m, 
    "&sd=", sprintf("%.2d", from.d), "&sy=", from.y, "&em=", 
    to.m, "&ed=", sprintf("%.2d", to.d), "&ey=", to.y, "&tm=d", 
    "&p=", page, sep = "")
Browse[3]> n
debug: fdoc <- XML::htmlParse(URL)
Browse[3]> URL
[1] "https://info.finance.yahoo.co.jp/history/?code=7203.T&sm=1&sd=01&sy=2007&em=7&ed=11&ey=2020&tm=d&p=1"
Browse[3]> fdoc <- XML::htmlParse(URL)
Error: XML content does not seem to be XML: '1'

step through the execution of a function is https://github.com/joshuaulrich/quantmod/blob/c9777da6140dee9d7750cf5474df0481c2bc7c57/R/getSymbols.R#L501

Next, open URL https://info.finance.yahoo.co.jp/history/?code=7203.T&sm=1&sd=01&sy=2007&em=7&ed=11&ey=2020&tm=d&p=1 is displayed stock price of TOYOTA.

Expected behavior

Get stock price of TOYOTA.

Minimal, reproducible example

> getSymbols("7203.T", src="yahooj")
Error in getSymbols.yahooj(Symbols = "7203.T", env = <environment>, verbose = FALSE,  : 
  Unable to import “7203.T”.
XML content does not seem to be XML: '1'

Session Info

R version 4.0.0 (2020-04-24)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/atlas-base/atlas/libblas.so.3.0
LAPACK: /usr/lib/atlas-base/atlas/liblapack.so.3.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C          
 [3] LC_TIME=C.UTF-8        LC_COLLATE=C.UTF-8    
 [5] LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C             
 [9] LC_ADDRESS=C           LC_TELEPHONE=C        
[11] LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

attached base packages:
[1] stats     graphics  grDevices utils     datasets 
[6] methods   base     

other attached packages:
[1] XML_3.99-0.4    quantmod_0.4.17 TTR_0.23-6     
[4] xts_0.12-0      zoo_1.8-8      

loaded via a namespace (and not attached):
[1] compiler_4.0.0  tools_4.0.0     curl_4.3       
[4] grid_4.0.0      lattice_0.20-41

[joshuaulrich]

Thanks for the report! I can replicate. It looks like the issue is that XML::htmlParse() doesn't recognize https:// as a URL. We should be able to fix by setting isURL = TRUE in the call.

[joshuaulrich]

That didn't work. Investigating other approaches now.

[joshuaulrich]

Just pushed a commit that fixes your example for me. I'd really appreciate it if you could test other stocks too. Especially some stocks that have splits and/or dividends in the time window you pull. I noticed that Toyota didn't have any splits between 2007-2020.

Let me know if you run into any issues while testing.

[ethanbsmith]

ive run into this before and used an explicit call to curl::curl(URL) to get around it. that said, moving to xml2 is prolly the right call here as well

[lobo-az]

I tryied branch 310-yahooj-error.

> library(devtools)
> devtools::install_github("joshuaulrich/quantmod", ref="310-yahooj-error")
> library(quantmod)
> > x<-getSymbols("4445.T", src="yahooj", from="2020-05-10", to="2020-05-15", auto.assign=FALSE)
> x
           YJ4445.T.Open YJ4445.T.High YJ4445.T.Low
2020-05-11          1380          1406         1315
2020-05-12          1423          1465         1375
2020-05-13          1465          1530         1465
2020-05-14          1514          1580         1439
2020-05-15          1471          1565         1471
> y<-getSymbols("8306.T", src="yahooj", from="2020-05-10", to="2020-05-15", auto.assign=FALSE)
> y
           YJ8306.T.Open YJ8306.T.High YJ8306.T.Low
2020-05-11         421.0         426.4        420.2
2020-05-12         420.0         420.8        414.3
2020-05-13         412.0         420.6        411.5
2020-05-14         414.4         415.4        406.1
2020-05-15         410.0         410.3        403.0
> z<-getSymbols("7203.T", src="yahooj", from="2020-05-10", to="2020-05-15", auto.assign=FALSE)
> z
           YJ7203.T.Open YJ7203.T.High YJ7203.T.Low
2020-05-11          6620          6675         6582
2020-05-12          6610          6655         6485
2020-05-13          6412          6460         6372
2020-05-14          6300          6326         6243
2020-05-15          6300          6320         6208

Thanks @joshuaulrich ! 😄

I'd really appreciate it if you could test other stocks too.

I was able to get it with no problem. 🙆‍♂️

[tchevri]

Thanks a lot!! This also solves #312 , much appreciated, including @lobo1981 's clear resolution. Just a pity Google never returned this thread. devtools::install_github("joshuaulrich/quantmod") solved the issue for me.