Yahoo Japan Error in getSymbols.yahooj XML content does not seem to be XML: '1'

[tchevri]

Description

I have a piece of code that used to run without issues for many years and here is a great opportunity to extend a warm public thank you for your awesome code (and all the help you have provided to so many folks across so many forums).

require(quantmod)
getSymbols('YJ1306.T', from="1900-01-01", src = 'yahooj', auto.assign=TRUE)

I just ran it for the first time in a few months (it was working fine in March) and I received:

Error in getSymbols.yahooj(Symbols = "YJ1306.T", env = , : Unable to import “YJ1306.T”. XML content does not seem to be XML: '1'

Expected behavior

Usually, this call creates a variable YJ1306.T with the downloaded series.

Minimal, reproducible example

require(quantmod)
getSymbols('YJ1306.T', from="1900-01-01", src = 'yahooj', auto.assign=TRUE)

New section, if I may: what I tried!

put debug() around the function call. I can see that the function calls fdoc <- XML::htmlParse(URL) where URL points to Yahoo Japan historical data

This looks correct, so I am a little shocked this call seems to be failing... Looks fine to me. This said, when I tried to browse directly to the security historical data page, I landed here. I am not sure this is relevant, but best is to let you decide, since you would know.

MUCH appreciated any help.

Session Info

R version 4.0.2 (2020-06-22)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] quantmod_0.4.17 TTR_0.24.1      xts_0.12-0      zoo_1.8-8      

loaded via a namespace (and not attached):
 [1] bookdown_0.20   lattice_0.20-41 XML_3.99-0.5    digest_0.6.25   grid_4.0.2      evaluate_0.14  
 [7] rlang_0.4.7     curl_4.3        rmarkdown_2.3.3 tools_4.0.2     xfun_0.16       yaml_2.2.1     
[13] compiler_4.0.2  htmltools_0.5.0 knitr_1.29  

[joshuaulrich]

Thanks for the report! This may be related to #310. Please install the latest from this repo and see if that fixes the problem for you. I just tried that and it worked for me.

[tchevri]

Thanks a ton! Same issue indeed - I am so sorry I wasted your time: epic Google failure which did not show #310 at all, bit infuriating...

[joshuaulrich]

No worries! It only took me a few minutes to test. And I'm happy it's already fixed!

[joshuaulrich]

epic Google failure

Not sure if you tried this, but sometimes doing an issue search on GitHub can help.

[tchevri]

Dear @joshuaulrich Looks like the issue is back! Any ideas?! Fyi - not the only one affected: see here

[joshuaulrich]

Hi @tchevri. I've been working on this. Could you give me a ticker that has a split and dividend(s)? That will help me test the code.

[tchevri]

I was using these lines of code dear @joshuaulrich

require(quantmod)
  # getSymbols('1306.T', from="2000-01-01", src = 'yahooj')
  # getSymbols("6758.T", src="yahooj", from="2013-01-01")
  # getSymbols("9984.T", src = "yahooj")
  # tmp <- getSymbols("9984.T", src = "yahoo", auto.assign = FALSE)
YJ1306.T <- getSymbols("1306.T", from="2000-01-01", src = "yahoo", auto.assign = FALSE)
init.d <- index(YJ1306.T)[1]
etf1306 <- YJ1306.T[, 6]/as.numeric(YJ1306.T[init.d, 6])
names(etf1306) <- "Topix Total Return Index"
plot(etf1306, main = paste0("Time Series Plot of ", names(etf1306)), cex.main=2, cex.axis=1.5, cex.sub=2)

So you can see, for now, I reverted to yahoo. Amusingly enough, same data, but not as much history... Yahoo goes back 2008, YJ goes back 2000 if I recall correctly... Thank you so much! As you know first hand, we can't thank you enough for all you do Joshua. thomas

[tchevri]

Just as an FYI, this is the error message I am getting, I see I did not include it earlier, apologies:

> getSymbols("6758.T", src="yahooj", from="2013-01-01", verbose = T)
downloading  6758.T .....

done.
Error in getSymbols.yahooj(Symbols = "6758.T", env = <environment>, verbose = TRUE,  : 
  Unable to import “6758.T”.
no applicable method for 'xml_find_all' applied to an object of class "NULL"

I am running quantmod 0.4.18, basically my Rstudio, R, and packages, etc are all fully updated on windows 11 (I just "upgraded" to the dev version with devtools::install_github("joshuaulrich/quantmod"), but that did not help this time)

[joshuaulrich]

@tchevri I just pushed a potential fix to the 312-yahooj branch. Can you please test? It doesn't handle splits and dividends any more, because it doesn't look like the table has that data now.

[tchevri]

Sorry for the big noob question @joshuaulrich . How do I test this? I just tried:

devtools::install_github("joshuaulrich/quantmod") Skipping install of 'quantmod' from a github remote, the SHA1 (51395f93) has not changed since last install. Useforce = TRUEto force installation

so sorry for not knowing.

[joshuaulrich]

Try remotes::install_github("joshuaulrich/quantmod@312-yahooj")

[tchevri]

It took a very long time to run (not sure if I am delusional, but it seems much much longer than before!), but most importantly it works without error! Thank you so much @joshuaulrich ! Awesome.

getSymbols('1306.T', from="2000-01-01", src = 'yahooj')  

[1] "1306.T"

[tchevri]

It wasn't working again...I realized it's because i had my vpn on. Here's the reason:

From Wednesday, April 6, 2022, Yahoo! JAPAN will no longer be available from the European Economic Area (EEA) and the United Kingdom. As announced from February 1, 2022 , Yahoo! JAPAN has determined that it is difficult to provide a continuous service usage environment for customers in the European Economic Area (EEA) and the United Kingdom, so the following "2022" Services available after Wednesday, April 6, 2022, except for the services listed in, will no longer be available from EEA and the United Kingdom after Wednesday, April 6, 2022.

If you use the Yahoo! JAPAN service in Japan, you can continue to use all services.

We deeply apologize for any inconvenience caused to our customers.

Not much you can do... I take it.

Here's the code:

getSymbols('1306.T', from="2000-01-01", src = 'yahooj', verbose = TRUE) 

downloading 1306.T .....

Error in getSymbols.yahooj(Symbols = "1306.T", env = , verbose = TRUE, : Unable to import “1306.T”. HTTP error 403. In addition: Warning messages: 1: In for (i in 1:length(args)) { : closing unused connection 6 (https://finance.yahoo.co.jp/quote/1306.T/history?from=20000101&to=20220620&timeFrame=d&page=1)

Solution: For me, simply change vpn...