Development version of xts breaks twsInstrument::get_quote()

[SamoPP]

Description

twsInstrument::get_quote() fails using latest github version of xts

> library(twsInstrument)
Loading required package: IBrokers
Loading required package: xts
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

IBrokers version 0.9-10.  Implementing API Version 9.64

IBrokers comes with NO WARRANTY.  Not intended for production use!

See ?IBrokers for details.
Loading required package: FinancialInstrument
Loading required package: quantmod
Loading required package: TTR
Registered S3 method overwritten by 'quantmod':
  method            from
  as.zoo.data.frame zoo 
Version 0.4-0 included new data defaults. See ?getSymbols.
Learn from a quantmod author: https://www.datacamp.com/courses/importing-and-managing-financial-data-in-r
> 
> quotes <- get_quote("SPY", eWrapper=eWrapper.data.BID_ASK_OHLCV)
Connected with clientId 100.
Checking to see if other 'type's have a pre-defined currency.
Request complete: SPY STK USD.
Disconnected.
2 -1 2104 Market data farm connection is OK:usfarm.nj 
2 -1 2104 Market data farm connection is OK:usfuture 
2 -1 2104 Market data farm connection is OK:usopt 
2 -1 2104 Market data farm connection is OK:usfarm 
2 -1 2106 HMDS data farm connection is OK:ushmds.nj 
2 -1 2106 HMDS data farm connection is OK:ushmds 
2 1 10090 Part of requested market data is not subscribed. Subscription-independent ticks are still active.SPY ARCA/Auction:225 
Error in index.xts(x) : unsupported ‘tclass’ indexing type: 
In addition: Warning messages:
1: In tclass.xts(x) : index does not have a ‘tclass’ attribute
2: In tclass.xts(x) :

 Error in index.xts(x) : unsupported ‘tclass’ indexing type: 
14.
stop(paste("unsupported", sQuote("tclass"), "indexing type:", 
    value[[1]])) at index.R#42
13.
index.xts(x) at index.R#3
12.
index(x) at data.frame.R#52
11.
as.data.frame.xts(X[[i]], ...) 
10.
FUN(X[[i]], ...) 
9.
lapply(eWrapper$.Data$data, as.data.frame) at get_quote.R#114
8.
do.call(rbind, lapply(eWrapper$.Data$data, as.data.frame)) at get_quote.R#114
7.
CALLBACK(conn, eWrapper = eventWrapper, timestamp = timeStamp, 
    file = file, playback = playback, timeout = NULL, ...) at reqMktData.R#207
6.
reqMktData(tws, contracts, eventWrapper = ew, CALLBACK = snapShot) at get_quote.R#154
5.
tryCatchList(expr, classes, parentenv, handlers) 
4.
tryCatch({
    if (suppressWarnings(isConnected(tws)) && isTRUE(verbose)) 
        cat(paste("Connected with clientId ", tws$clientId, ".\n Requesting ", 
            Symbols, "\n", sep = "")) ... at get_quote.R#141
3.
get_quote.IB(Symbols = "SPY", eWrapper = function (n) 
{
    eW <- eWrapper(NULL)
    eW$assign.Data("data", rep(list(structure(.xts(matrix(rep(NA_real_,  ... 
2.
do.call(paste("get_quote", src, sep = "."), list(Symbols = Symbols, 
    ...)) at get_quote.R#9
1.
get_quote("SPY", eWrapper = eWrapper.data.BID_ASK_OHLCV) 

Expected behavior

Backward compatibility expected.

Minimal, reproducible example

library(twsInstrument)

quotes <- get_quote("SPY", eWrapper=eWrapper.data.BID_ASK_OHLCV)

Session Info

R version 3.6.0 (2019-04-26)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] twsInstrument_1.4-10      FinancialInstrument_1.3.1 quantmod_0.4-14           TTR_0.23-4                IBrokers_0.9-12          
[6] xts_0.12-1                zoo_1.8-5                

loaded via a namespace (and not attached):
[1] compiler_3.6.0  tools_3.6.0     curl_3.3        grid_3.6.0      packrat_0.5.0   lattice_0.20-38

[joshuaulrich]

Thanks for the report! I will investigate.

[joshuaulrich]

Hi Samo. I'm trying to replicate this using the demo account, but I'm not able to. Can you try replicating this issue with the demo account? That would confirm whether my inability to replicate is due to the demo account, or something with my local setup.

[SamoPP]

Hi Joshua. Have you tried during regular trading hours (RTH) for SPY (form 09:30 to 16:00 EST)? Most of the time, if I run this outside RTH there is no data being sent from IB so therefore no error can be triggered/generated/replicated...

[joshuaulrich]

It's hard for me to test during regular trading hours, because I'm almost always at work and I don't have access to TWS. I tried to replicate this issue using ES futures, since that market is open before I go to work. I also tried it using a FX ticker. I assume this error occurs for instruments other than SPY?

[joshuaulrich]

I still wasn't able to reproduce the error using TWS, but the source of the problem is eWrapper.data.BID_ASK_OHLCV.R:L30, which is:

attr(data[[id]],"index") <- as.numeric(Sys.time())

That line assigns to the index directly, and does not include the required tclass and tzone attributes on the index. I could patch xts to work around that error, but the fix should be done in the twsInstrument package. I'll reach out to @gsee to see how he would like to handle this.

My proposed patch would be to change the attr(data[[id]], "index") <- to .index(data[[id]]) <-, but even that wouldn't fix the error. .index<- would need to ensure that the existing index attributes are added to the new numeric index value.

[joshuaulrich]

The error is thrown by this switch() statement, because value[[1]] (the first element of the class attribute) is "".

There's a check to see if is.null(value[[1]]) is TRUE. value is the result of tclass(x), which will never be NULL because tclass() currently returns "" if it can't find an index class attribute. The relevant lines of code start on tclass.R:38. I would have to investigate what might break if tclass() returned NULL instead of "" in these cases.

[joshuaulrich]

I think the reason I made tclass() return "" instead of NULL is because Sys.timezone() returns "" when it can't determine the operating system's TZ.