Issue with reqIds/ reqMktData v10.2 of IBrokers

[PradeepNanduri]

Description

I wish to report a problem w.r.t latest version of IBrokers (10.2). There seems to be a conflict processing messages when reqMktData and reqIds are run in two parallel R sessions. I've got two R scripts that run in parallel, Script 1 collects data using reqMktData whereas Script 2 places orders using statistics derived from Script 1. With the latest version of IBrokers, only one the scripts works ok and the other fails with following message.

Error in if (curMsg == .twsIncomingMSG$TICK_PRICE) {msg : argument is of length zero

Minimal, reproducible example

You can reproduce the problem via below mentioned scripts running in parallel R sessions. To confirm, this problem doesn't occur in older version of IBrokers (9.12)

### Script_1 (reqMktData)

library(data.table)
library(IBrokers)

memory.limit(size = 100000)
tws <- twsConnect(port = 4002)

snapShot= function (twsCon, eWrapper, timestamp, file, playback = 1, ...)
{
  if (missing(eWrapper))
    eWrapper <- eWrapper()

  names(eWrapper$.Data$data) <- eWrapper$.Data$symbols
  con <- twsCon[[1]]
  if (inherits(twsCon, "twsPlayback")) {
    sys.time <- NULL
    while (TRUE) {
      if (!is.null(timestamp)) {
        last.time <- sys.time
        sys.time <- as.POSIXct(strptime(paste(readBin(con,
                                                      character(), 2), collapse = " "), timestamp))
        if (!is.null(last.time)) {
          Sys.sleep((sys.time - last.time) * playback)
        }
        curMsg <- .Internal(readBin(con, "character",
                                    1L, NA_integer_, TRUE, FALSE))
        if (length(curMsg) < 1)
          next
        processMsg(curMsg, con, eWrapper, format(sys.time,
                                                 timestamp), file, ...)
      }
      else {
        curMsg <- readBin(con, character(), 1)
        if (length(curMsg) < 1)
          next
        processMsg(curMsg, con, eWrapper, timestamp,
                   file, ...)
        if (curMsg == .twsIncomingMSG$REAL_TIME_BARS)
          Sys.sleep(5 * playback)
      }
    }
  }
  else {
    while (TRUE) {
      socketSelect(list(con), FALSE, NULL)
      curMsg <- .Internal(readBin(con, "character", 1L,
                                  NA_integer_, TRUE, FALSE))
      if (!is.null(timestamp)) {
        processMsg(curMsg, con, eWrapper, format(Sys.time(),
                                                 timestamp), file, ...)
      }
      else {
        processMsg(curMsg, con, eWrapper, timestamp,
                   file, ...)
      }
      if (!any(sapply(eWrapper$.Data$data, is.na)))
        return(do.call(rbind, lapply(eWrapper$.Data$data,
                                     as.data.frame)))
    }
  }
}

sec = twsFuture(symbol = 'NQ',exch = 'CME', expiry = '20230915', currency = 'USD',multiplier = '20')
tm = Sys.time()
Dt = Sys.Date()
ed = paste(Dt,"21:01:00", "BST", sep = " ")

while(tm < ed) {
  dwn_data = reqMktData(tws,CALLBACK = snapShot,eventWrapper = eWrapper.data(length(1)),Contract = sec);
  print(dwn_data)
  Sys.sleep(3)
}

### Script_2 (reqIds) - 

library(data.table)
library(IBrokers)

options(digits.secs = 2)
tws <- twsConnect(port = 4002)
reqIds(tws)

[SamoPP]

Hi. This is quite critical... Any idea how to solve this? Thanks.

[joshuaulrich]

I looked into this, but I'm not very familiar with the TWS API or this code... @jaryan is the expert. That said, I ran Wireshark on the connection and found that the first write to the TWS API connection in Script 2 causes TWS to reset the connection (i.e. it sends a TCP RST response and closes the connection). That's what causes the error later.

?twsConnect() says that clientId must be unique for each client and it will throw an error if you try to create another connection to the API with a clientId that's already in use. But it doesn't throw an error like documented. Script 2 doesn't error if you use tws <- twsConnect(clientId = 2, port = 4002). Please try that. I'm not sure it's actually a solution, but it prevents the error.

[PradeepNanduri]

@joshuaulrich thanks for taking a look at the issue.

I've tried implementing your suggestion but it's not working. I've set 'ClientId' = 2 while defining 'tws' object in my script 2 but it's failing to generate an order ID with function 'reqIds'. The error message is still the same. And script 1 is successfully running collecting market data where tws <- twsConnect(port = 7496)

Want to highlight a point that it works all ok with previous version of IBrokers package.

[joshuaulrich]

I'm not confident I can help more than I have. I can't replicate the issue when I use the steps in my previous comment.

A few questions that might help find the source of the issue:

• When did you upgrade IBrokers? I assume it was right before you created this issue. What was the old version IBrokers and what is your current version?
• Did you upgrade R around this time? What was the old version and what's your current version?
• Was there a TWS upgrade recently?
• Does it work if you downgrade to the old version of IBrokers you were using?

[PradeepNanduri]

@joshuaulrich thanks again! This issue is NOT impacting my work as i moved back to V 0.9.12. So I would wait to see if anyone else reports a similar issue. The problem occurs with i upgrade to V 0.10.2.

As of now i am using RStudio/2023.06.0+421 with R 4.3.1. I've tested with previous versions of R and R Studio but the problem persists with V 0.10.2 of IB. And my TWS is the latest one available.