mrc-ide / rrq

:runner::runner::runner: Lightweight Redis queues
https://mrc-ide.github.io/rrq/
Other
24 stars 4 forks source link

Issue with environment resolution while running in bulk mode #98

Closed richfitz closed 1 year ago

richfitz commented 1 year ago

This is a stripped down version of the bug reported by @cwhittaker1000

with src.R containing:

thing <- R6::R6Class(
  "thing",
  public = list(fn = function(x) x + 1))

f <- function(x) {
  thing$new()$fn(x)
}
id <- ids::random_id(1, 4)
r <- rrq::rrq_controller$new(id)
r$envir(rrq::rrq_envir(sources = c("src.R")))
w1 <- rrq::rrq_worker$new(r$queue_id)

## Works fine:
t1 <- r$enqueue(f(1))
w1$step(TRUE)
r$task_result(t1)

## This works in a blocking worker:
source("src.R")
g1 <- r$lapply(1, f, timeout_task_wait = 0)
w1$step(TRUE)
r$task_result(g1$task_ids)

## Also works fine in a real worker:
w2 <- rrq::rrq_worker_spawn(r)
t2 <- r$enqueue(f(1))
r$task_wait(t2) # 2 as expected

## But this fails in a real worker:
g2 <- r$lapply(1, f, timeout_task_wait = 10)

The error object:

<rrq_task_error>
  from:   `2110a78728ee57f02b634a8498f1845c`(1)
  error:  object 'thing' not found
  queue:  ec732816
  task:   e90ce910790f9964ffbc6ededff04a6c
  status: ERROR
  * To throw this error, use stop() with it
  * This error has a stack trace, use '$trace' to see it

the trace is not very illuminating:


  1. ├─base::tryCatch(...)
  2. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
  3. │   └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
  4. │     └─base (local) doTryCatch(return(expr), name, parentenv, handler)
  5. ├─base::withCallingHandlers(...)
  6. ├─base::eval(expr, envir)
  7. │ └─base::eval(expr, envir)
  8. │   └─global `2110a78728ee57f02b634a8498f1845c`(1)
  9. └─base::.handleSimpleError(...) at src.R:6:2
 10.   └─rrq (local) h(simpleError(msg, call))