"Reactive context was created in one process and accessed from another". The same code behaves differently in docker and outside

[alexmosc]

I cannot adapt my app that has been running well outside docker for the docker. I traced the difference in library versions, but I cannot figure out an exact cause of error in my code.

(Read the caveats and limitations of promises for Shiny, but still staying confused.)

Outside docker:

centos 8

promises_1.0.1 future_1.14.0 shiny_1.3.2

Inside docker:

FROM rocker/shiny-verse
ubuntu 20

promises_1.2.0.1 future_1.21.0 shiny_1.6.0

Sample app:

## load libs ---------------------------------------

library(shiny)
library(DT)
library(highcharter)
library(data.table)
library(magrittr)
library(lubridate)
library(future)
library(promises)
library(xlsx)

## UI processing -----------------------------------

ui <- navbarPage(
  title = "DIGITASS"

  , tabPanel(
    title = "TEST in DOCKER"
    , sidebarLayout(
      sidebarPanel(
        h1("Управление")
        , br()
        , dateRangeInput(
          inputId = "dates_topics"
          , label = h3("Даты (вчера и ранее): ")
          , start = Sys.Date() - 31
          , end = Sys.Date() - 2
          , min = "2019-06-08"
          , max = Sys.Date() - 2
          , format = 'yyyy-mm-dd'
        )
        , br()
        , numericInput(
          inputId = 'topics_plot', label = 'Вывести график для топовых сюжетов (максимум 12):'
          , value = 4
          , max = 12
        )
        , br()
        , actionButton(
          "run_topics"
          , label = "RUNIT!"
          , style="color: #fff; background-color: #337ab7; border-color: #2e6da4"
        )
        , br()
        , br()
        , downloadLink('downloadDataWeb', 'Скачать книгу excel с данными')

      )
      , mainPanel(
        conditionalPanel(condition="$('html').hasClass('shiny-busy')",
                         tags$div("Пожалуйста, ожидайте несколько секунд...", id = "workmessage"))
        , textOutput("topics_alert")
        , br()
        , tabsetPanel(
          type = "tabs"
          , tabPanel('Таблица'
                     , br()
                     , DTOutput('topics_tbl')
                     , br()
          )

          , tabPanel('График'
                     , br()
                     #, plotOutput('the_topics_plot')
                     , highchartOutput("the_topics_plot", height = "750px")
                     , br()
          )
        )
      )
    )
  )
  , position = "static-top"
)

## Server processing ----------------------------------------------------------

server <- function(input, output, session)
{

  ## Tass.ru Topics ----------------

  dt_topics <- eventReactive(
    input$run_topics,
    {

      plan(multicore)

      setwd('/app/stories')

      min_input <- input$dates_topics[1]

      max_input <- input$dates_topics[2]

      ## update date range in ui

      updateDateRangeInput(
        session = session
        , inputId = "dates_topics"
        , start = min_input
        , end = max_input
        , min = "2019-06-08"
        , max = Sys.Date() - 2
      )

      date_seq <- seq.Date(as.Date(min_input), as.Date(max_input), 1)

      year_month <- unique(
        paste0(
          lubridate::year(date_seq)
          , '_'
          , lubridate::month(date_seq)
        )
      )

      saved_metric_dat <- list.files(path = ".", pattern = "metric_dat.csv")

      saved_metric_dat <- saved_metric_dat[grepl(paste(year_month, collapse = '|'), saved_metric_dat)]

      plot_input <- ifelse(
        as.integer(input$topics_plot[1]) > 12
        , 12
        , as.integer(input$topics_plot[1])
      )

      dat_func <- 
        function(
        min_inp
        , max_inp
        , saved_metric_d
        , plot_inp
        )
      {

        start_time <- Sys.time()

          Sys.sleep(5)

          output_tbl <- DT::datatable(data.table(x = 1:10, y = 1:10))

          plo1 <- hchart(data.table(x = 1:10, y = 1:10), "line", hcaes(x = x, y = y))

          dat_full <- data.table(x = 1:10, y = 1:10)

        list(
          output_tbl
          , plo1
          , paste0('Время обработки: ', round(Sys.time() - start_time), ' сек.')
          , dat_full
        )
      }

      ## launch future to return

      future({
        dat_func(
          min_inp = min_input
            , max_inp = max_input
            , saved_metric_d = saved_metric_dat
            , plot_inp = plot_input
            )
            })

    }
)

  ## Output

  output$topics_tbl <- 
    renderDT({
        dt_topics() %...>% `[[`(1)
      })

  output$the_topics_plot <- 
    renderHighchart({
      dt_topics() %...>% `[[`(2)
    })

  output$topics_alert <- 
    renderText({
      dt_topics() %...>% `[[`(3)
      })

  output$downloadDataWeb <-
    downloadHandler(

      filename = "tass_ru_web_topics.xlsx",

      content = function(file)
      {

        dat_full <- value(dt_topics())[[4]]

        write.xlsx2(x = dat_full, file = file, sheetName = 'web_topics', row.names = FALSE)

      }

    )

}

## Run

shinyApp(ui = ui, server = server)

Error log:

Listening on http://127.0.0.1:40519 Warning: Error in : Reactive context was created in one process and accessed from another. [No stack trace available]

[jcheng5]

I don't think this is the problem, but, I wouldn't put plan(multicore) nor setwd('/app/stories') inside of an eventReactive; the plan(multicore) should probably go at the top of the file right after all the library() calls, and ideally you should not do the setwd() call at all, as it's customary for Shiny apps to have their working directory set to the app directory while running.

Can you try options(shiny.fullstacktrace=TRUE) at the top of the file? Maybe that will show a stack trace.

You might also play with the contents of dat_func to see if you can figure out what logic in there is causing this (it's almost certainly related to an object traversing between future parent/child processes).

[alexmosc]

I don't think this is the problem, but, I wouldn't put plan(multicore) nor setwd('/app/stories') inside of an eventReactive; the plan(multicore) should probably go at the top of the file right after all the library() calls, and ideally you should not do the setwd() call at all, as it's customary for Shiny apps to have their working directory set to the app directory while running.

Can you try options(shiny.fullstacktrace=TRUE) at the top of the file? Maybe that will show a stack trace.

You might also play with the contents of dat_func to see if you can figure out what logic in there is causing this (it's almost certainly related to an object traversing between future parent/child processes).

First of all, thank you for your comments about using Shiny.

I removed setwd() and plan(...) outside of the reactive and will do so in other apps as well, thanks.

That did not help, though. If you took a look at my dat_func(), it is so simple, not even using the function's params for simplicity, so apparently there is no chaos with the enviroments.

Did this: options(shiny.fullstacktrace=TRUE), thanks again.

Listening on http://127.0.0.1:41417 Warning: Error in : Reactive context was created in one process and accessed from another. 37: domain$onError 36: promiseDomain$onError 35: 34: stop 33: signalConditions 32: value.Future 31: future::value 30: withCallingHandlers 29: doTryCatch 28: tryCatchOne 27: tryCatchList 26: doTryCatch 25: tryCatchOne 24: tryCatchList 23: base::tryCatch 22: tryCatch 21: 20: evalq 19: evalq 18: doTryCatch 17: tryCatchOne 16: tryCatchList 15: doTryCatch 14: tryCatchOne 13: tryCatchList 12: tryCatch 11: execCallbacks 10: run_now 9: service 8: serviceApp 7: ..stacktracefloor.. 6: withCallingHandlers 5: domain$wrapSync 4: promises::with_promise_domain 3: captureStackTraces 2: ..stacktraceoff.. 1: runApp Warning: Error in : Reactive context was created in one process and accessed from another. 37: domain$onError 36: promiseDomain$onError 35: 34: stop 33: signalConditions 32: value.Future 31: future::value 30: withCallingHandlers 29: doTryCatch 28: tryCatchOne 27: tryCatchList 26: doTryCatch 25: tryCatchOne 24: tryCatchList 23: base::tryCatch 22: tryCatch 21: 20: evalq 19: evalq 18: doTryCatch 17: tryCatchOne 16: tryCatchList 15: doTryCatch 14: tryCatchOne 13: tryCatchList 12: tryCatch 11: execCallbacks 10: run_now 9: service 8: serviceApp 7: ..stacktracefloor.. 6: withCallingHandlers 5: domain$wrapSync 4: promises::with_promise_domain 3: captureStackTraces 2: ..stacktraceoff.. 1: runApp

From earlier call: 176: base$wrapOnRejected 175: base$wrapOnRejected 174: base$wrapOnRejected 173: base$wrapOnRejected 172: base$wrapOnRejected 171: base$wrapOnRejected 170: base$wrapOnRejected 169: base$wrapOnRejected 168: base$wrapOnRejected 167: base$wrapOnRejected 166: base$wrapOnRejected 165: base$wrapOnRejected 164: base$wrapOnRejected 163: base$wrapOnRejected 162: domain$wrapOnRejected 161: promiseDomain$onThen

It seems all is very confusing, BUT:

I did an even simpler reproducible app without libraries DTand highcharter; using instead ggplot2and data.table (and corresponding render... functions), and it worked as a charm.

And I noticed that when I install DTand highcharter during an image building (I had to install them as they are out of the docker image scope) I get several dependency installation failures, and everything is fine otherwise (ggplot2and data.table are part of the image).

I will take time to dig into what exactly fails.

Noting, though, that I do not have the logs about any library failing to start after calling library(...). Hm.

Update1: app fails when using DTlibrary with ggplot2, no errors during installation.

Update2: app works when not using DT library, but with the highcharter and some errors (hmm) during installation.

So, any combination without DT: data.table & ggplot; data.table & highcharter works, and any combination wit the DT does not.

What is your opinion, how can I deal with this problem?