Creating widgets/plots from evaluating code (eval + parse or pander::evals)

[seanbirchall]

I'm unable to display widgets that users create by writing code into a shiny app, this app has no issue running as a shiny app, but when I export it as a shinylive app the reactable created by eval-ing code no longer works. I've tried shimming it in R too without much luck, so I'm starting to think it's related to base::system calls somewhere in htmlwidgets/specific widget libraries?

# similar issue with plots
# sorry for the beefy reprex
library(shiny)
library(shinyAce)
library(bslib)
library(reactable)
library(ggplot2)
library(pander)

ui <- bslib::page(

  title = "reprex",

  # 1) this works
  shiny::fluidRow(
    class = "m-0",
    bslib::card(
      height = "500",
      bslib::card_header(
        "No Issue Displaying Pre-Defined Widgets/Plots or nested ones being called from renderUI"
      ),
      bslib::card_body(
        shiny::fluidRow(
          shiny::column(
            width = 6,
            # straight forward
            reactableOutput(
              outputId = "table",
              height = "350"
            )
          ),
          shiny::column(
            width = 6,
            # a little different, nested
            shiny::uiOutput(
              outputId = "plot"
            ) 
          )
        )
      )
    )
  ),

  # 2) this doesn't work
  shiny::fluidRow(
    class = "m-0",
    bslib::card(
      height = "500",
      bslib::card_header(
        "Issue Creating Widgets Via Code, using an eval + parse or pander::evals"
      ),
      bslib::card_body(
        shiny::fluidRow(
          shiny::column(
            width = 6,
            shiny::actionButton(
              inputId = "run",
              label = "Run"
            ),
            shinyAce::aceEditor(
              outputId = "ace",
              height = "350",
              value = "reactable(iris, height=350)",
              readOnly = TRUE
            )
          ),
          shiny::column(
            width = 6,
            shiny::uiOutput(
              outputId = "code_output"
            )
          ) 
        ),
        shiny::column(
          width = 12,
          shiny::h3("What is evals returning?"),
          shiny::verbatimTextOutput(
            outputId = "what_is_my_reactiveValues"
          )
        )
      )
    )
  )

)

server <- function(input, output, session){

  # 1) this works
  output$table <- reactable::renderReactable({
    reactable(iris)
  })

  output$plot <- shiny::renderUI({
    shiny::plotOutput(
      outputId = "plot_ggplot",
      height = "350"
    )
  })

  output$plot_ggplot <- shiny::renderPlot({
    ggplot(data=iris) +
      geom_point(mapping=aes(x=Petal.Length,y=Petal.Width))
  })
  # END

  # 2) this doesn't work
  output$code_output <- shiny::renderUI({
    user_inputs_reactiveValues$code_widget
  })

  output$what_is_my_reactiveValues <- shiny::renderPrint({
    str(user_inputs_reactiveValues$code_widget)
  })

  user_inputs_reactiveValues <- shiny::reactiveValues(
    code_widget = NULL
  )

  shiny::observeEvent(input$run, {
    # eval code input
    code_run <- pander::evals(txt = input$ace, env = .GlobalEnv)

    # assign widget to reactivevalues to be used in renderUI 
    user_inputs_reactiveValues$code_widget <- code_run[[1]][["result"]]

  })
  # END

}

shiny::shinyApp(ui = ui, server = server)

I've tried shimming calls to htmlwidget functions like reactable to then save it as a self-contained html file using htmlwidgets::saveWidget and htmltools::save_html. Instead of creating a reactiveValues object that holds the widget, the widget is just saved to a directory called widget which I point to an render that way. Again this only works in shiny apps, not shinylive apps.

I've also started trying to shim it so that any call to an htmlwidget also surrounds it with relevant output and render functions, so far it's not working, but I think I should know for sure this weekend.

I could also just be going about this wrong and maybe it has something to do with me using reactiveValues when I should be using a reactive or maybe have the code in a render function... I'll test out more with that too.

If it is related to system calls and some other issues I've seen on the webR repo regarding browseURL then maybe the saving as a self-contained html file could work but I'd need to pass the file up as a custom webR channel message?

[seanbirchall]

Wrapping it in a render function does appear to work, so it should be doable for me to write a clunky shim in R to wrap calls to html widgets in their respective render functions.

Replacing input$ace which = reactable(iris, height=350) with a static reactable::renderReactable({reactable(iris, height=350)}) does work.

But still am definitely curious why this doesn't work out the box in the above app from shiny to shinylive so any insights there would be super helpful!

[georgestagg]

I can't speak too authoritatively about evaluating custom htmlwidgets in Shiny, but I can talk a little more generally about working with htmlwidgets under webR. I wanted to write this comment to add a little context.

---

When htmlwidgets are added to a page, they may load additional assets and dependencies like JavaScript or CSS files into the containing web page. These may be loaded from CDN, or from a relative directly installed by the R package providing the widget.

For this to work well, R must be able to serve those additional dependencies to the user in some way. Normally, this is not an issue with R because it can start a httpuv/shiny process to serve the files, or the containing app is running on some remote R server that can provide the files.

In the case of shinylive, htmlwidgets are able to be shown in the app because there is a JavaScript Service Worker running on the page. This plays the role that httpuv would normally play with a native R session, emulating an R-powered web server.

As you may have noticed, the webR REPL application does not run a Service Worker in the page (by default). As such, htmlwidgets do not currently work in that environment because the additional assets are unable to be served for the page to load. This is the same reason browseURL() does not currently work in webR.

There is a potential solution to this problem in that tools such as saveWidget() are able to save a widget as a self-contained HTML file. With this, as you suggest, the HTML can be sent to the main JS thread and added to the page directly. And, since the HTML is self-contained, there is no need to serve any additional dependencies using a JS service worker.

This is a good solution, but with a drawback: saving htmlwidets to a self-contained HTML file requires processing outside R using pandoc. You can see an attempt to do this in the shinylive error log for your example app:

Warning in Sys.which("pandoc") : 'which' was not found on this platform

So, a real and full solution to this requires two additional engineering efforts:

1. Pandoc is compiled for Wasm, so that it can run in the browser without an external server.
2. WebR can launch additional Wasm processes in new JavaScript Worker threads, and block while it waits for the process to complete.

I am currently working on number 1 (https://github.com/georgestagg/pandoc-wasm), but number 2 is not ready yet in webR. Once there's enough in place in webR to launch pandoc via the system() command, saving htmlwidgets as self-contained HTML and displaying that in a page should hopefully "just work".

---

Most of what I say above applies to webR, but not Shinylive, due to some additional infrastructure Shinylive has in place. Shinylive is able to serve a htmlwidget directly using its Service Worker, so my understanding is that in Shinylive things should already work OK. I'll try to see if I can work out why your workaround in the previous comment is required, in case it can be fixed by changes to webR, though I must admit I'm not personally an expert in writing Shiny apps.

[timelyportfolio]

Not sure if is helpful, but I have started exploring other ways of incoporating htmlwidgets in a webR context.

1) Most times htmlwidgets do not need Pandoc to produce self-contained https://www.jsinr.me/2024/01/10/selfcontained-htmlwidgets/. Pandoc is usually only required for a full HTML page (often md, HTML, tagList). This probably works well if iframe is acceptable. 2) Adding htmwidgets dependencies manually https://www.jsinr.me/2023/12/11/htmlwidgets-webr-experiment/, creating the htmlwidget in webR, and then rendering the htmlwidget in JavaScript.

[georgestagg]

https://www.jsinr.me/2024/01/10/selfcontained-htmlwidgets/

Thank you for reminding me of this, it is indeed a very neat demo!

Thinking long term, my personal opinion is that putting in the engineering effort to run a real Pandoc is worth it. While this approach might selectively work in the short term, I'm always very cautious of working on HTML with regex-based appraoches. After all, HTML is not a regular language.

[timelyportfolio]

@georgestagg having Pandoc in browser would be amazing! One note - I used regex only to avoid dependency on xml2/rvest which require the unavailable curl. The crude regex is only somewhat reliable because we get a consistent result from the htmltools::save_html function.

[seanbirchall]

Agree pandoc in the browser would be amazing, glad to hear you're working on it @georgestagg. Thank you for the detailed comment.

@timelyportfolio I'm going to mess around more with the two examples above. I'd like to create a temporary solution at least in the interim. Right now mine is bad, I'm just calling pander::evals twice, first pass evals all code in the editor, second pass wraps any widgets with their respective render functions. I could fork pander and maybe try adding this functionality as a regex so the two passes are not needed, but it's probably not worth the effort.

[seanbirchall]

Some weirdness happens with this but this snippet is working for me to display reactable and ggplot visuals in shinylive + using pander::evals to run code.

viewer <- tryCatch(
    render_funs <- lapply(seq_along(run), function(v){
        if(any(grepl("reactable", run[[v]][["src"]], fixed = TRUE))){
            rerun_code <- paste0("reactable::renderReactable({", run[[v]][["src"]], "})")
            run_code <- evals(txt = rerun_code, env = .GlobalEnv)
            run_code[[1]][["result"]]
        }else if(any(grepl("ggplot", run[[v]][["src"]], fixed = TRUE))){
            rerun_code <- paste0("shiny::renderPlot({", run[[v]][["src"]], "})")
            run_code <- evals(txt = rerun_code, env = .GlobalEnv)
            run_code[[1]][["result"]]
        }
    }),
    error = function(e){
        NULL
    })

It's hard to debug and you need to delete whatever directory you created when using shinylive::export("app", "out_dir_delete") if the app fails to run from httpuv::runStaticServer and also I believe need to delete any plot directory that ggplot creates. It should make shinylive Robj construction for this JS object is not yet supported go away.

EDIT: always remove plot directory it crashes the app.