wlandau / crew

A distributed worker launcher
https://wlandau.github.io/crew/
Other
129 stars 4 forks source link

Promises #149

Closed wlandau closed 9 months ago

wlandau commented 9 months ago

Prework

Related GitHub issues and pull requests

Summary

This PR adds a new promise() method which allows controllers and controller groups to create promises. A full tutorial is in the new promises.Rmd vignette. Thanks @jcheng5 for the idea.

I am happy with how this initial implementation came together, so I will merge when the automated checks succeed. But there are a couple follow-ups:

  1. The crew promise uses later::later() for local asynchronicity because right now it is the only convenient option I know about. The loop should be efficient because it pretty much just polls an NNG condition variable, but a non-polling approach would be more responsive and kinder to the CPU. Maybe {nanonext} and {promises} can work together on this somehow.
  2. I would have liked to include Shiny in the new promises.Rmd vignette, but I am having trouble getting an app with promises to work. The reprex below shows what I mean. The app has button to click to run a 2-second task in a future promise, and it has a clock that updates every tenth of a second. I expect the clock to keep running no matter what is going on inside the promise, but whenever I click the button, the clock pauses until the task completes. Maybe this is an obvious failure mode and I am missing something about how promises work in Shiny.
library(promises)
library(shiny)

ui <- fluidPage(
  actionButton("task", "Submit a task"),
  textOutput("status"),
  textOutput("result")
)

server <- function(input, output, session) {
  # The workers do actually launch. I checked with {htop}.
  future::plan(future::multisession, workers = 4)

  # Submit a task with a button and create a promise for its value.
  reactive_promise <- eventReactive(
    eventExpr = input$task,
    valueExpr = {
      # print(future::plan()) # shows "multisession:..." as expected
      future_promise(
        expr = {
          Sys.sleep(2)
          runif(1)
        },
        seed = sample.int(n = 1e9, size = 1)
      )
    }
  )

  # Show the status of the computation
  output$status <- renderText({
    invalidateLater(millis = 100)
    sprintf("Current time: %s", Sys.time())
  })

  # Display the result as a reactive value.
  output$result <- renderText({
    req(reactive_promise())
    reactive_promise() %...>%
       paste(collapse = " ")
  })
}

shinyApp(ui = ui, server = server)\
codecov-commenter commented 9 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (fc9a564) 100.00% compared to head (54687b8) 100.00%.

:exclamation: Current head 54687b8 differs from pull request most recent head e288a26. Consider uploading reports for the commit e288a26 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #149 +/- ## ========================================= Coverage 100.00% 100.00% ========================================= Files 28 28 Lines 1958 1963 +5 ========================================= + Hits 1958 1963 +5 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

wlandau commented 9 months ago

I am having trouble getting an app with promises to work.

After searching more, I think the cause of (2) may be that promises unblock sessions from other sessions, but they cannot (yet) make tasks asynchronous within the same session. I would probably need to open multiple browser windows for the same app to see any asynchronicity.

wlandau commented 9 months ago

Update: crew 0.9.0 is now on CRAN, and it supports the {promises} integration in https://wlandau.github.io/crew/articles/promises.html.