is there a way to "yield"?

[r2evans]

Related to #131 (as an alternative to recursive latering itself) it might be good for a task to yield() control to the main interpreter so that long-running bg-task doesn't interpreter time. I'm guessing this jumps into "reentrancy", so ... perhaps too difficult. Thanks.

[r2evans]

I'm thinking that workaround (if anybody else is looking for this functionality) is for two steps:

• design the function with baked-in steps:

myfunc <- function(step = 1L, loop = later::current_loop(), delay = 1) {
  if (step == 1L) {
  } else if (step == 2L) {
  } else if (step == 3L) {
  } else {
    stop("oops")
  }
  step <- step + 1L
  if (step < total_number_of_steps)
    later(function() myfunc(step, loop = loop, delay = delay), delay = delay, loop = loop)
}

This is definitely a workaround, and not something I would expect formalized or integrated into later. One problem with this is that it takes a lot of deliberate effort in the function to break the task up, whereas with a yield() function it might be more easily incorporated into just about anything with well-placed if (someiter %% 100 == 0) yield() or if (thisruntime() > 3) yield() (3 seconds ... just a thought).

[jcheng5]

A proper yield isn't currently possible, no. An alternative syntax to what you're doing here is to use the promises library (which uses later under the hood):

library(promises)

promise_resolve(TRUE) %...>% {
  # Step 1
  1 + 1
} %...>% {
  # Step 2
  . + 1
} %...>% {
  # Step 3
  print(.)
}

Still much less nice than a proper yield, and has an additional disadvantage versus your approach in that the delay is hardcoded to 0.

AFAIK, most other languages that added yield/generators after-the-fact did it with compile-time source transformations, into basically the form you have here (JS example). @lionel- has experimented with doing this for R, but I don't remember how far he got.

[jcheng5]

Ah, here it is. https://github.com/lionel-/flowery

[jcheng5]

It turns out @lionel- got pretty far :smile: This seems to work (implementation at top, example at bottom):

library(flowery)

run_with_yield <- function(gen, delay = 0.1, loop = later::current_loop()) {
  success <- NULL
  failure <- NULL

  p <- promise(function(resolve, reject) {
    success <<- resolve
    failure <<- reject
  })

  run <- function() {
    tryCatch(
      {
        value <- gen()
        if (is.null(value)) {
          value <- delay
        }
        if (is_done(gen)) {
          success(invisible())
        } else {
          later::later(run, delay = value, loop = loop)
        }
      },
      error = failure
    )
  }
  run()

  p
}

# Example

g <- generator({
  message("Step 1")
  yield()
  message("Step 2 - long delay")
  yield(2) # yield for 2 seconds
  message("Step 3 - Done!")
})

run_with_yield(g, delay = 0.1) %...>% {
  message("Operation complete!")
}

[r2evans]

Wow ... I was fully expecting your first comment ("isn't currently possible") to be the first and final answer, very interesting that @lionel- had already looked into it (and I've seen utility for something like flowery's generators over the years, bummed this package hasn't had more visibility).

It seems then that in order to provide yield()-like functionality to later, then later would have to be generator-aware. This seems likely not on the immediate roadmap. Thanks for the info, @jcheng5.

[jcheng5]

Is this close enough? It's not built-in to later (and it's hard to imagine later taking a direct dependency on flowery anytime soon) but it's a drop-in replacement for later::later--the only thing it adds is yield() support.

later_yield <- function(func, delay = 0, loop = later::current_loop()) {
  f <- rlang::as_function(func)
  gen <- eval(substitute(flowery::generator(expr), list(expr = body(f))),
    envir = environment(f))

  run <- function() {
    value <- gen()
    if (is.null(value) || !is.numeric(value) || length(value) != 1) {
      # If yield() is called without a value, yield for 10 milliseconds
      value <- 0.01
    }
    if (!flowery::is_done(gen)) {
      canceller <<- later::later(run, delay = value, loop = loop)
    }
  }
  canceller <- later::later(run, delay = delay, loop = loop)

  invisible(function() {
    canceller()
  })
}

# Example
later_yield(~{
  message("Task begin")

  message("Short yield")
  yield()

  message("Long (2 second) yield")
  yield(2)

  message("Done!")
}, delay = 2)
message("Task should start in 2 seconds...")

[r2evans]

That is definitely addressing the initial FR, thank you Joe! I am confident I wouldn't have found flowery (and the name doesn't really help me with its discovery), but this definitely gives the effect intended. I'm getting several deprecation messages (node, list_along, new_language, and set_attrs, so it seems that flowery is trailing a little in the rlang dependency.

Do you (personally) see utility in yield in general? I agree that depending on flowery just for this seems like a bit of effort. Just being able to do a proper yield seems so elegant:

for (i in seq_len(gazillion)) {
  if (i %% 1000 == 0) yield(3)
  # do something
}

I haven't looked enough at flowery to know how much of it is "portable" enough to be adapted to this effort.

[lionel-]

@r2evans I've just refreshed flowery to fix these deprecation messages.

[lionel-]

@r2evans https://github.com/r-lib/coro now has full support for promises-based async/await functions, including error handling. Any testing would be welcome!

[r2evans]

coro implements what I was suggesting, thanks.