r-lib / cli

Tools for making beautiful & useful command line interfaces
https://cli.r-lib.org/
Other
648 stars 70 forks source link

Using CLI progress with Future #575

Closed koenniem closed 1 year ago

koenniem commented 1 year ago

Describe the issue

I am looping over a function with purrr::map() and displaying progress steps so the user knows how long the process should take. To speed things up, I decided to parallelize this loop with future and furrr. However, the progress updates are delayed until after the worker has completed, which means that the progress is only signalled when the current loop has already completed, which is not very useful.

Possibly related issue #93

Expected behaviour

I expected progress steps to appear as normal (i.e. without future). The only difference is that, since there are multiple workers, multiple progress steps will be active at the same time.

Actual behaviour

Progress steps were delayed until the worker was finished, thereby only signalling the msg_done condition.

Minimal reproducible example

# Create some function that displays progress steps
fun <- function(x) {
  cli::cli_progress_step("Adding {x}...",
                         msg_done = "Successfully added {x}!")
  Sys.sleep(0.5)
}

# This works as it should
purrr::walk(1:3, fun)
#> ℹ Adding 1...✔ Successfully added 1! [565ms]
#> ℹ Adding 2...✔ Successfully added 2! [537ms]
#> ℹ Adding 3...✔ Successfully added 3! [525ms]

# But this doesn't; The output is delayed until the function is already finished
# Thus, only the end is signalled, not the start.
future::plan("multisession")
furrr::future_walk(1:3, fun)
#> ℹ Adding 1...
#> ✔ Successfully added 1! [598ms]
#> 
#> ℹ Adding 2...
#> ✔ Successfully added 2! [574ms]
#> 
#> ℹ Adding 3...
#> ✔ Successfully added 3! [591ms]
future::plan("sequential")

Created on 2023-01-30 with reprex v2.0.2

Interestingly, the example above also doesn't work when future::plan("multisession") is not used (i.e. sequential processing).

gaborcsardi commented 1 year ago

This is not something that can be fixed in cli, as cli only runs in the subprocess and this would require support in both processes.

It could be potentially fixed in future or furrr, but I am not sure.