I am looping over a function with purrr::map() and displaying progress steps so the user knows how long the process should take. To speed things up, I decided to parallelize this loop with future and furrr. However, the progress updates are delayed until after the worker has completed, which means that the progress is only signalled when the current loop has already completed, which is not very useful.
Possibly related issue #93
Expected behaviour
I expected progress steps to appear as normal (i.e. without future). The only difference is that, since there are multiple workers, multiple progress steps will be active at the same time.
Actual behaviour
Progress steps were delayed until the worker was finished, thereby only signalling the msg_done condition.
Minimal reproducible example
# Create some function that displays progress steps
fun <- function(x) {
cli::cli_progress_step("Adding {x}...",
msg_done = "Successfully added {x}!")
Sys.sleep(0.5)
}
# This works as it should
purrr::walk(1:3, fun)
#> ℹ Adding 1...✔ Successfully added 1! [565ms]
#> ℹ Adding 2...✔ Successfully added 2! [537ms]
#> ℹ Adding 3...✔ Successfully added 3! [525ms]
# But this doesn't; The output is delayed until the function is already finished
# Thus, only the end is signalled, not the start.
future::plan("multisession")
furrr::future_walk(1:3, fun)
#> ℹ Adding 1...
#> ✔ Successfully added 1! [598ms]
#>
#> ℹ Adding 2...
#> ✔ Successfully added 2! [574ms]
#>
#> ℹ Adding 3...
#> ✔ Successfully added 3! [591ms]
future::plan("sequential")
Describe the issue
I am looping over a function with
purrr::map()
and displaying progress steps so the user knows how long the process should take. To speed things up, I decided to parallelize this loop withfuture
andfurrr
. However, the progress updates are delayed until after the worker has completed, which means that the progress is only signalled when the current loop has already completed, which is not very useful.Possibly related issue #93
Expected behaviour
I expected progress steps to appear as normal (i.e. without
future
). The only difference is that, since there are multiple workers, multiple progress steps will be active at the same time.Actual behaviour
Progress steps were delayed until the worker was finished, thereby only signalling the
msg_done
condition.Minimal reproducible example
Created on 2023-01-30 with reprex v2.0.2
Interestingly, the example above also doesn't work when
future::plan("multisession")
is not used (i.e. sequential processing).