Inner progress bars displayed after outer progress finishes

[andrewGhazi]

I'm working on a package that has two potentially long running functions, both of which I would like to set up to take advantage of progressr. The second function repeatedly calls the first. In the following mock example, calling f2() correctly shows progress for the outer map(), but after completing it then displays a single "inner" progress bar. I tested this on both Windows and Linux.

I assume it's capturing all of the inner progress bars then showing them all simultaneously at the end once the outer map is complete? The behavior I was expecting was the outer progress bar to show, but not the inner bars.

I know that nested progress notifications aren't supported, but I just want the inner progress bars to turn off when called from f2() (yet still be available when calling f1() directly). Is it possible from the developer side to tell a progressor to not show if there are higher level progressors operating? Or is the solution just to give another argument to f1() that can turn off the progressor?

Apologies if I missed any existing issues on this question, I couldn't find any.

library(purrr)
library(progressr)
handlers(global = TRUE)

f1 = function(x = 5) {
  p = progressr::progressor(steps = x)

  map(1:x, ~{Sys.sleep(.2)
             p()
             .x})
}

f2 = function(x = 10) {
  p = progressr::progressor(steps = x)
  map(1:x, ~{p()
             f1()})
}

f1()
f2()

[HenrikBengtsson]

Thanks for reporting. Here's a minimal base-R example that illustrates the problem:

library(progressr)
handlers(global = TRUE)

handlers(handler_progress(clear = FALSE))

f_inner <- function(n = 4) {
  p <- progressr::progressor(n)
  for (kk in seq_len(n)) {
    Sys.sleep(0.1)
    p(sprintf("inner #%d", kk))
  }
}

f_outer <- function(n = 6) {
  p <- progressr::progressor(n)
  for (kk in seq_len(n)) {
    p(sprintf("outer #%d", kk))
    f_inner()
  }
}

f_outer()

which will output:

| [=========================================] 100% outer #6
- [=========================================] 100% inner #4

Troubleshooting

Mostly for my own troubleshooting, the progress conditions signalled and captured are:

handlers("debug")
...
[11:00:19.935] (0.000s => +0.001s) initiate: 0/6 (+0) '' {clear=TRUE, enabled=TRUE, status=}
[11:00:19.936] (0.001s => +0.001s) update: 1/6 (+1) 'outer #1' {clear=TRUE, enabled=TRUE, status=}
[11:00:20.349] (0.414s => +0.001s) update: 2/6 (+1) 'outer #2' {clear=TRUE, enabled=TRUE, status=}
[11:00:20.756] (0.821s => +0.001s) update: 3/6 (+1) 'outer #3' {clear=TRUE, enabled=TRUE, status=}
[11:00:21.162] (1.227s => +0.001s) update: 4/6 (+1) 'outer #4' {clear=TRUE, enabled=TRUE, status=}
[11:00:21.571] (1.637s => +0.001s) update: 5/6 (+1) 'outer #5' {clear=TRUE, enabled=TRUE, status=}
[11:00:21.979] (2.044s => +0.001s) update: 6/6 (+1) 'outer #6' {clear=TRUE, enabled=TRUE, status=}
[11:00:21.979] (2.044s => +0.001s) update: 6/6 (+0) 'outer #6' {clear=TRUE, enabled=TRUE, status=}
[11:00:21.982] (0.000s => +0.001s) initiate: 0/4 (+0) '' {clear=TRUE, enabled=TRUE, status=}
[11:00:22.083] (0.102s => +0.001s) update: 1/4 (+1) 'inner #1' {clear=TRUE, enabled=TRUE, status=}
[11:00:22.185] (0.203s => +0.001s) update: 2/4 (+1) 'inner #2' {clear=TRUE, enabled=TRUE, status=}
[11:00:22.287] (0.305s => +0.001s) update: 3/4 (+1) 'inner #3' {clear=TRUE, enabled=TRUE, status=}
[11:00:22.389] (0.407s => +0.001s) update: 4/4 (+1) 'inner #4' {clear=TRUE, enabled=TRUE, status=}
[11:00:22.389] (0.408s => +0.001s) update: 4/4 (+0) 'inner #4' {clear=TRUE, enabled=TRUE, status=}

[HenrikBengtsson]

TL;DR: Use

f2 = function(x = 10) {
  p = progressr::progressor(steps = x)
  map(1:x, ~{
    res <- f1()
    p()
    res
})
}

to avoid this.

This is because with:

f_outer <- function(n = 6) {
  p <- progressr::progressor(n)
  for (kk in seq_len(n)) {
    p(sprintf("outer #%d", kk))
    f_inner()
  }
}

the progressor p completes when calling p(sprintf("outer #%d", kk)) at the last kk = n iteration. When it completes, it also "auto finishes", i.e. it removes itself automatically, letting any following progressor to take over. In this example, calling f_inner() immediately after, will create a new progressor and there is no longer an "outer" progressor blocking it's signals from propagating all the way to the progress handler. Swapping the order of p() and f_inner(), avoids this;

f_outer <- function(n = 6) {
  p <- progressr::progressor(n)
  for (kk in seq_len(n)) {
    res <- f_inner()
    p(sprintf("outer #%d", kk))
    res
  }
}

[andrewGhazi]

Ah okay, that makes sense, thank you! Putting p() after the inner function call has the added benefit of updating the outer progressor after each slow inner step, which probably aligns better with the expected behavior.