Text progress bar not showing up in Jupyter notebook

[imathews]

I've been unable to get text progress bars (or really any progressr handler) to display within a Jupyter notebook. Given the following test case in a Jupyter environment, nothing displays when using progressr, though displays as expected if using utils::txtProgressBar. Both scenarios work just fine in RStudio.

library(progressr)
handlers('txtprogressbar')

test_slow_sum <- function(x) {
  # If we uncomment txtProgressBar lines, it works just as expected. But if we use progressr, nothing is displayed.
  # pb <- txtProgressBar(min = 0, max = length(x), initial = 0) 
  # p <- progressr::progressor(along = x)
  for (kk in seq_along(x)) {
    Sys.sleep(0.1)
    # setTxtProgressBar(pb,kk)
    # p(kk)
  }
}

with_progress(test_slow_sum(1:10))

https://github.com/HenrikBengtsson/progressr/assets/5659375/ec29b8ec-6020-4e5e-a281-dd41df6a5a23

[HenrikBengtsson]

Sounds like Jupyter Notebook handles output differently. I don't use it, so I haven't tested it there.

What happens if you use:

handlers(handler_txtprogressbar(file=stdout()))

?

[imathews]

Unfortunately that doesn't seem to work either. I've put together a sample colab notebook to reproduce this easily (I'm fairly certain colab is using the same IRKernel as Jupyter under the hood to communicate with R, at the least we see the same behavior).

One other piece of information is that calling a simple cat("\r", val) within the for loop doesn't yield any output until the end, though if we add a flush.console() after cat(), we can see the output updated live.

I tried writing my own custom handler to leverage this behavior, though was unsuccessful - likely due to my limited knowledge here.

[nicolaschapados]

Adding my vote here. I'd really like this to work properly.

[HenrikBengtsson]

I've looked more into this, and it's quite convoluted; Jupyter Notebook hacks utils::flush.console(). If we print(utils::flush.console) that function in vanilla R, we get:

> utils::flush.console
function () 
invisible(.Call(C_flushconsole))
<environment: namespace:utils>

by when doing so in Jupyter Notebook, we get:

function () 
{
    backup_env$utils_flush_console()
    flush_console()
}
<bytecode: 0x62b7266e87f0>
<environment: 0x62b726ea48b0>                 

After some more digging, that internal flush_console() is evaluate::flush_console(). I think that can be customized, and Jupyter Notebook customize it in ways I don't control. I don't think their hack can handle output to the standard error (stderr); it might be that it just swallows such output. Here's an example without progressr:

pbcon <- stderr()
pb <- utils::txtProgressBar(max = 100, file = pbcon)
for (value in c(0, 30, 40, 60, 80, 99)) {
  utils::setTxtProgressBar(pb, value = value)
  Sys.sleep(0.1)
}
cat("\n", file = pbcon)

That does not produce any progress output in the Jupyter Notebook GUI. If we switch to pbcon <- stdout(), we see the progress output.

So, a workaround is to output to have progressr output to stdout. Here is a proof of concept:

library(progressr)

## Force progressr bar even when running in non-interactive mode,
## e.g. interactive() is FALSE in Jupyter Notebook.
options(progressr.enable = TRUE)

## Jupyter Notebook hacks utils::flush.console(), but it appears
## to be incomplete and fail to handle output to the standard error
## (stderr), which is where we ideally would send progress output.
## Because of this limitation, we have to tell progressr to output
## to the standard output (stdout) instead.

handlers(handler_txtprogressbar(file = stdout()))

## For this to work, we also have to disable progressr's buffering
## of stdout
options(progressr.delay_stdout = FALSE)

## Test
progressr::with_progress({
  p <- progressor(10)
  for (value in 1:10) {
    p()
    Sys.sleep(1)
  }
})

I've confirmed that this works.

I think the best would be to see if there is an official Progress API for Jupyter Notebook and rely on that to report on progress, i.e. to create a handler_jupyter().