futureverse / progressr

三 R package: An Inclusive, Unifying API for Progress Updates
https://progressr.futureverse.org
281 stars 12 forks source link

Text progress bar not showing up in Jupyter notebook #170

Open imathews opened 4 months ago

imathews commented 4 months ago

I've been unable to get text progress bars (or really any progressr handler) to display within a Jupyter notebook. Given the following test case in a Jupyter environment, nothing displays when using progressr, though displays as expected if using utils::txtProgressBar. Both scenarios work just fine in RStudio.

library(progressr)
handlers('txtprogressbar')

test_slow_sum <- function(x) {
  # If we uncomment txtProgressBar lines, it works just as expected. But if we use progressr, nothing is displayed.
  # pb <- txtProgressBar(min = 0, max = length(x), initial = 0) 
  # p <- progressr::progressor(along = x)
  for (kk in seq_along(x)) {
    Sys.sleep(0.1)
    # setTxtProgressBar(pb,kk)
    # p(kk)
  }
}

with_progress(test_slow_sum(1:10))

https://github.com/HenrikBengtsson/progressr/assets/5659375/ec29b8ec-6020-4e5e-a281-dd41df6a5a23

HenrikBengtsson commented 4 months ago

Sounds like Jupyter Notebook handles output differently. I don't use it, so I haven't tested it there.

What happens if you use:

handlers(handler_txtprogressbar(file=stdout()))

?

imathews commented 4 months ago

Unfortunately that doesn't seem to work either. I've put together a sample colab notebook to reproduce this easily (I'm fairly certain colab is using the same IRKernel as Jupyter under the hood to communicate with R, at the least we see the same behavior).

One other piece of information is that calling a simple cat("\r", val) within the for loop doesn't yield any output until the end, though if we add a flush.console() after cat(), we can see the output updated live.

I tried writing my own custom handler to leverage this behavior, though was unsuccessful - likely due to my limited knowledge here.

nicolaschapados commented 3 months ago

Adding my vote here. I'd really like this to work properly.

HenrikBengtsson commented 2 weeks ago

I've looked more into this, and it's quite convoluted; Jupyter Notebook hacks utils::flush.console(). If we print(utils::flush.console) that function in vanilla R, we get:

> utils::flush.console
function () 
invisible(.Call(C_flushconsole))
<environment: namespace:utils>

by when doing so in Jupyter Notebook, we get:

function () 
{
    backup_env$utils_flush_console()
    flush_console()
}
<bytecode: 0x62b7266e87f0>
<environment: 0x62b726ea48b0>                 

After some more digging, that internal flush_console() is evaluate::flush_console(). I think that can be customized, and Jupyter Notebook customize it in ways I don't control. I don't think their hack can handle output to the standard error (stderr); it might be that it just swallows such output. Here's an example without progressr:

pbcon <- stderr()
pb <- utils::txtProgressBar(max = 100, file = pbcon)
for (value in c(0, 30, 40, 60, 80, 99)) {
  utils::setTxtProgressBar(pb, value = value)
  Sys.sleep(0.1)
}
cat("\n", file = pbcon)

That does not produce any progress output in the Jupyter Notebook GUI. If we switch to pbcon <- stdout(), we see the progress output.

So, a workaround is to output to have progressr output to stdout. Here is a proof of concept:

library(progressr)

## Force progressr bar even when running in non-interactive mode,
## e.g. interactive() is FALSE in Jupyter Notebook.
options(progressr.enable = TRUE)

## Jupyter Notebook hacks utils::flush.console(), but it appears
## to be incomplete and fail to handle output to the standard error
## (stderr), which is where we ideally would send progress output.
## Because of this limitation, we have to tell progressr to output
## to the standard output (stdout) instead.

handlers(handler_txtprogressbar(file = stdout()))

## For this to work, we also have to disable progressr's buffering
## of stdout
options(progressr.delay_stdout = FALSE)

## Test
progressr::with_progress({
  p <- progressor(10)
  for (value in 1:10) {
    p()
    Sys.sleep(1)
  }
})

I've confirmed that this works.

I think the best would be to see if there is an official Progress API for Jupyter Notebook and rely on that to report on progress, i.e. to create a handler_jupyter().