process$new(stdout = "") like system2()?

[yihui]

I guess this must be a stupid question, but I'm looking for a way to let the command write to stdout just like system2() does by default. Current possible options are TRUE/FALSE, a file, and |. None of these are what I'm looking for.

It is hard to tell what exactly the command is currently doing in the background without live feedback.

[gaborcsardi]

files <- processx::run("ls", echo = TRUE)

Maybe?

[yihui]

I need the process to run in the background. The current use case is that I'm trying to start hugo server in blogdown, and this command is blocking and will keep running until it is interrupted, so I have to use process$new() instead of run().

[gaborcsardi]

Then you can create a pipe to the process with "|" and call poll_io() to get its output. This way you can make a nice progress bar or status bar.

[yihui]

Yeah I thought about that, but it will require me to use an infinite loop in R to keep polling, right? The loop will block my current R session, which is not what I want.

[gaborcsardi]

You need to keep polling, yes, but that does not block your current R session, you can write a simple event loop, e.g. https://github.com/r-lib/revdepcheck/blob/65fbda0fbb4ddd91958df806ae4ee9de5d13e389/R/event-loop.R#L60-L70

[yihui]

Interesting. I'll take a look. Thanks for the tips!

[gaborcsardi]

Or, you want to keep an interactive R session open, while running the process in the background, and get its output?

For that you'll need an additional tool, I am afraid, e.g. the later package can do the polling and printing.

[gaborcsardi]

Direct printing to the R process's stdout is not supported, because that just messes up the UI, and also does not compose. I.e. if another package is running something in the background, and they both just print their output on the R console stdout, that'll be a big mess.

[yihui]

Yeah, I understand the potential problem of multiple processes writing to stdout at the same time. If this is the only reason that you do not support it, I guess you may support it with a warning in the documentation. In my case, I don't care if the stdout will be a big mess or not. It is more informative than complete silence.

[gaborcsardi]

I implemented this now for Unix, but it does not seem to be any meaningful way that works in Windows GUIs.

I think putting the background process on rstudio job pane might be better: https://github.com/r-lib/processx/pull/256

Or just polling in later, which is actually quite simple, hugodown does it already: https://github.com/r-lib/hugodown/blob/18911fca8633029e403a856034b82ec56661bed5/R/hugo-server.R#L81-L94 (Not entirely correct, but almost.)

[gaborcsardi]

OK, I found a way to do it in RStudio, on Windows as well. Luckily RStudio sets the standard output and error to pipes, which are streamed back to the console, by a background thread. So it all just works, assuming one queries the Unix streams (i.e. _get_osfhandle(1L), instead of the Windows console streams with GetStdHandle(STD_OUTPUT_HANDLE), because system(..., wait = FALSE) aggressively removed the standard handles in RGui (and thus RStudio as well): https://github.com/wch/r-source/blob/d22ee2fc0dc8142b23eed9f46edf76ea9d3ca69a/src/gnuwin32/sys-win32.c#L258-L260

The syntax is similar to system2(), but unlike system2(..., stdout = "", wait = FALSE), this actually works:

bg <- process$new("ls", stdout = "", stderr = "")

If the R process does not have a standard output/error stream open (e.g. RGui on Windows, then this fails with an error). To avoid the error, you can query if the R process has a standard output and error stream, and if not, then do something else. (Not sure what, though, at this point.) This is how to query that stdout and stderr are valid streams:

❯ is_valid_fd(1L)
[1] TRUE

❯ is_valid_fd(2L)
[1] TRUE

sys::exec_background() redirects the output to a pipe, which is then read by a background thread, that puts all output into fprintf(stderr): https://github.com/jeroen/sys/blob/c4fd835c83ea3b018d28c45e7b565e33e93f2de6/src/win32/exec.c#L117 This still does not work in RGui because there is no stderr, so the background thread just fails on the first write and then exits. But maybe this is a fixable approach.