Capture stderr with knitr engine like to match Jupyter engine output

[hadley]

Render this doc. Where does the message go?

```{r}
cat("Hi!", file = stderr())

[hadley]

Maybe this only doesn't work in RStudio?

[hadley]

Now I can see it 😭

[cscheid]

For what's worth, I don't see any output when I try to repro this myself:

---
keep-md: true
---
```{r}
cat("Hi!", file = stderr())

The `keep-md` output is:

---

keep-md: true

::: {.cell}

cat("Hi!", file = stderr())

:::

I'm not sure this is a bug or not (or what's the expected knitr behavior), but I can reproduce the "where's the output" confusion.

[cderv]

The stderr() goes into same place at what is logged by knitr as progress.

==> quarto preview test2.qmd --to default --no-watch-inputs --no-browse

++ Activating rlang global_entrace

processing file: test2.qmd
  |...................................                 |  67% [unnamed-chunk-1]
Hi!

output file: test2.knit.md

pandoc 
  to: html
  output-file: test2.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: test

Output created: test2.html

Watching files for changes
Browse at http://localhost:4877/

I don't think knitr does anything specific and it is at the evaluate layer, where stderr() is used to log things.

@hadley when you said you could see it, is this now in the expected part ?

@cscheid don't you see the Hi! in the progress message we show while rendering ?

[cscheid]

Ah, yes. This is what I see (and you'll see why I missed it)

[cscheid]

I think we could be doing better than just printing to stderr; I'm not sure how to standardize this across engines, but my feeling is that this output should end up in the document by default.

[cderv]

I think we could be doing better than just printing to stderr;

You mean quarto showing the knitr logging in red as stderr ?

this output should end up in the document by default.

You mean the Hi! from cat("Hi!", file = stderr()) ? This code explicitly print to stderr(). Using stdout() will go in document as it will be catched by evaluate. But using stderr(), evaluate::evaluate() does not handle it in a special manner, and it is sent to stderr(), as other type of logging if you ask for it.

> evaluate::evaluate('cat("Hi!\n", file = stderr())')
Hi!
<evaluation>
Source code:
  cat("Hi!
  ", file = stderr())

> evaluate::evaluate('cat("Hi!\n", file = stderr())', log_echo = TRUE)
cat("Hi!
", file = stderr())

Hi!
<evaluation>
Source code:
  cat("Hi!
  ", file = stderr())

Not something we can really change at quarto level IMO. We have this behavior for knitr and evaluate itself.

[cscheid]

My point is that having stderr end up not at the document is a problem, because

• it's often hard to have access to the process's stderr.
• it goes against the idea that the execution of the process is more-or-less captured by the document output
• it's incompatible with the way Jupyter does it:

---
format: html
---

```{python}
import sys
print("Hi!", file=sys.stderr)

<img width="693" alt="image" src="https://github.com/user-attachments/assets/94eb43bb-ae37-4b06-a6cc-ff12b463c097">

[hadley]

Yeah, the problem is that I forgot the \n, so my output ended muddled up in the progress bar.

I do think it's correct to not capture stderr() because that gives you some standard way to escape knitr's capture and emit text into the log.

[cscheid]

@hadley So, aside from the increasing the visibility of the output, you don't think there's a bug here? Hm.

This works entirely differently in Jupyter, and I think the Jupyter behavior is less bad. I think printing into the log should require a more explicit API call...

[hadley]

@cscheid it's possible that the jupyter way is the right way here, but if you change this behaviour in quarto, it's going to mess up a LOT of existing logging code from Rmds.

[cscheid]

Noted.

I wonder if the best solution here is to add an option to both sides, and have different defaults for different engines. We already have some settings where jupyter behaves differently from knitr...

[hadley]

It's not currently possible to capture stderr in evaluate so I think it would be hard to implement currently, but that's definitely something I could add.

[cscheid]

It's not high priority - maybe something I can try at the 2026 dev day...