https://blog.orhun.dev/stdout-vs-stderr/

[utterances-bot]

Why stdout is faster than stderr? - Orhun's Blog

FOSS • Linux • Programming

https://blog.orhun.dev/stdout-vs-stderr/

[johslarsen]

First of all, nice article, and lots of interesting discussion. People should know these things, and you are good at writing captive blog posts, so this is probably going to help educate people. Also, getting people to play with these things is a very good idea.

However, you should probably mention pretty early (maybe alongside the I/O streams listing) that this difference is intentional, and that using the intended pipe for the right writes is better than changing the buffering behavior in most cases. In particular this helps getting the expected behavior from CLI utilities called via stdbuf, which is very useful to force line buffered output in combination with e.g. tee (or redirected to a logfile you are tailing somewhere else) to get back the immediate response (instead of waiting for it to fill a 4K page, when the output is not directly a TTY). Like the difference with and without the stdbuf in this simulated example:

(for ((i=0;;++i)); do echo $i; sleep 0.1; done) | stdbuf -oL grep '5$' | tee /dev/null

In your use case with TUI applications changing defaults might be warranted, although using stdout to begin with is probably the most common approach.

There is probably a lot of rational written around the web about how these streams are meant to be used in case you are curious. For stdout you figured out the reason that buffering helps a lot with performance (especially when the output is not a TTY), and the gist of it for stderr is summarized nicely in the last sentence here from the notes section in the man page:

The stream stderr is unbuffered. The stream stdout is line-buffered when it points to a terminal. Partial lines will not appear until fflush(3) or exit(3) is called, or a newline is printed. This can produce unexpected results, especially with debugging output. [man 3 stderr]

[sweisman]

The investigation and details described were interesting, but I, and many others, already knew that STDOUT is buffered by default and STDERR is not.

It is easy to disable buffering, and sometimes that is desired. In Perl:

$| = 1;

If you are piping data from one program to another, you don't want it buffered.

[lambda-aw]

the unbuffered output is good for CICD, I am using it with Python, so I can check actual state, and not waiting for test end - which can take a while with hardware stuff. python3 -u -m pytest ... is your friend.