sosreport / sos

A unified tool for collecting system logs and other debug information
http://sos.rtfd.org
GNU General Public License v2.0
508 stars 542 forks source link

Handle stderr better in sosreport? #2705

Open pponnuvel opened 3 years ago

pponnuvel commented 3 years ago

Backstory: I was parsing the output ceph report in Python. The command produces a json output but also prints a line to stderr. Since both stdout & stderr are redirected to the same file, it couldn't be loaded as a json directly (had to remove the last line to make it a valid json).

I wonder whether stderr of the commands need to be handled in a different way. For example, redirect to <cmd_options>.stderr and the stdout would go to <cmd_options>.stdout or similar. A number of commands, Ceph in particular, produce json output. So doing any post-processing would be difficult when mixing stdout and stderr. I'd imagine more commands producing a formatted output going forward than a plain text, especially for printing out configs, perf counters, etc.

Just wanted to start a discussion and check whether this could be addressed in some way.

bmr-cymru commented 3 years ago

There's some subtlety here as there are cases where the merged output is more useful as well as the reverse. Especially for commands that produce structured output (JSON, XML etc.) I think there is value in allowing plugin authors to optionally redirect the error stream. On the other hand there are commands where it's useful to have stderr and stdout merged because it allows the errors to be correlated with some previous logged output.

pmoravec commented 3 years ago

You speak about

https://github.com/sosreport/sos/blob/main/sos/report/plugins/ceph_mon.py#L39

right? Or some other cmds as well?

We do have an option to collect stderr which is enabled by default:

https://github.com/sosreport/sos/blob/main/sos/report/plugins/__init__.py#L1718-L1746

Maybe worth just disabling it in this case (with the risk that in case of errors, you will not see the stderr output - is that acceptable?).