search

scalyr / scalyr-agent-2

The source code for Scalyr Agent 2, the daemon process Scalyr customers run on their servers to collect metrics and logs.
Apache License 2.0
70 stars 56 forks source link

Print warnings of scalyr-agent-2-config to stderr #704

Open schra opened 3 years ago

schra commented 3 years ago

What

Following the instructions from https://app.scalyr.com/help/scalyr-agent-k8s#export-config fails

root@scalyr-84g88:/# scalyr-agent-2-config --export-config - | tar -xz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Why

Warnings are printed to stdout instead of stderr and thus are also piped to tar:

root@scalyr-84g88:/# scalyr-agent-2-config --export-config  -
2021-01-19 15:52:36.895Z WARNING [monitor:__main__] [configuration.py:327] Configured option min_request_spacing_interval is being overridden due to max_send_rate_enforcement setting.
D`�o0���K��JJk��6A߽�T}
�OH�f�\�jF�ߑʪ ͫty͂$͓lF�gv8�d3�p�\+�g�3*%Uhn$N�ޠG+a�^�7�O�Y%⺏gxy߰9�
.az�v��˸���r��g���Fsa���5�_:y{q��ƫ�~NIUR�yEQ�[U�_+i����濦62Z���%��F��>�y�?~ϊ�� 5ǹ(root@scalyr-84g88:/# `��

Manually removing the warning is a workaround that works:

root@scalyr-84g88:/# scalyr-agent-2-config --export-config  - | tail -n +2 | tar -xz
root@scalyr-84g88:/#

Docker image version: 2.1.15

Definition of done

The logger should print warnings and errors to stderr and not strdout. I think this should probably be done here: https://github.com/scalyr/scalyr-agent-2/blob/539b012d662e264c9058bc5bac26b4cb4f62c479/scalyr_agent/scalyr_logging.py#L75-L91

jmorascalyr commented 3 years ago

Thanks for submitting this. This has been submitted to engineering, and we will let you know when there are any updates on it. Thanks again for such a thorough analysis, troubleshooting, and repro steps.

jmorascalyr commented 3 years ago

Digging into this deeper, can you give me an idea of what you are trying to modify? Or did you just stumble upon this bug? Though this is a way to modify the agent config, it is sort of old. It is still useful, but there are easier ways to make changes.

schra commented 3 years ago

Digging into this deeper, can you give me an idea of what you are trying to modify? Or did you just stumble upon this bug? Though this is a way to modify the agent config, it is sort of old.

I followed the documentation (https://app.scalyr.com/help/scalyr-agent-k8s#export-config) which told me to execute scalyr-agent-2-config --export-config - | tar -xz and so I executed it and it didn't work due to the warning being printed to stdout.

It is still useful, but there are easier ways to make changes.

Maybe updating this paragraph here https://app.scalyr.com/help/scalyr-agent-k8s#export-config and deprecating --export-config would be helpful if there are easier ways to make changes.