`vector validate` doesn't respect `--log-format`

josh-m-sharpe commented 2 months ago

A note for the community

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

We're launching vector with --log-format json

If I monitor journalctl -u vector -f I see output like below which is not JSON. And I'm fairly certain based on the messaging it's not systemd/journal that's creating those messages. This makes it somewhat difficult to pipe vector logs from journalctl off the system since some are JSON formatted and others are not. These also don't have timestamps or log level appended the way the JSON formatted logs do.

o│Sep 11 02:51:10 ip-10-20-11-15.us-east-2.compute.internal vector[350465]: √ Loaded ["/etc/vector/vector.yaml"]
y│Sep 11 02:51:10 ip-10-20-11-15.us-east-2.compute.internal vector[350465]: √ Component configuration
 │Sep 11 02:51:10 ip-10-20-11-15.us-east-2.compute.internal vector[350465]: √ Health check "loki_sink_journald"
(│Sep 11 02:51:10 ip-10-20-11-15.us-east-2.compute.internal vector[350465]: ------------------------------------
m│Sep 11 02:51:10 ip-10-20-11-15.us-east-2.compute.internal vector[350465]:                            Validated

Configuration

No response

Version

vector 0.41.0 (aarch64-unknown-linux-gnu 97952b5 2024-09-09 16:10:33.247365706)

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

josh-m-sharpe commented 2 months ago

I see now these lines appear to be related to ExecReload=/usr/bin/vector validate in my systemd file. But upon fiddling a bit more It seems that vector validate doesn't support --log-format JSON.

So, the problem is the same. The lines that end up in journald are mixed formatting and it makes getting them off the system and in a viewable tool (loki, grafana in my case) much more difficult than it would be if all the logs were JSON.

jszwedko commented 2 months ago

Thanks for this report @josh-m-sharpe . It does appear that vector validate doesn't respect --log-format. I'll update the issue title here.

vectordotdev / vector