Update Syslog source to accept non UTF-8 encoding in syslog message

[Neko-Follower]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Vector drops logs when encounter a syslog message with non UTF-8 characters. Can you add an option to replace non utf-8 characters with U+FFFD or allow passing non-UTF8 text as-is like a Promtail do.

Configuration

No response

Version

0.37.1-distroless-static

Debug Output

No response

Example Data

2024-05-08T07:35:43.209847Z DEBUG source{component_kind="source" component_id=rsyslog component_type=syslog}:connection{peer_addr=172.22.0.4:44600}: vector::sources::util::net::tcp: Accepted a new connection. peer_addr=172.22.0.4:44600

2024-05-08T07:35:44.293974Z ERROR source{component_kind="source" component_id=rsyslog component_type=syslog}:connection{peer_addr=172.22.0.4:44594}: vector::internal_events::codecs: Failed framing bytes. error=Unable to decode input as UTF8 error_code="decoder_frame" error_type="parser_failed" stage="processing" internal_log_rate_limit=true

2024-05-08T07:35:44.294029Z ERROR source{component_kind="source" component_id=rsyslog component_type=syslog}:connection{peer_addr=172.22.0.4:44594}: vector::internal_events::codecs: Internal log [Failed framing bytes.] is being suppressed to avoid flooding.

Additional Context

No response

References

No response

[jszwedko]

Agreed, this could be modeled like the existing decoding.codec.json.lossy option which replaces invalid UTF-8 characters.

[jszwedko]

We'd be happy to see a PR for this if someone is motivated! It should be a relatively straightforward change.

[osas1111]

I have the same problem with Fluent source