Hard to extract syslog rfc5424 structured data from other syslog fields

[ethuleau]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

The parse_syslog stores all the structured-data sd_id at the same level of other fields of a rfc5424 syslog message. Which makes it hard to extract it from the others. I managed to do it by using in dev version 0.23.0 which change the parsed format of structured-data fields https://github.com/vectordotdev/vector/commit/b4a1cd353e291e3c51cb75024a6882a2e6c69297

    .syslog = parse_syslog!(.message)
    for_each(object!(.syslog)) -> |sd_id, sd_param| {
        if is_object(sd_param) {
            for_each(object!(sd_param)) -> |param_name, param_value| {
                .mm.keyValList = push!(.mm.keyValList, "{{ sd_id }} {{ param_name }}="{{ param_value }}"" ?? "")
            }
        }
    }

Configuration

No response

Version

0.23.0

Debug Output

No response

Example Data

<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog 42 ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] BOMAn application event log entry... ### Additional Context _No response_ ### References https://discord.com/channels/742820443487993987/996045325028569109

[jszwedko]

Hi @ethuleau !

Thanks for this report. To clarify, is the behavior in the development version working for you? Or do you need additional functionality?

[ethuleau]

Hi @jszwedko, Development version help me to distinguish structured data fields from other syslog fields. But it will be simpler to put parsed structured data in a separate filed like structured_data. That will prevent the outer for_each loop in the above snippet.

[jszwedko]

Ah, I see @ethuleau . That makes sense.