Need support for Key Value processor

[Prince-Ajudiya]

At the moment, we're working on processing unstructured logs using the GROK processor, but we've been encountering some issues. For instance, since there are various patterns for different types of logs, we need to include multiple GROK expressions, which can be time-consuming. Moreover, if there are any modifications to the logs in the future, the GROK processor will become ineffective. Below is a sample log that I've included for reference.

time="2018-07-04T09:36:25Z" level=info msg="this is a message" logSource="source".

Here, I got similar reference of Key Value processor that is exists in Elastic Processor. https://www.elastic.co/guide/en/elasticsearch/reference/current/kv-processor.html

For that, Do we have a Key value processor if not then it will be a good add-on WDYT?

[peczenyj]

I find this package that may help

https://github.com/jjeffery/kv

I wrote a small example

https://go.dev/play/p/8XJ8zsDHcSL

package main

import (
    "fmt"

    "github.com/jjeffery/kv"
)

func main() {
    line := `time="2018-07-04T09:36:25Z" level=info msg="this is a message" logSource="source"`
    msg, list := kv.Parse([]byte(line))

    fmt.Printf("msg: %s\nlist: %#v len: %d", msg, list, len(list))
}

Should be easier create a map[string][]string from the kv.List

However it will fail if we add an . In the end of the log line