search

observIQ / bindplane-agent

observIQ’s distribution of the OpenTelemetry collector providing a simple and unified solution to collect, refine, and ship telemetry data anywhere
Apache License 2.0
103 stars 27 forks source link

Add marshal processor #1830

Closed kuiperda closed 1 month ago

kuiperda commented 2 months ago

Proposed Change

Add first iteration of Marshal processor to our Agent. (Formerly "Serialize Processor").

The intent is to allow sending data to security platforms that do their own parsing and do not want the body to be in OTLP format. Currently these use cases cannot easily use Bindplane for data reduction on those logs since they must retain the original format.

This processor supports marshaling to JSON and KV.

XML support is going to be through an OTTL function that is being submitted to OTEL by Sam H.

This processor makes assumptions about the format of the body. For now we are going to expect users to parse and flatten body fields correctly before using this processor.

This is an initial draft that I expect to iterate on, but wanted to open it up for feedback.

Checklist
kuiperda commented 2 months ago

@BinaryFissionGames when you have some time, this is ready for re-review. Thanks!

kuiperda commented 1 month ago

@BinaryFissionGames This is ready for re-review when you have some time. Thanks!

kuiperda commented 1 month ago

Thanks for the thorough review on this, it is much improved from where it started.