search

opensearch-project / data-prepper

Data Prepper is a component of the OpenSearch project that accepts, filters, transforms, enriches, and routes data at scale.
https://opensearch.org/docs/latest/clients/data-prepper/index/
Apache License 2.0
259 stars 190 forks source link

Feature request - Support MessagePack #852

Open ryn9 opened 2 years ago

ryn9 commented 2 years ago

Is your feature request related to a problem? Please describe. Not related to a problem - this is a feature request

Describe the solution you'd like Like the requests to better support parsing and encoding json - please look to support MessagePack Please see https://msgpack.org/

Describe alternatives you've considered (Optional) N/A.

Additional context Requests regarding parsing and encoding json: https://github.com/opensearch-project/data-prepper/issues/832 https://github.com/opensearch-project/data-prepper/issues/831

dlvenable commented 2 years ago

@ryn9 , I want to clarify your request.

You mentioned this in the context of the encode/decode JSON features. These expect an Event which has serialized JSON in a specific field and then deserializes it. Do you have events with MessagePack fields which then need to be decoded?

Or are you looking to accept MessagePack as an entire input into Data Prepper?

ryn9 commented 1 year ago

Or are you looking to accept MessagePack as an entire input into Data Prepper?

If it proves more performant - it would be good to support this as its popular for use with fluentbit/fluentd