ConduitIO / conduit

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
https://conduit.io
Apache License 2.0
399 stars 49 forks source link

Feature: Decode record before evaluating processor condition #1937

Open lovromazgon opened 2 weeks ago

lovromazgon commented 2 weeks ago

Feature description

A processor condition is currently evaluated on the record as-is. If a record has a schema attached and encoded avro data in the payload, it has no access to the data. Conduit should decode the data before passing it to the condition template.

The workaround right now is to use the field.set processor to evaluate the condition and set a metadata field with the result, then use that metadata field in the condition on the next processor (the metadata is not encoded and is accessible in the condition).

srikary12 commented 1 week ago

I would like to work on this