siddhi-io / siddhi-io-kafka

Extension that can be used to receive events from a Kafka cluster and to publish events to a Kafka cluster
https://siddhi-io.github.io/siddhi-io-kafka/
Apache License 2.0
18 stars 50 forks source link

Support record transformation during consumption #125

Open pcnfernando opened 4 years ago

pcnfernando commented 4 years ago

Description: It would be more efficient during consumption, if Siddhi Source is improved to be configured with transformations to make lightweight message-at-a-time modifications.

For example, if there are 50 fields in a record and user only needs 2 fields to be input to Siddhi, there'll be a performance hit during consumption since mapping is done after consumption of the whole message.

We can refer https://docs.confluent.io/current/connect/transforms/index.html