search

ibm-messaging / kafka-connect-mq-sink

This repository contains a Kafka Connect sink connector for copying data from Apache Kafka into IBM MQ.
Apache License 2.0
35 stars 43 forks source link

Filter messages from Kafka topic based on header value #25

Closed harishanandan124 closed 4 years ago

harishanandan124 commented 4 years ago

Hi All,

Is there any way we can filter incoming messages from a topic based on its header field value? I tried writing transformations but none of them worked. Also, I do not see a configuration for 'filter' option in the list of config options provided. Has anyone tried it before?

AndrewJSchofield commented 4 years ago

Hi, you could write a single-message transform and configure the connector to use that. Because the SMT technology exists, I personally wouldn't want to put filtering directly in the connector.

A good, general-purpose filtering SMT would be quite complicated. A simpler one with just a single purpose in mind would be fairly simple, but you will need to understand quite a bit about how Kafka Connect works to do it.

AndrewJSchofield commented 4 years ago

Closing dormant issue