Add support for Single Message Transforms (SMTs) - Githubissues

Azure / kafka-sink-azure-kusto

Kafka sink for Kusto

MIT License

48 stars 24 forks source link

Add support for Single Message Transforms (SMTs) #58

Closed AskMeiPaaS closed 3 years ago

AskMeiPaaS commented 3 years ago

The Kafka Connect API also provides a simple interface for manipulating records as they flow through both the source and sink side of your data pipeline. This API is known as Single Message Transforms (SMTs), and as the name suggests, it operates on every single message in your data pipeline as it passes through the Kafka Connect connector.

Some common uses for transforms are:

Renaming fields
Masking values
Routing records to topics based on a value
Converting or inserting timestamps into the record
Manipulating keys, like setting a key from a field’s value

ohadbitt commented 3 years ago

Hi SMTs are part of Kafka connect platform and not of the sink you can look here for an example of how to integrate it https://github.com/microsoft/kafka-connect-cosmosdb/blob/b5e29740bd53cc0f9d9688e50c8961eb8580a163/doc/README_Sink.md#single-message-transforms

yogilad commented 3 years ago

Closing