search

SAP / kafka-connect-sap

Kafka Connect SAP is a set of connectors, using the Apache Kafka Connect framework for reliably connecting Kafka with SAP systems
Apache License 2.0
119 stars 54 forks source link

Add a generic field name escape transformer #129

Closed elakito closed 2 years ago

elakito commented 2 years ago

This is a utility SMT to transform HANA column names that do not fit to the avro field naming scheme.

The escape encoding scheme is similar to that of URLEncoding where each character not belonging to the valid character set will be replaced with its UTF-8 byte sequence <esc><xx>, where <esc> is the specified the escape character and <xx> is the hexadecimal representation of its value.

For example, to encode arbitrary field names into valid avro names, add this transformer with the following setting at the source connector.

        "transforms": "escapeChars",
        "transforms.escapeChars.type": "com.sap.kafka.connect.transforms.EscapeFieldNameCharacters$Value",
        "transforms.escapeChars.valid.chars.default": "[A-Za-z0-9_]",
        "transforms.escapeChars.valid.chars.first": "[A-Za-z_]",
        "transforms.escapeChars.escape.char": "_",

To revert the field names back to the original field names, add this transformer with the following setting at the sink connector.

        "transforms": "escapeChars",
        "transforms.escapeChars.type": "com.sap.kafka.connect.transforms.EscapeFieldNameCharacters$Value",
        "transforms.escapeChars.escape.char": "_",

This transformer maybe be used to solve problems mentioned in #126.