questdb / kafka-questdb-connector

QuestDB connector for Kafka.
https://github.com/questdb/kafka-questdb-connector/releases/
Apache License 2.0
31 stars 5 forks source link

feat: native support for parsing string timestamps #7

Closed jerrinot closed 1 year ago

jerrinot commented 1 year ago

The TimestampTransform shipped with Kafka Connect does not support microseconds at all.

Native support also means users are not forced to learn about Kafka Connect SMT. It's been proven difficult in cases where users just want to move data from Kafka to QuestDB, but are not Kafka experts.

This PR introduces 2 new properties: timestamp.string.fields - Comma-separated list of string fields that should be parsed as timestamps. timestamp.string.format - Timestamp format. Used when parsing timestamp string fields. See https://questdb.io/docs/reference/function/date-time/#date-and-timestamp-format for info about formatting.