search

tabular-io / iceberg-kafka-connect

Apache License 2.0
202 stars 46 forks source link

Question: How the kafka iceberg connector maps types in kafka connect to types in iceberg? #263

Open arshahmad1 opened 3 months ago

arshahmad1 commented 3 months ago

Hi, Can I please get any document reference on how does the Kafka Iceberg connector map types in Kafka Connect to types in Iceberg? For example, with a number in JSON, the JsonSchemaConverter or JsonConverter will infer its Kafka Connect type during deserialization. Then, the connector will map this Connect type to an Iceberg type. What Iceberg type will it use?

tabmatfournier commented 3 months ago

You can find the answers here: https://github.com/tabular-io/iceberg-kafka-connect/blob/main/kafka-connect/src/main/java/io/tabular/iceberg/connect/data/SchemaUtils.java#L209