Support of Iceberg compression configuration

10xfuturetechnologies / kafka-connect-iceberg

Kafka Connector for Iceberg tables

Apache License 2.0

16 stars 5 forks source link

Support of Iceberg compression configuration #35

Open github-raphael-douyere opened 1 year ago

github-raphael-douyere commented 1 year ago

I'm writing content to an Iceberg table and it seems the connector only writes content in GZIP. Even if the table is configured with write.parquet.compression-codec=zstd.

Are you aware of this limitation or is it something wrong on my side? If it is a limitation, would it be possible to fix it? Thanks