tabular-io / iceberg-kafka-connect

Apache License 2.0
203 stars 46 forks source link

one more slash was added on S3 prefix when using iceberg.tables.auto-create-enabled option #127

Closed okayhooni closed 11 months ago

okayhooni commented 11 months ago

When I using the connector with iceberg.tables.auto-create-enabled option, one more slash was added on S3 prefix like below

if) default db LOCATION of temp_beta database : s3://test_bucket/temp_beta

I think it's not intended result unlike https://github.com/tabular-io/iceberg-kafka-connect/issues/118

image

By the way, is iceberg.tables.evolve-schema-enabled option working well..? If so, how about updating Tabular medium blog for this connector..? (on Future enhancements phrase, it say there is no table schema evolution feature as yet)

https://tabular.medium.com/intro-to-the-iceberg-kafka-connect-sink-f0f86eae2937

bryanck commented 11 months ago

I believe this is an issue with Iceberg itself, you may want to raise an issue there. Schema evolution is working and we are currently working on a series of follow-up blog posts to detail some of the new features, so stay tuned.

bryanck commented 11 months ago

Thanks for pointing out the outdated docs, I'll update the future enhancements section.

bryanck commented 11 months ago

This PR updates the design doc: https://github.com/tabular-io/iceberg-kafka-connect/pull/131

okayhooni commented 11 months ago

Thank you @bryanck !

bryanck commented 11 months ago

By the way, there will be a PR in the Iceberg repo shortly, to fix the extra slash issue when using the Glue catalog, in case you want to follow along. Thanks for reporting that.