databricks / iceberg-kafka-connect

Apache License 2.0
220 stars 49 forks source link

one more slash was added on S3 prefix when using iceberg.tables.auto-create-enabled option #127

Closed okayhooni closed 1 year ago

okayhooni commented 1 year ago

When I using the connector with iceberg.tables.auto-create-enabled option, one more slash was added on S3 prefix like below

if) default db LOCATION of temp_beta database : s3://test_bucket/temp_beta

I think it's not intended result unlike https://github.com/tabular-io/iceberg-kafka-connect/issues/118

image

By the way, is iceberg.tables.evolve-schema-enabled option working well..? If so, how about updating Tabular medium blog for this connector..? (on Future enhancements phrase, it say there is no table schema evolution feature as yet)

https://tabular.medium.com/intro-to-the-iceberg-kafka-connect-sink-f0f86eae2937

bryanck commented 1 year ago

I believe this is an issue with Iceberg itself, you may want to raise an issue there. Schema evolution is working and we are currently working on a series of follow-up blog posts to detail some of the new features, so stay tuned.

bryanck commented 1 year ago

Thanks for pointing out the outdated docs, I'll update the future enhancements section.

bryanck commented 1 year ago

This PR updates the design doc: https://github.com/tabular-io/iceberg-kafka-connect/pull/131

okayhooni commented 1 year ago

Thank you @bryanck !

bryanck commented 1 year ago

By the way, there will be a PR in the Iceberg repo shortly, to fix the extra slash issue when using the Glue catalog, in case you want to follow along. Thanks for reporting that.