search

tabular-io / iceberg-kafka-connect

Apache License 2.0
192 stars 41 forks source link

Unclosed S3FileIO instance warning was raised when using Glue catalog #122

Closed okayhooni closed 10 months ago

okayhooni commented 10 months ago

When I tested this connector with AWS Glue catalog with S3FileIO option, I got the error like below.

Isn't it some bug..?

WARN Unclosed S3FileIO instance created by:
    org.apache.iceberg.aws.s3.S3FileIO.initialize(S3FileIO.java:362)
    org.apache.iceberg.CatalogUtil.loadFileIO(CatalogUtil.java:325)
    org.apache.iceberg.aws.glue.GlueTableOperations.initializeFileIO(GlueTableOperations.java:223)
    org.apache.iceberg.aws.glue.GlueTableOperations.io(GlueTableOperations.java:115)
    org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$0(BaseMetastoreTableOperations.java:189)
    org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$1(BaseMetastoreTableOperations.java:208)
    org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
    org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
    org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
    org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
    org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:208)
    org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:185)
    org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:176)
    org.apache.iceberg.aws.glue.GlueTableOperations.doRefresh(GlueTableOperations.java:141)
    org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:97)
    org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:80)
    org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:47)
    io.tabular.iceberg.connect.channel.Coordinator.commitToTable(Coordinator.java:180)
    io.tabular.iceberg.connect.channel.Coordinator.lambda$doCommit$1(Coordinator.java:147)
    org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
    org.apache.iceberg.util.Tasks$Builder.access$300(Tasks.java:69)
    org.apache.iceberg.util.Tasks$Builder$1.run(Tasks.java:315)
    java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    java.base/java.lang.Thread.run(Thread.java:833) (org.apache.iceberg.aws.s3.S3FileIO) [Finalizer]
bryanck commented 10 months ago

This is a warning that you can safely ignore. When the sink is updated to Iceberg 1.4 those should improve.

okayhooni commented 10 months ago

This is a warning that you can safely ignore. When the sink is updated to Iceberg 1.4 those should improve.

Thank you for quick answer! :)