search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.49k stars 3.02k forks source link

Reading Delta Lake tables with icebergCompatV1(2) writer features fails #22106

Closed ebyhr closed 5 months ago

ebyhr commented 6 months ago 
CREATE TABLE default.test_iceberg
(a INT)
USING DELTA
LOCATION 's3://test-bucket/test_iceberg'
TBLPROPERTIES ('delta.enableIcebergCompatV1'=true);
TABLE test_iceberg;
Query 20240524_085914_00002_2n6ua failed: Both reader and writer features should must the same value for 'columnMapping'. reader: false, writer: true
java.lang.IllegalArgumentException: Both reader and writer features should must the same value for 'columnMapping'. reader: false, writer: true
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:448)
    at io.trino.plugin.deltalake.transactionlog.DeltaLakeSchemaSupport.getColumnMappingMode(DeltaLakeSchemaSupport.java:188)
    at io.trino.plugin.deltalake.DeltaLakeMetadata.getTableHandle(DeltaLakeMetadata.java:621)
    at io.trino.plugin.deltalake.DeltaLakeMetadata.getTableHandle(DeltaLakeMetadata.java:323)
    at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.getTableHandle(ClassLoaderSafeConnectorMetadata.java:1259)
    at io.trino.tracing.TracingConnectorMetadata.getTableHandle(TracingConnectorMetadata.java:140)
    at io.trino.metadata.MetadataManager.lambda$getTableHandle$5(MetadataManager.java:286)
    at java.base/java.util.Optional.flatMap(Optional.java:289)
    at io.trino.metadata.MetadataManager.getTableHandle(MetadataManager.java:277)
    at io.trino.metadata.MetadataManager.getRedirectionAwareTableHandle(MetadataManager.java:1936)
    at io.trino.metadata.MetadataManager.getRedirectionAwareTableHandle(MetadataManager.java:1928)
    at io.trino.tracing.TracingMetadata.getRedirectionAwareTableHandle(TracingMetadata.java:1493)
    at io.trino.sql.analyzer.StatementAnalyzer$Visitor.getTableHandle(StatementAnalyzer.java:5837)
    at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:2290)
    at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:520)
    at io.trino.sql.tree.Table.accept(Table.java:60)
    at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
    at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:539)
    at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:547)
    at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:1568)
    at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:520)
    at io.trino.sql.tree.Query.accept(Query.java:118)
    at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
    at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:539)
    at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:499)
    at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:488)
    at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:97)
    at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:86)
    at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:285)
    at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:218)
    at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:884)
    at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:153)
    at io.trino.$gen.Trino_testversion____20240524_085826_71.call(Unknown Source)
    at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
    at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:76)
    at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1570)

Probably, this is a wrong implementation in Delta Lake side. https://github.com/delta-io/delta/issues/3154 Writer Requirements for Column Mapping mentions:

write a protocol action to add the feature columnMapping to both readerFeatures and writerFeatures

findepi commented 5 months ago

Both reader and writer features should must the same value

there is grammar error in this message

(https://github.com/trinodb/trino/pull/22313)