search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.44k stars 959 forks source link

[Bug] Error in querying table with row type column under trino 422 #2094

Closed macdoor closed 1 year ago

macdoor commented 1 year ago

Search before asking

Paimon version

paimon-trino-422-0.6-20231007.000450-31.jar

Compute Engine

Linux hb3-dev-gem-bnpmp-000 5.10.0-60.91.0.115.oe2203.x86_64 #1 SMP Thu Apr 20 07:08:44 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

openjdk version "1.8.0_352" OpenJDK Runtime Environment Bisheng (build 1.8.0_352-b08) OpenJDK 64-Bit Server VM Bisheng (build 25.352-b08, mixed mode)

Flink 1.17.1

Trino 422 - 428

Minimal reproduce step

create 3 tables

drop table if exists svc1_paimon_prod.dxx.with_row;
CREATE TABLE svc1_paimon_prod.dxx.with_row (
  id                    string,
  order_attrs           row<
    ordernum            string,
    paymentmode         string
  >,
  skuNum                string,
  PRIMARY KEY (id,skuNum) NOT ENFORCED
) 
  PARTITIONED BY (skuNum);

drop table if exists svc1_paimon_prod.dxx.without_row;
CREATE TABLE svc1_paimon_prod.dxx.without_row (
  id                    string,
  skuNum                string,
  PRIMARY KEY (id,skuNum) NOT ENFORCED
) 
  PARTITIONED BY (skuNum);

drop table if exists svc1_paimon_prod.dxx.with_row_append;
CREATE TABLE svc1_paimon_prod.dxx.with_row_append (
  id                    string,
  order_attrs           row<
    ordernum            string,
    paymentmode         string
  >,
  skuNum                string
) 
  PARTITIONED BY (skuNum);

insert 1 row data into each table 

INSERT INTO svc1_paimon_prod.dxx.with_row VALUES ('1', row('2', '2'), '3');
INSERT INTO svc1_paimon_prod.dxx.without_row VALUES ('1', '3');
INSERT INTO svc1_paimon_prod.dxx.with_row_append VALUES ('1', row('2', '2'), '3');

What doesn't meet your expectations?

execute sql in sql-client.sh

SELECT t.* FROM with_row t;

SELECT t.* FROM with_row_append t;

throw exception

[2023-10-07 20:17:26] [65536] Query failed (#20231007_121724_00000_xafpd): 'io.trino.spi.block.BlockBuilder io.trino.spi.block.BlockBuilder.beginBlockEntry()'
[2023-10-07 20:17:26] java.lang.NoSuchMethodError: 'io.trino.spi.block.BlockBuilder io.trino.spi.block.BlockBuilder.beginBlockEntry()'

execute okay SELECT t.* FROM without_row t;

Anything else?

Full exception

2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.IClassLoader    About to call "findIClass({0})"
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.ClassLoaderIClassLoader clazz={0}
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.IClassLoader    {0}: Defined type "{0}"
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    {0}: {1} <=> {2}
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    Applicable!
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    {0}: {1} <=> {2}
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    Applicable!
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    {0}: {1} <=> {2}
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    Applicable!
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    maximallySpecificIInvocables={0}
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    maximallySpecificIInvocables={0}
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    maximallySpecificIInvocables={0}
2023-10-07T20:17:34.782+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.IClassLoader    About to call "findIClass({0})"
2023-10-07T20:17:34.783+0800    DEBUG   20231007_121734_00001_xafpd.1.0.0-0-179 org.apache.paimon.shade.org.codehaus.janino.UnitCompiler    Applicable!
2023-10-07T20:17:34.880+0800    ERROR   SplitRunner-68-179  io.trino.execution.executor.timesharing.TimeSharingTaskExecutor Error processing Split 20231007_121734_00001_xafpd.1.0.0-0 {} (start = 4.657444088895251E9, wall = 132 ms, cpu = 0 ms, wait = 0 ms, calls = 1)
java.lang.NoSuchMethodError: 'io.trino.spi.block.BlockBuilder io.trino.spi.block.BlockBuilder.beginBlockEntry()'
    at org.apache.paimon.trino.TrinoPageSourceBase.writeBlock(TrinoPageSourceBase.java:251)
    at org.apache.paimon.trino.TrinoPageSourceBase.appendTo(TrinoPageSourceBase.java:201)
    at org.apache.paimon.trino.TrinoPageSourceBase.nextPage(TrinoPageSourceBase.java:138)
    at org.apache.paimon.trino.TrinoPageSourceBase.lambda$getNextPage$0(TrinoPageSourceBase.java:119)
    at org.apache.paimon.trino.ClassLoaderUtils.runWithContextClassLoader(ClassLoaderUtils.java:30)
    at org.apache.paimon.trino.TrinoPageSourceBase.getNextPage(TrinoPageSourceBase.java:116)
    at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:299)
    at io.trino.operator.Driver.processInternal(Driver.java:395)
    at io.trino.operator.Driver.lambda$process$8(Driver.java:298)
    at io.trino.operator.Driver.tryWithLock(Driver.java:694)
    at io.trino.operator.Driver.process(Driver.java:290)
    at io.trino.operator.Driver.processForDuration(Driver.java:261)
    at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:887)
    at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
    at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
    at io.trino.$gen.Trino_428____20231007_121623_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)

2023-10-07T20:17:34.883+0800    ERROR   remote-task-callback-2  io.trino.execution.scheduler.PipelinedStageExecution    Pipelined stage execution for stage 20231007_121734_00001_xafpd.1 failed
java.lang.NoSuchMethodError: 'io.trino.spi.block.BlockBuilder io.trino.spi.block.BlockBuilder.beginBlockEntry()'
    at org.apache.paimon.trino.TrinoPageSourceBase.writeBlock(TrinoPageSourceBase.java:251)
    at org.apache.paimon.trino.TrinoPageSourceBase.appendTo(TrinoPageSourceBase.java:201)
    at org.apache.paimon.trino.TrinoPageSourceBase.nextPage(TrinoPageSourceBase.java:138)
    at org.apache.paimon.trino.TrinoPageSourceBase.lambda$getNextPage$0(TrinoPageSourceBase.java:119)
    at org.apache.paimon.trino.ClassLoaderUtils.runWithContextClassLoader(ClassLoaderUtils.java:30)
    at org.apache.paimon.trino.TrinoPageSourceBase.getNextPage(TrinoPageSourceBase.java:116)
    at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:299)
    at io.trino.operator.Driver.processInternal(Driver.java:395)
    at io.trino.operator.Driver.lambda$process$8(Driver.java:298)
    at io.trino.operator.Driver.tryWithLock(Driver.java:694)
    at io.trino.operator.Driver.process(Driver.java:290)
    at io.trino.operator.Driver.processForDuration(Driver.java:261)
    at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:887)
    at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
    at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
    at io.trino.$gen.Trino_428____20231007_121623_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)

2023-10-07T20:17:34.883+0800    ERROR   stage-scheduler io.trino.execution.scheduler.PipelinedQueryScheduler    Failure in distributed stage for query 20231007_121734_00001_xafpd
java.lang.NoSuchMethodError: 'io.trino.spi.block.BlockBuilder io.trino.spi.block.BlockBuilder.beginBlockEntry()'
    at org.apache.paimon.trino.TrinoPageSourceBase.writeBlock(TrinoPageSourceBase.java:251)
    at org.apache.paimon.trino.TrinoPageSourceBase.appendTo(TrinoPageSourceBase.java:201)
    at org.apache.paimon.trino.TrinoPageSourceBase.nextPage(TrinoPageSourceBase.java:138)
    at org.apache.paimon.trino.TrinoPageSourceBase.lambda$getNextPage$0(TrinoPageSourceBase.java:119)
    at org.apache.paimon.trino.ClassLoaderUtils.runWithContextClassLoader(ClassLoaderUtils.java:30)
    at org.apache.paimon.trino.TrinoPageSourceBase.getNextPage(TrinoPageSourceBase.java:116)
    at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:299)
    at io.trino.operator.Driver.processInternal(Driver.java:395)
    at io.trino.operator.Driver.lambda$process$8(Driver.java:298)
    at io.trino.operator.Driver.tryWithLock(Driver.java:694)
    at io.trino.operator.Driver.process(Driver.java:290)
    at io.trino.operator.Driver.processForDuration(Driver.java:261)
    at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:887)
    at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
    at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
    at io.trino.$gen.Trino_428____20231007_121623_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)

2023-10-07T20:17:34.885+0800    ERROR   stage-scheduler io.trino.execution.StageStateMachine    Stage 20231007_121734_00001_xafpd.1 failed
java.lang.NoSuchMethodError: 'io.trino.spi.block.BlockBuilder io.trino.spi.block.BlockBuilder.beginBlockEntry()'
    at org.apache.paimon.trino.TrinoPageSourceBase.writeBlock(TrinoPageSourceBase.java:251)
    at org.apache.paimon.trino.TrinoPageSourceBase.appendTo(TrinoPageSourceBase.java:201)
    at org.apache.paimon.trino.TrinoPageSourceBase.nextPage(TrinoPageSourceBase.java:138)
    at org.apache.paimon.trino.TrinoPageSourceBase.lambda$getNextPage$0(TrinoPageSourceBase.java:119)
    at org.apache.paimon.trino.ClassLoaderUtils.runWithContextClassLoader(ClassLoaderUtils.java:30)
    at org.apache.paimon.trino.TrinoPageSourceBase.getNextPage(TrinoPageSourceBase.java:116)
    at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:299)
    at io.trino.operator.Driver.processInternal(Driver.java:395)
    at io.trino.operator.Driver.lambda$process$8(Driver.java:298)
    at io.trino.operator.Driver.tryWithLock(Driver.java:694)
    at io.trino.operator.Driver.process(Driver.java:290)
    at io.trino.operator.Driver.processForDuration(Driver.java:261)
    at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:887)
    at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
    at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
    at io.trino.$gen.Trino_428____20231007_121623_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)

Are you willing to submit a PR?

macdoor commented 1 year ago

solved with paimon-trino-422-0.6-20231007.082435-32.jar