search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.46k stars 3.01k forks source link

Flaky test TestIcebergSparkCompatibility with REST catalog #19189

Open dominikzalewski opened 1 year ago

dominikzalewski commented 1 year ago

https://github.com/trinodb/trino/actions/runs/6326824172

@ebyhr' s advice:

We could add @Flaky annotation with https://github.com/trinodb/trino/blob/master/testing/trino-product-tests/src/main/java/io/trino/tests/product/utils/HadoopTestUtils.java Another approach is using MinIO instead of HDFS in Iceberg REST catalog product tests. (edited)

Logs of failure below:

2023-09-27 20:27:26 INFO: FAILURE     /    io.trino.tests.product.iceberg.TestIcebergSparkCompatibility.testHandlingPartitionSchemaEvolutionInPartitionMetadata (Groups: iceberg_jdbc, profile_specific_tests, iceberg, iceberg_rest) took 5.3 seconds
2023-09-27 20:27:26 SEVERE: Failure cause:
io.trino.tempto.query.QueryExecutionException: java.sql.SQLException: Query failed (#20230927_144225_00164_re69f): Malformed request: Self-suppression not permitted
    at io.trino.tempto.query.JdbcQueryExecutor.execute(JdbcQueryExecutor.java:119)
    at io.trino.tempto.query.JdbcQueryExecutor.executeQuery(JdbcQueryExecutor.java:84)
    at io.trino.tests.product.utils.QueryExecutors$1.lambda$executeQuery$0(QueryExecutors.java:55)
    at dev.failsafe.Functions.lambda$toCtxSupplier$11(Functions.java:243)
    at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
    at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)
    at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)
    at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)
    at dev.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:112)
    at io.trino.tests.product.utils.QueryExecutors$1.executeQuery(QueryExecutors.java:55)
    at io.trino.tests.product.iceberg.TestIcebergSparkCompatibility.testHandlingPartitionSchemaEvolutionInPartitionMetadata(TestIcebergSparkCompatibility.java:2370)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.sql.SQLException: Query failed (#20230927_144225_00164_re69f): Malformed request: Self-suppression not permitted
    at io.trino.jdbc.AbstractTrinoResultSet.resultsException(AbstractTrinoResultSet.java:1937)
    at io.trino.jdbc.TrinoResultSet$ResultsPageIterator.computeNext(TrinoResultSet.java:294)
    at io.trino.jdbc.TrinoResultSet$ResultsPageIterator.computeNext(TrinoResultSet.java:254)
    at io.trino.jdbc.$internal.guava.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
    at io.trino.jdbc.$internal.guava.collect.AbstractIterator.hasNext(AbstractIterator.java:140)
    at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1855)
    at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:292)
    at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
    at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
    at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:298)
    at java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
    at io.trino.jdbc.TrinoResultSet$AsyncIterator.lambda$new$1(TrinoResultSet.java:179)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    ... 3 more
    Suppressed: java.lang.Exception: Query: INSERT INTO iceberg.default.test_handling_partition_schema_evolution_zju8c5cngk VALUES (5, 50, date '2022-08-15')
        at io.trino.tempto.query.JdbcQueryExecutor.executeQueryNoParams(JdbcQueryExecutor.java:136)
        at io.trino.tempto.query.JdbcQueryExecutor.execute(JdbcQueryExecutor.java:112)
        at io.trino.tempto.query.JdbcQueryExecutor.executeQuery(JdbcQueryExecutor.java:84)
        at io.trino.tests.product.utils.QueryExecutors$1.lambda$executeQuery$0(QueryExecutors.java:55)
        at dev.failsafe.Functions.lambda$toCtxSupplier$11(Functions.java:243)
        at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
        at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)
        at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)
        at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)
        at dev.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:112)
        at io.trino.tests.product.utils.QueryExecutors$1.executeQuery(QueryExecutors.java:55)
        at io.trino.tests.product.iceberg.TestIcebergSparkCompatibility.testHandlingPartitionSchemaEvolutionInPartitionMetadata(TestIcebergSparkCompatibility.java:2370)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
        at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
        at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
        at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
        at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
        at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
        ... 3 more
Caused by: org.apache.iceberg.exceptions.BadRequestException: Malformed request: Self-suppression not permitted
    at org.apache.iceberg.rest.ErrorHandlers$DefaultErrorHandler.accept(ErrorHandlers.java:153)
    at org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:88)
    at org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:71)
    at org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:176)
    at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:285)
    at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:219)
    at org.apache.iceberg.rest.HTTPClient.post(HTTPClient.java:330)
    at org.apache.iceberg.rest.RESTClient.post(RESTClient.java:112)
    at org.apache.iceberg.rest.RESTTableOperations.commit(RESTTableOperations.java:144)
    at org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$5(BaseTransaction.java:422)
    at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
    at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
    at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
    at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
    at org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:418)
    at org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:302)
    at io.trino.plugin.iceberg.IcebergMetadata.finishInsert(IcebergMetadata.java:1008)
    at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.finishInsert(ClassLoaderSafeConnectorMetadata.java:590)
    at io.trino.tracing.TracingConnectorMetadata.finishInsert(TracingConnectorMetadata.java:668)
    at io.trino.metadata.MetadataManager.finishInsert(MetadataManager.java:1109)
    at io.trino.tracing.TracingMetadata.finishInsert(TracingMetadata.java:670)
    at io.trino.sql.planner.LocalExecutionPlanner.lambda$createTableFinisher$4(LocalExecutionPlanner.java:4200)
    at io.trino.operator.TableFinishOperator.getOutput(TableFinishOperator.java:319)
    at io.trino.operator.Driver.processInternal(Driver.java:395)
    at io.trino.operator.Driver.lambda$process$8(Driver.java:298)
    at io.trino.operator.Driver.tryWithLock(Driver.java:694)
    at io.trino.operator.Driver.process(Driver.java:290)
    at io.trino.operator.Driver.processForDuration(Driver.java:261)
    at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:887)
    at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
    at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
    at io.trino.$gen.Trino_427_18_g8cad909____20230927_143832_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
hashhar commented 1 year ago

@dominikzalewski Please include full stack trace instead of truncated one. The logs from the link of the run will get removed in 90 days IIRC (GitHub only retains CI logs for limited time).

dominikzalewski commented 1 year ago

7_Product Tests.zip

ebyhr commented 1 year ago

testSparkReadingNestedTrinoData also failed in https://github.com/trinodb/trino/actions/runs/6539051188/job/17757387372

I changed the issue title because all tests depending on HDFS in TestIcebergSparkCompatibility may face the flaky issue.

findepi commented 1 year ago

Malformed request: Self-suppression not permitted is something to be fixed on REST Catalog's server side