search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.51k stars 3.03k forks source link

Throw TrinoException when Iceberg commit fails #24042

Closed pajaks closed 2 weeks ago

pajaks commented 2 weeks ago

Description

In case Iceberg commit fails throw TrinoException with clear error type.

Additional context and related issues

Example stack trace:

org.apache.iceberg.exceptions.CommitFailedException: Commit failed: Failed to commit to table ...
org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:89)
org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:80)
org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:211)
org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:323)
org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:262)
org.apache.iceberg.rest.HTTPClient.post(HTTPClient.java:368)
org.apache.iceberg.rest.RESTClient.post(RESTClient.java:112)
org.apache.iceberg.rest.RESTTableOperations.commit(RESTTableOperations.java:152)
org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$3(BaseTransaction.java:427)
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:423)
org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:318)
io.trino.plugin.iceberg.IcebergMetadata.commitTransaction(IcebergMetadata.java:1850)
io.trino.plugin.iceberg.IcebergMetadata.commitUpdateAndTransaction(IcebergMetadata.java:1840)
io.trino.plugin.iceberg.IcebergMetadata.finishInsert(IcebergMetadata.java:1323)

and

org.apache.iceberg.exceptions.CommitStateUnknownException: Service failed: 500: Invalid snapshot with sequence number 73 greater than last sequence number 72
Cannot determine whether the commit was successful or not, the underlying data files may or may not be needed. Manual intervention via the Remove Orphan Files Action can remove these files when a connection to the Catalog can be re-established if the commit was actually unsuccessful.
Please check to see whether or not your commit was successful before retrying this commit. Retrying an already successful operation will result in duplicate records or unintentional modifications.
At this time no files will be deleted including possibly unused manifest lists.
org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:94)
org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:80)
org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:211)
org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:323)
org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:262)
org.apache.iceberg.rest.HTTPClient.post(HTTPClient.java:368)
org.apache.iceberg.rest.RESTClient.post(RESTClient.java:112)
org.apache.iceberg.rest.RESTTableOperations.commit(RESTTableOperations.java:152)
org.apache.iceberg.BaseTransaction.lambda$commitReplaceTransaction$1(BaseTransaction.java:381)
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
org.apache.iceberg.BaseTransaction.commitReplaceTransaction(BaseTransaction.java:365)
org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:310)
io.trino.plugin.iceberg.IcebergMetadata.commitTransaction(IcebergMetadata.java:1850)
io.trino.plugin.iceberg.IcebergMetadata.commitUpdateAndTransaction(IcebergMetadata.java:1840)
io.trino.plugin.iceberg.IcebergMetadata.finishCreateTable(IcebergMetadata.java:1124)
io.trino.plugin.iceberg.IcebergMetadata.createTable(IcebergMetadata.java:991)

Release notes

(x) This is not user-visible or is docs only, and no release notes are required. ( ) Release notes are required. Please propose a release note for me. ( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)