trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.49k stars 3.02k forks source link

Reduce repeated operations for a table in testCommentTable #24070

Closed ebyhr closed 2 weeks ago

ebyhr commented 2 weeks ago

Description

Hopefully, this fixes issue in BigQuery connector:

Error:  io.trino.plugin.bigquery.TestBigQueryAvroConnectorTest.testCommentTable -- Time elapsed: 8.318 s <<< ERROR!
io.trino.testing.QueryFailedException: Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas
    at io.trino.testing.AbstractTestingTrinoClient.execute(AbstractTestingTrinoClient.java:138)
    at io.trino.testing.DistributedQueryRunner.executeInternal(DistributedQueryRunner.java:565)
    at io.trino.testing.DistributedQueryRunner.execute(DistributedQueryRunner.java:548)
    at io.trino.testing.QueryRunner.execute(QueryRunner.java:82)
    at io.trino.testing.sql.TestTable.close(TestTable.java:134)
    at io.trino.testing.BaseConnectorTest.testCommentTable(BaseConnectorTest.java:4158)
    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1458)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2034)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:189)
    Suppressed: java.lang.Exception: SQL: DROP TABLE test_comment_illsqwpwl1
        at io.trino.testing.DistributedQueryRunner.executeInternal(DistributedQueryRunner.java:572)
        ... 9 more
Caused by: com.google.cloud.bigquery.BigQueryException: Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas
    at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:116)
    at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.deleteTable(HttpBigQueryRpc.java:383)
    at com.google.cloud.bigquery.BigQueryImpl$10.call(BigQueryImpl.java:601)
    at com.google.cloud.bigquery.BigQueryImpl$10.call(BigQueryImpl.java:598)
    at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:102)
    at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
    at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
    at com.google.cloud.bigquery.BigQueryImpl.delete(BigQueryImpl.java:597)
    at io.trino.plugin.bigquery.BigQueryClient.dropTable(BigQueryClient.java:405)
    at io.trino.plugin.bigquery.BigQueryMetadata.dropTable(BigQueryMetadata.java:642)
    at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.dropTable(ClassLoaderSafeConnectorMetadata.java:459)
    at io.trino.tracing.TracingConnectorMetadata.dropTable(TracingConnectorMetadata.java:396)
    at io.trino.metadata.MetadataManager.dropTable(MetadataManager.java:1053)
    at io.trino.tracing.TracingMetadata.dropTable(TracingMetadata.java:563)
    at io.trino.execution.DropTableTask.execute(DropTableTask.java:88)
    at io.trino.execution.DropTableTask.execute(DropTableTask.java:36)
    at io.trino.execution.DataDefinitionExecution.start(DataDefinitionExecution.java:146)
    at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:272)
    at io.trino.dispatcher.LocalDispatchQuery.startExecution(LocalDispatchQuery.java:150)
    at io.trino.dispatcher.LocalDispatchQuery.lambda$waitForMinimumWorkers$2(LocalDispatchQuery.java:134)
    at io.airlift.concurrent.MoreFutures.lambda$addSuccessCallback$12(MoreFutures.java:570)
    at io.airlift.concurrent.MoreFutures$3.onSuccess(MoreFutures.java:545)
    at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1137)
    at io.trino.$gen.Trino_testversion____20241108_042128_141.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
DELETE https://bigquery.googleapis.com/bigquery/v2/projects/sep-bq-cicd/datasets/tpch/tables/test_comment_illsqwpwl1
{
  "code": 403,
  "errors": [
    {
      "domain": "usageLimits",
      "message": "Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas",
      "reason": "rateLimitExceeded"
    }
  ],
  "message": "Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas",
  "status": "PERMISSION_DENIED"
}
    at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:479)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:565)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:506)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:616)
    at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.deleteTable(HttpBigQueryRpc.java:380)
    ... 25 more

https://github.com/trinodb/trino/actions/runs/11735989749/job/32694540331?pr=24068

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.