googleapis / java-bigquery

Apache License 2.0
109 stars 119 forks source link

BigQuery: Multiple queries with a single Connection #3337

Open filipegeric opened 4 months ago

filipegeric commented 4 months ago

I'm trying to run multiple queries using the same instance of Connection, but it fails with RejectedExecutionException.

Environment details

  1. API: BigQuery
  2. OS type and version: Linux Ubuntu 22.04
  3. Java version: 17
  4. version(s): google-cloud-bigquery:2.39.1

Steps to reproduce

  1. Run the code example

Code example

import com.google.cloud.bigquery.BigQueryOptions

fun main() {
    val count = 3
    println("Start")
    val connection = BigQueryOptions.getDefaultInstance().service.createConnection()

    repeat(count) {
        val result = connection.executeSelect("SELECT RAND() as random_value$it")
        val resultSet = result.resultSet
        while (resultSet.next()) {
            println(resultSet.getDouble(0))
        }
        println("Result for $it: $result")
    }
}

Output and stack trace

Start
Jun 04, 2024 4:28:30 PM com.google.cloud.bigquery.ConnectionImpl getExecuteSelectResponse
INFO: 
 Using Fast Query Path
0.4408623091866403
Result for 0: com.google.cloud.bigquery.BigQueryResultImpl@24ba9639
Jun 04, 2024 4:28:32 PM com.google.cloud.bigquery.ConnectionImpl getExecuteSelectResponse
INFO: 
 Using Fast Query Path
Exception in thread "main" java.util.concurrent.RejectedExecutionException: Task com.google.cloud.bigquery.ConnectionImpl$$Lambda$100/0x0000000801191b18@8c3619e rejected from java.util.concurrent.ThreadPoolExecutor@4331d187[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3]
    at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
    at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
    at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
    at com.google.cloud.bigquery.ConnectionImpl.runNextPageTaskAsync(ConnectionImpl.java:628)
    at com.google.cloud.bigquery.ConnectionImpl.processQueryResponseResults(ConnectionImpl.java:579)
    at com.google.cloud.bigquery.ConnectionImpl.queryRpc(ConnectionImpl.java:474)
    at com.google.cloud.bigquery.ConnectionImpl.getExecuteSelectResponse(ConnectionImpl.java:239)
    at com.google.cloud.bigquery.ConnectionImpl.executeSelect(ConnectionImpl.java:198)
    at org.example.MainKt.main(Main.kt:12)
    at org.example.MainKt.main(Main.kt)

External references such as API reference guides

Does this mean that for each query a new connection needs to be created?