fabric8io / kubernetes-client

Java client for Kubernetes & OpenShift
http://fabric8.io
Apache License 2.0
3.41k stars 1.46k forks source link

RejectedExecutionException executor rejected after upgrading from openshift-client 6.13.1 to either (6.13.2 and 6.13.3) #6278

Closed vicozohili closed 1 month ago

vicozohili commented 2 months ago

Describe the bug

Getting below exception since upgrade from openshift-client 6.13.1 to either (6.13.2 and 6.13.3)

io.fabric8.kubernetes.client.dsl.internal.ExecWebSocketListener - Exec Failure
java.util.concurrent.CompletionException: java.io.InterruptedIOException: executor rejected
    at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
    at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1177)
    at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
    at io.fabric8.kubernetes.client.http.StandardHttpClient.buildWebSocketOnce(StandardHttpClient.java:271)
    at io.fabric8.kubernetes.client.http.StandardHttpClient.lambda$buildWebSocket$13(StandardHttpClient.java:240)
    at io.fabric8.kubernetes.client.utils.AsyncUtils.retryWithExponentialBackoff(AsyncUtils.java:75)
    at io.fabric8.kubernetes.client.utils.AsyncUtils.lambda$null$2(AsyncUtils.java:83)
    at io.fabric8.kubernetes.client.utils.Utils.lambda$schedule$6(Utils.java:486)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.io.InterruptedIOException: executor rejected
    at okhttp3.RealCall$AsyncCall.executeOn(RealCall.java:186)
    at okhttp3.Dispatcher.promoteAndExecute(Dispatcher.java:186)
    at okhttp3.Dispatcher.enqueue(Dispatcher.java:137)
    at okhttp3.RealCall.enqueue(RealCall.java:127)
    at okhttp3.internal.ws.RealWebSocket.connect(RealWebSocket.java:193)
    at okhttp3.OkHttpClient.newWebSocket(OkHttpClient.java:435)
    at io.fabric8.kubernetes.client.okhttp.OkHttpWebSocketImpl.buildAsync(OkHttpWebSocketImpl.java:70)
    at io.fabric8.kubernetes.client.okhttp.OkHttpClientImpl.buildWebSocketDirect(OkHttpClientImpl.java:426)
    at io.fabric8.kubernetes.client.http.StandardHttpClient.buildWebSocketOnce(StandardHttpClient.java:266)
    ... 10 more
Caused by: java.util.concurrent.RejectedExecutionException: Task okhttp3.RealCall$AsyncCall@51f549e9 rejected from java.util.concurrent.ThreadPoolExecutor@2c94aba3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 12]
    at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
    at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
    at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
    at okhttp3.RealCall$AsyncCall.executeOn(RealCall.java:183)
    ... 18 more

Fabric8 Kubernetes Client version

6.13.3

Steps to reproduce

upgrade from openshift-client 6.13.1 to either (6.13.2 and 6.13.3)

Expected behavior

.

Runtime

OpenShift

Kubernetes API Server version

1.25.3@latest

Environment

Windows, Linux, Amazon

Fabric8 Kubernetes Client Logs

No response

Additional context

No response

rohanKanojia commented 2 months ago

@vicozohili : Could you please share your OpenShift version as well? How can we reproduce this issue? Is it an intermittent issue or is it happening all the time?

shawkins commented 2 months ago

Is this only happening when the client / websocket is closed?

vicozohili commented 1 month ago

finally got back to reviewing this issue and looks like corrupted dependencies cleaning dependency location and allowing IDE to pull them down again seems to have resolved the issue. closing