search

apache / shardingsphere

Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
Apache License 2.0
19.81k stars 6.7k forks source link

Integration tests occasionally stuck in waiting for container ready #19648

Closed TeslaCN closed 1 year ago

TeslaCN commented 2 years ago

Bug Report

Which version of ShardingSphere did you use?

master - 9284c595a727ca70b845ad7ec4792cc549db7fa7

ShardingSphere-Proxy integration tests

Maybe this is similar to

Environment

~ ➤ docker info                                                                                                                                                                                                                                                                          
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 16
  Running: 4
  Paused: 0
  Stopped: 12
 Images: 160
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.15.0-41-generic
 Operating System: Ubuntu 20.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.25GiB
 Name: wuweijie-ubuntu
 ID: NTWV:UXYU:53NR:XS22:OJ7L:BVPD:V4KW:ZUEH:6RJQ:7BNV:7KGN:VZ5Y
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: shardingsphere
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://gbkpkqgm.mirror.aliyuncs.com/
 Live Restore Enabled: false

Expected behavior

Tests could be finished.

Actual behavior

Container ZooKeeper has readied for tens of minutes.

image

image

Logs of maven:

[INFO ] 2022-07-28 16:58:02.510 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,510 [myid:1] - INFO  [main:ZKDatabase@152] - zookeeper.commitLogCount=500
[INFO ] 2022-07-28 16:58:02.514 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,514 [myid:1] - INFO  [main:SnapStream@61] - zookeeper.snapshot.compression.method = CHECKED
[INFO ] 2022-07-28 16:58:02.514 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,514 [myid:1] - INFO  [main:FileTxnSnapLog@470] - Snapshotting: 0x0 to /data/version-2/snapshot.0
[INFO ] 2022-07-28 16:58:02.516 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,516 [myid:1] - INFO  [main:ZKDatabase@289] - Snapshot loaded in 6 ms, highest zxid is 0x0, digest is 1371985504
[INFO ] 2022-07-28 16:58:02.516 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,516 [myid:1] - INFO  [main:FileTxnSnapLog@470] - Snapshotting: 0x0 to /data/version-2/snapshot.0
[INFO ] 2022-07-28 16:58:02.517 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,516 [myid:1] - INFO  [main:ZooKeeperServer@529] - Snapshot taken in 0 ms
[INFO ] 2022-07-28 16:58:02.522 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,522 [myid:1] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@136] - PrepRequestProcessor (sid:0) started, reconfigEnabled=false
[INFO ] 2022-07-28 16:58:02.522 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,522 [myid:1] - INFO  [main:RequestThrottler@74] - zookeeper.request_throttler.shutdownTimeout = 10000
[INFO ] 2022-07-28 16:58:02.531 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,530 [myid:1] - INFO  [main:ContainerManager@83] - Using checkIntervalMs=60000 maxPerMinute=10000 maxNeverUsedIntervalMs=0
[INFO ] 2022-07-28 16:58:02.531 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 08:58:02,531 [myid:1] - INFO  [main:ZKAuditProvider@42] - ZooKeeper audit is disabled.
[INFO ] 2022-07-28 17:18:54.509 [docker-java-stream-2041008728] dbtbl_with_readwrite_splitting:zookeeper - STDOUT: 2022-07-28 09:18:54,508 [myid:1] - INFO  [SyncThread:0:FileTxnLog@284] - Creating new log file: log.1

But the surefire process was still waiting for ready:

{} ~ jstack 2290346
2022-07-28 17:17:22
Full thread dump Java HotSpot(TM) 64-Bit Server VM (17.0.1+12-LTS-39 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007fec54001f60, length=18, elements={
0x00007feca80259b0, 0x00007feca8164e40, 0x00007feca8166220, 0x00007feca816b580,
0x00007feca816c930, 0x00007feca816dd40, 0x00007feca816f770, 0x00007feca8170ca0,
0x00007feca8172110, 0x00007feca817d960, 0x00007feca8c02320, 0x00007feca8c77c40,
0x00007feca8cdfc00, 0x00007febec016eb0, 0x00007febdc089ad0, 0x00007feca92711d0,
0x00007feca927f250, 0x00007fec54001550
}

"main" #1 prio=5 os_prio=0 cpu=1161262.57ms elapsed=1164.15s tid=0x00007feca80259b0 nid=0x22f2ae runnable  [0x00007fecadfb5000]
   java.lang.Thread.State: RUNNABLE
    at org.testcontainers.shaded.okio.Buffer.getByte(Buffer.java:312)
    at org.testcontainers.shaded.okio.RealBufferedSource.readHexadecimalUnsignedLong(RealBufferedSource.java:310)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.readChunkSize(Http1ExchangeCodec.java:492)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.java:471)
    at org.testcontainers.shaded.okhttp3.internal.Util.skipAll(Util.java:204)
    at org.testcontainers.shaded.okhttp3.internal.Util.discard(Util.java:186)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.close(Http1ExchangeCodec.java:511)
    at org.testcontainers.shaded.okio.ForwardingSource.close(ForwardingSource.java:43)
    at org.testcontainers.shaded.okhttp3.internal.connection.Exchange$ResponseBodySource.close(Exchange.java:313)
    at org.testcontainers.shaded.okio.RealBufferedSource.close(RealBufferedSource.java:476)
    at org.testcontainers.shaded.okhttp3.internal.Util.closeQuietly(Util.java:139)
    at org.testcontainers.shaded.okhttp3.ResponseBody.close(ResponseBody.java:192)
    at org.testcontainers.shaded.okhttp3.Response.close(Response.java:290)
    at org.testcontainers.shaded.com.github.dockerjava.okhttp.OkDockerHttpClient$OkResponse.close(OkDockerHttpClient.java:285)
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$null$0(DefaultInvocationBuilder.java:272)
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder$$Lambda$178/0x0000000800ec12b0.close(Unknown Source)
    at com.github.dockerjava.api.async.ResultCallbackTemplate.close(ResultCallbackTemplate.java:77)
    at org.testcontainers.containers.output.FrameConsumerResultCallback.close(FrameConsumerResultCallback.java:100)
    at org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy.waitUntilReady(LogMessageWaitStrategy.java:51)
    at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:35)
    at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:892)
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:440)
    at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:325)
    at org.testcontainers.containers.GenericContainer$$Lambda$194/0x0000000800ece220.call(Unknown Source)
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:323)
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:311)
    at org.apache.shardingsphere.test.integration.env.container.atomic.DockerITContainer.start(DockerITContainer.java:49)
    at org.apache.shardingsphere.test.integration.env.container.atomic.ITContainers$$Lambda$95/0x0000000800d9ac08.accept(Unknown Source)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(java.base@17.0.1/ForEachOps.java:183)
    at java.util.stream.ReferencePipeline$2$1.accept(java.base@17.0.1/ReferencePipeline.java:179)
    at java.util.LinkedList$LLSpliterator.forEachRemaining(java.base@17.0.1/LinkedList.java:1242)
    at java.util.stream.AbstractPipeline.copyInto(java.base@17.0.1/AbstractPipeline.java:509)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@17.0.1/AbstractPipeline.java:499)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@17.0.1/ForEachOps.java:150)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.{} ~ jstack 2290346
2022-07-28 17:17:22
Full thread dump Java HotSpot(TM) 64-Bit Server VM (17.0.1+12-LTS-39 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007fec54001f60, length=18, elements={
0x00007feca80259b0, 0x00007feca8164e40, 0x00007feca8166220, 0x00007feca816b580,
0x00007feca816c930, 0x00007feca816dd40, 0x00007feca816f770, 0x00007feca8170ca0,
0x00007feca8172110, 0x00007feca817d960, 0x00007feca8c02320, 0x00007feca8c77c40,
0x00007feca8cdfc00, 0x00007febec016eb0, 0x00007febdc089ad0, 0x00007feca92711d0,
0x00007feca927f250, 0x00007fec54001550
}

"main" #1 prio=5 os_prio=0 cpu=1161262.57ms elapsed=1164.15s tid=0x00007feca80259b0 nid=0x22f2ae runnable  [0x00007fecadfb5000]
   java.lang.Thread.State: RUNNABLE
    at org.testcontainers.shaded.okio.Buffer.getByte(Buffer.java:312)
    at org.testcontainers.shaded.okio.RealBufferedSource.readHexadecimalUnsignedLong(RealBufferedSource.java:310)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.readChunkSize(Http1ExchangeCodec.java:492)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.java:471)
    at org.testcontainers.shaded.okhttp3.internal.Util.skipAll(Util.java:204)
    at org.testcontainers.shaded.okhttp3.internal.Util.discard(Util.java:186)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.close(Http1ExchangeCodec.java:511)
    at org.testcontainers.shaded.okio.ForwardingSource.close(ForwardingSource.java:43)
    at org.testcontainers.shaded.okhttp3.internal.connection.Exchange$ResponseBodySource.close(Exchange.java:313)
    at org.testcontainers.shaded.okio.RealBufferedSource.close(RealBufferedSource.java:476)
    at org.testcontainers.shaded.okhttp3.internal.Util.closeQuietly(Util.java:139)
    at org.testcontainers.shaded.okhttp3.ResponseBody.close(ResponseBody.java:192)
    at org.testcontainers.shaded.okhttp3.Response.close(Response.java:290)
    at org.testcontainers.shaded.com.github.dockerjava.okhttp.OkDockerHttpClient$OkResponse.close(OkDockerHttpClient.java:285)
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$null$0(DefaultInvocationBuilder.java:272)
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder$$Lambda$178/0x0000000800ec12b0.close(Unknown Source)
    at com.github.dockerjava.api.async.ResultCallbackTemplate.close(ResultCallbackTemplate.java:77)
    at org.testcontainers.containers.output.FrameConsumerResultCallback.close(FrameConsumerResultCallback.java:100)
    at org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy.waitUntilReady(LogMessageWaitStrategy.java:51)
    at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:35)
    at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:892)
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:440)
    at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:325)
    at org.testcontainers.containers.GenericContainer$$Lambda$194/0x0000000800ece220.call(Unknown Source)
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:323)
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:311)
    at org.apache.shardingsphere.test.integration.env.container.atomic.DockerITContainer.start(DockerITContainer.java:49)
    at org.apache.shardingsphere.test.integration.env.container.atomic.ITContainers$$Lambda$95/0x0000000800d9ac08.accept(Unknown Source)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(java.base@17.0.1/ForEachOps.java:183)
    at java.util.stream.ReferencePipeline$2$1.accept(java.base@17.0.1/ReferencePipeline.java:179)
    at java.util.LinkedList$LLSpliterator.forEachRemaining(java.base@17.0.1/LinkedList.java:1242)
    at java.util.stream.AbstractPipeline.copyInto(java.base@17.0.1/AbstractPipeline.java:509)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@17.0.1/AbstractPipeline.java:499)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@17.0.1/ForEachOps.java:150)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(java.base@17.0.1/ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate(java.base@17.0.1/AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(java.base@17.0.1/ReferencePipeline.java:596)
    at org.apache.shardingsphere.test.integration.env.container.atomic.ITContainers.start(ITContainers.java:82)
    - locked <0x000000061cefdc20> (a org.apache.shardingsphere.test.integration.env.container.atomic.ITContainers)
    at org.apache.shardingsphere.test.integration.container.compose.mode.ClusterComposedContainer.start(ClusterComposedContainer.java:64)
    at org.apache.shardingsphere.test.integration.engine.BaseITCase.setUp(BaseITCase.java:78)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@17.0.1/Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@17.0.1/NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17.0.1/DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(java.base@17.0.1/Method.java:568)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
    at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runners.Suite.runChild(Suite.java:128)
    at org.junit.runners.Suite.runChild(Suite.java:27)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0.73ms elapsed=1164.12s tid=0x00007feca8164e40 nid=0x22f2b5 waiting on condition  [0x00007fec71142000]
   java.lang.Thread.State: RUNNABLE
    at java.lang.ref.Reference.waitForReferencePendingList(java.base@17.0.1/Native Method)
    at java.lang.ref.Reference.processPendingReferences(java.base@17.0.1/Reference.java:253)
    at java.lang.ref.Reference$ReferenceHandler.run(java.base@17.0.1/Reference.java:215)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=0.53ms elapsed=1164.12s tid=0x00007feca8166220 nid=0x22f2b6 in Object.wait()  [0x00007fec707fe000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(java.base@17.0.1/Native Method)
    - waiting on <no object reference available>
    at java.lang.ref.ReferenceQueue.remove(java.base@17.0.1/ReferenceQueue.java:155)
    - locked <0x000000061c5b4578> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(java.base@17.0.1/ReferenceQueue.java:176)
    at java.lang.ref.Finalizer$FinalizerThread.run(java.base@17.0.1/Finalizer.java:172)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.22ms elapsed=1164.12s tid=0x00007feca816b580 nid=0x22f2b7 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=0.57ms elapsed=1164.12s tid=0x00007feca816c930 nid=0x22f2b8 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Monitor Deflation Thread" #6 daemon prio=9 os_prio=0 cpu=13.62ms elapsed=1164.12s tid=0x00007feca816dd40 nid=0x22f2b9 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=1485.55ms elapsed=1164.12s tid=0x00007feca816f770 nid=0x22f2ba waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #15 daemon prio=9 os_prio=0 cpu=489.43ms elapsed=1164.12s tid=0x00007feca8170ca0 nid=0x22f2bb waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #19 daemon prio=9 os_prio=0 cpu=16.61ms elapsed=1164.12s tid=0x00007feca8172110 nid=0x22f2bc runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #20 daemon prio=8 os_prio=0 cpu=1.42ms elapsed=1164.11s tid=0x00007feca817d960 nid=0x22f2bd in Object.wait()  [0x00007fec5b78b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(java.base@17.0.1/Native Method)
    - waiting on <no object reference available>
    at java.lang.ref.ReferenceQueue.remove(java.base@17.0.1/ReferenceQueue.java:155)
    - locked <0x000000061c5c00f8> (a java.lang.ref.ReferenceQueue$Lock)
    at jdk.internal.ref.CleanerImpl.run(java.base@17.0.1/CleanerImpl.java:140)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)
    at jdk.internal.misc.InnocuousThread.run(java.base@17.0.1/InnocuousThread.java:162)

"Notification Thread" #22 daemon prio=9 os_prio=0 cpu=0.06ms elapsed=1163.86s tid=0x00007feca8c02320 nid=0x22f2cb runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"surefire-forkedjvm-command-thread" #23 daemon prio=5 os_prio=0 cpu=14.96ms elapsed=1163.80s tid=0x00007feca8c77c40 nid=0x22f2cf runnable  [0x00007fec5a484000]
   java.lang.Thread.State: RUNNABLE
    at java.io.FileInputStream.readBytes(java.base@17.0.1/Native Method)
    at java.io.FileInputStream.read(java.base@17.0.1/FileInputStream.java:276)
    at java.io.BufferedInputStream.fill(java.base@17.0.1/BufferedInputStream.java:244)
    at java.io.BufferedInputStream.read(java.base@17.0.1/BufferedInputStream.java:263)
    - locked <0x000000061c5b9378> (a java.io.BufferedInputStream)
    at java.io.DataInputStream.readInt(java.base@17.0.1/DataInputStream.java:393)
    at org.apache.maven.surefire.booter.MasterProcessCommand.decode(MasterProcessCommand.java:115)
    at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:391)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"surefire-forkedjvm-ping-30s" #24 daemon prio=5 os_prio=0 cpu=746.88ms elapsed=1163.76s tid=0x00007feca8cdfc00 nid=0x22f2d0 waiting on condition  [0x00007fec5a383000]
   java.lang.Thread.State: TIMED_WAITING (parking)
    at jdk.internal.misc.Unsafe.park(java.base@17.0.1/Native Method)
    - parking to wait for  <0x000000061cc01470> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(java.base@17.0.1/LockSupport.java:252)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@17.0.1/AbstractQueuedSynchronizer.java:1672)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@17.0.1/ScheduledThreadPoolExecutor.java:1182)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@17.0.1/ScheduledThreadPoolExecutor.java:899)
    at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@17.0.1/ThreadPoolExecutor.java:1062)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.1/ThreadPoolExecutor.java:1122)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.1/ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"process reaper" #25 daemon prio=10 os_prio=0 cpu=271.39ms elapsed=1163.74s tid=0x00007febec016eb0 nid=0x22f2d3 waiting on condition  [0x00007fec700f7000]
   java.lang.Thread.State: TIMED_WAITING (parking)
    at jdk.internal.misc.Unsafe.park(java.base@17.0.1/Native Method)
    - parking to wait for  <0x000000061cc02490> (a java.util.concurrent.SynchronousQueue$TransferStack)
    at java.util.concurrent.locks.LockSupport.parkNanos(java.base@17.0.1/LockSupport.java:252)
    at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@17.0.1/SynchronousQueue.java:401)
    at java.util.concurrent.SynchronousQueue.poll(java.base@17.0.1/SynchronousQueue.java:903)
    at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@17.0.1/ThreadPoolExecutor.java:1061)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.1/ThreadPoolExecutor.java:1122)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.1/ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"OkHttp ConnectionPool" #28 daemon prio=5 os_prio=0 cpu=34.74ms elapsed=1162.53s tid=0x00007febdc089ad0 nid=0x22f2db in Object.wait()  [0x00007fec58be2000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(java.base@17.0.1/Native Method)
    - waiting on <no object reference available>
    at java.lang.Object.wait(java.base@17.0.1/Object.java:472)
    at org.testcontainers.shaded.okhttp3.internal.connection.RealConnectionPool.lambda$new$0(RealConnectionPool.java:62)
    - locked <0x000000061cd9a6d0> (a org.testcontainers.shaded.okhttp3.internal.connection.RealConnectionPool)
    at org.testcontainers.shaded.okhttp3.internal.connection.RealConnectionPool$$Lambda$129/0x0000000800db4438.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.1/ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.1/ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"testcontainers-ryuk" #31 daemon prio=5 os_prio=0 cpu=25.04ms elapsed=1161.40s tid=0x00007feca92711d0 nid=0x22f3f0 in Object.wait()  [0x00007fec5a585000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(java.base@17.0.1/Native Method)
    - waiting on <no object reference available>
    at org.testcontainers.utility.ResourceReaper.lambda$null$1(ResourceReaper.java:170)
    - locked <0x000000061e4602e0> (a java.util.ArrayList)
    at org.testcontainers.utility.ResourceReaper$$Lambda$188/0x0000000800ec8228.run(Unknown Source)
    at org.rnorth.ducttape.ratelimits.RateLimiter.doWhenReady(RateLimiter.java:27)
    at org.testcontainers.utility.ResourceReaper.lambda$start$2(ResourceReaper.java:159)
    at org.testcontainers.utility.ResourceReaper$$Lambda$187/0x0000000800ec8000.run(Unknown Source)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"docker-java-stream-2041008728" #33 daemon prio=5 os_prio=0 cpu=23.42ms elapsed=1160.63s tid=0x00007feca927f250 nid=0x22f564 runnable  [0x00007fec5b488000]
   java.lang.Thread.State: RUNNABLE
    at org.testcontainers.shaded.com.github.dockerjava.okhttp.UnixDomainSocket.recv(Native Method)
    at org.testcontainers.shaded.com.github.dockerjava.okhttp.UnixDomainSocket$UnixSocketInputStream.read(UnixDomainSocket.java:246)
    at java.io.FilterInputStream.read(java.base@17.0.1/FilterInputStream.java:132)
    at org.testcontainers.shaded.com.github.dockerjava.okhttp.UnixSocketFactory$1$1.read(UnixSocketFactory.java:45)
    at org.testcontainers.shaded.okio.Okio$2.read(Okio.java:140)
    at org.testcontainers.shaded.okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
    at org.testcontainers.shaded.okio.RealBufferedSource.request(RealBufferedSource.java:72)
    at org.testcontainers.shaded.okio.RealBufferedSource.require(RealBufferedSource.java:65)
    at org.testcontainers.shaded.okio.RealBufferedSource.readHexadecimalUnsignedLong(RealBufferedSource.java:307)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.readChunkSize(Http1ExchangeCodec.java:492)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.java:471)
    at org.testcontainers.shaded.okhttp3.internal.connection.Exchange$ResponseBodySource.read(Exchange.java:286)
    at org.testcontainers.shaded.okio.RealBufferedSource$1.read(RealBufferedSource.java:436)
    at org.testcontainers.shaded.com.github.dockerjava.core.FramedInputStreamConsumer.accept(FramedInputStreamConsumer.java:30)
    at org.testcontainers.shaded.com.github.dockerjava.core.FramedInputStreamConsumer.accept(FramedInputStreamConsumer.java:12)
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:275)
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder$$Lambda$177/0x0000000800ec0558.run(Unknown Source)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"Attach Listener" #35 daemon prio=9 os_prio=0 cpu=0.60ms elapsed=693.45s tid=0x00007fec54001550 nid=0x22fa58 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=130.82ms elapsed=1164.12s tid=0x00007feca8160df0 nid=0x22f2b4 runnable  

"GC Thread#0" os_prio=0 cpu=30.00ms elapsed=1164.13s tid=0x00007feca805adb0 nid=0x22f2af runnable  

"GC Thread#1" os_prio=0 cpu=27.36ms elapsed=1163.95s tid=0x00007fec680068c0 nid=0x22f2c1 runnable  

"GC Thread#2" os_prio=0 cpu=30.48ms elapsed=1163.95s tid=0x00007fec680072f0 nid=0x22f2c2 runnable  

"GC Thread#3" os_prio=0 cpu=25.00ms elapsed=1163.95s tid=0x00007fec68007d20 nid=0x22f2c3 runnable  

"GC Thread#4" os_prio=0 cpu=32.47ms elapsed=1163.95s tid=0x00007fec68008b40 nid=0x22f2c4 runnable  

"GC Thread#5" os_prio=0 cpu=31.29ms elapsed=1163.95s tid=0x00007fec68009980 nid=0x22f2c5 runnable  

"GC Thread#6" os_prio=0 cpu=28.47ms elapsed=1163.95s tid=0x00007fec6800a7e0 nid=0x22f2c6 runnable  

"GC Thread#7" os_prio=0 cpu=34.28ms elapsed=1163.95s tid=0x00007fec6800b640 nid=0x22f2c7 runnable  

"GC Thread#8" os_prio=0 cpu=35.57ms elapsed=1163.95s tid=0x00007fec6800c4a0 nid=0x22f2c8 runnable  

"GC Thread#9" os_prio=0 cpu=34.38ms elapsed=1163.95s tid=0x00007fec6800d300 nid=0x22f2c9 runnable  

"GC Thread#10" os_prio=0 cpu=30.56ms elapsed=1163.95s tid=0x00007fec6800e160 nid=0x22f2ca runnable  

"GC Thread#11" os_prio=0 cpu=30.47ms elapsed=1162.22s tid=0x00007fec680186d0 nid=0x22f30b runnable  

"GC Thread#12" os_prio=0 cpu=25.04ms elapsed=1162.22s tid=0x00007fec68018d80 nid=0x22f30c runnable  

"G1 Main Marker" os_prio=0 cpu=0.41ms elapsed=1164.13s tid=0x00007feca806b5d0 nid=0x22f2b0 runnable  

"G1 Conc#0" os_prio=0 cpu=3.12ms elapsed=1164.13s tid=0x00007feca806c530 nid=0x22f2b1 runnable  

"G1 Conc#1" os_prio=0 cpu=3.06ms elapsed=1162.53s tid=0x00007fec74000d30 nid=0x22f2da runnable  

"G1 Conc#2" os_prio=0 cpu=1.45ms elapsed=1162.53s tid=0x00007fec74001770 nid=0x22f2dc runnable  

"G1 Refine#0" os_prio=0 cpu=0.10ms elapsed=1164.13s tid=0x00007feca8131e10 nid=0x22f2b2 runnable  

"G1 Service" os_prio=0 cpu=110.02ms elapsed=1164.13s tid=0x00007feca8132d00 nid=0x22f2b3 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=308.21ms elapsed=1163.86s tid=0x00007feca80273a0 nid=0x22f2cc waiting on condition  

JNI global refs: 33, weak refs: 45
evaluateSequential(java.base@17.0.1/ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate(java.base@17.0.1/AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(java.base@17.0.1/ReferencePipeline.java:596)
    at org.apache.shardingsphere.test.integration.env.container.atomic.ITContainers.start(ITContainers.java:82)
    - locked <0x000000061cefdc20> (a org.apache.shardingsphere.test.integration.env.container.atomic.ITContainers)
    at org.apache.shardingsphere.test.integration.container.compose.mode.ClusterComposedContainer.start(ClusterComposedContainer.java:64)
    at org.apache.shardingsphere.test.integration.engine.BaseITCase.setUp(BaseITCase.java:78)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@17.0.1/Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@17.0.1/NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17.0.1/DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(java.base@17.0.1/Method.java:568)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
    at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runners.Suite.runChild(Suite.java:128)
    at org.junit.runners.Suite.runChild(Suite.java:27)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:383)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:344)
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:417)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0.73ms elapsed=1164.12s tid=0x00007feca8164e40 nid=0x22f2b5 waiting on condition  [0x00007fec71142000]
   java.lang.Thread.State: RUNNABLE
    at java.lang.ref.Reference.waitForReferencePendingList(java.base@17.0.1/Native Method)
    at java.lang.ref.Reference.processPendingReferences(java.base@17.0.1/Reference.java:253)
    at java.lang.ref.Reference$ReferenceHandler.run(java.base@17.0.1/Reference.java:215)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=0.53ms elapsed=1164.12s tid=0x00007feca8166220 nid=0x22f2b6 in Object.wait()  [0x00007fec707fe000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(java.base@17.0.1/Native Method)
    - waiting on <no object reference available>
    at java.lang.ref.ReferenceQueue.remove(java.base@17.0.1/ReferenceQueue.java:155)
    - locked <0x000000061c5b4578> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(java.base@17.0.1/ReferenceQueue.java:176)
    at java.lang.ref.Finalizer$FinalizerThread.run(java.base@17.0.1/Finalizer.java:172)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.22ms elapsed=1164.12s tid=0x00007feca816b580 nid=0x22f2b7 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=0.57ms elapsed=1164.12s tid=0x00007feca816c930 nid=0x22f2b8 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Monitor Deflation Thread" #6 daemon prio=9 os_prio=0 cpu=13.62ms elapsed=1164.12s tid=0x00007feca816dd40 nid=0x22f2b9 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=1485.55ms elapsed=1164.12s tid=0x00007feca816f770 nid=0x22f2ba waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #15 daemon prio=9 os_prio=0 cpu=489.43ms elapsed=1164.12s tid=0x00007feca8170ca0 nid=0x22f2bb waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #19 daemon prio=9 os_prio=0 cpu=16.61ms elapsed=1164.12s tid=0x00007feca8172110 nid=0x22f2bc runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #20 daemon prio=8 os_prio=0 cpu=1.42ms elapsed=1164.11s tid=0x00007feca817d960 nid=0x22f2bd in Object.wait()  [0x00007fec5b78b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(java.base@17.0.1/Native Method)
    - waiting on <no object reference available>
    at java.lang.ref.ReferenceQueue.remove(java.base@17.0.1/ReferenceQueue.java:155)
    - locked <0x000000061c5c00f8> (a java.lang.ref.ReferenceQueue$Lock)
    at jdk.internal.ref.CleanerImpl.run(java.base@17.0.1/CleanerImpl.java:140)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)
    at jdk.internal.misc.InnocuousThread.run(java.base@17.0.1/InnocuousThread.java:162)

"Notification Thread" #22 daemon prio=9 os_prio=0 cpu=0.06ms elapsed=1163.86s tid=0x00007feca8c02320 nid=0x22f2cb runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"surefire-forkedjvm-command-thread" #23 daemon prio=5 os_prio=0 cpu=14.96ms elapsed=1163.80s tid=0x00007feca8c77c40 nid=0x22f2cf runnable  [0x00007fec5a484000]
   java.lang.Thread.State: RUNNABLE
    at java.io.FileInputStream.readBytes(java.base@17.0.1/Native Method)
    at java.io.FileInputStream.read(java.base@17.0.1/FileInputStream.java:276)
    at java.io.BufferedInputStream.fill(java.base@17.0.1/BufferedInputStream.java:244)
    at java.io.BufferedInputStream.read(java.base@17.0.1/BufferedInputStream.java:263)
    - locked <0x000000061c5b9378> (a java.io.BufferedInputStream)
    at java.io.DataInputStream.readInt(java.base@17.0.1/DataInputStream.java:393)
    at org.apache.maven.surefire.booter.MasterProcessCommand.decode(MasterProcessCommand.java:115)
    at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:391)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"surefire-forkedjvm-ping-30s" #24 daemon prio=5 os_prio=0 cpu=746.88ms elapsed=1163.76s tid=0x00007feca8cdfc00 nid=0x22f2d0 waiting on condition  [0x00007fec5a383000]
   java.lang.Thread.State: TIMED_WAITING (parking)
    at jdk.internal.misc.Unsafe.park(java.base@17.0.1/Native Method)
    - parking to wait for  <0x000000061cc01470> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(java.base@17.0.1/LockSupport.java:252)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@17.0.1/AbstractQueuedSynchronizer.java:1672)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@17.0.1/ScheduledThreadPoolExecutor.java:1182)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@17.0.1/ScheduledThreadPoolExecutor.java:899)
    at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@17.0.1/ThreadPoolExecutor.java:1062)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.1/ThreadPoolExecutor.java:1122)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.1/ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"process reaper" #25 daemon prio=10 os_prio=0 cpu=271.39ms elapsed=1163.74s tid=0x00007febec016eb0 nid=0x22f2d3 waiting on condition  [0x00007fec700f7000]
   java.lang.Thread.State: TIMED_WAITING (parking)
    at jdk.internal.misc.Unsafe.park(java.base@17.0.1/Native Method)
    - parking to wait for  <0x000000061cc02490> (a java.util.concurrent.SynchronousQueue$TransferStack)
    at java.util.concurrent.locks.LockSupport.parkNanos(java.base@17.0.1/LockSupport.java:252)
    at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@17.0.1/SynchronousQueue.java:401)
    at java.util.concurrent.SynchronousQueue.poll(java.base@17.0.1/SynchronousQueue.java:903)
    at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@17.0.1/ThreadPoolExecutor.java:1061)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.1/ThreadPoolExecutor.java:1122)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.1/ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"OkHttp ConnectionPool" #28 daemon prio=5 os_prio=0 cpu=34.74ms elapsed=1162.53s tid=0x00007febdc089ad0 nid=0x22f2db in Object.wait()  [0x00007fec58be2000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(java.base@17.0.1/Native Method)
    - waiting on <no object reference available>
    at java.lang.Object.wait(java.base@17.0.1/Object.java:472)
    at org.testcontainers.shaded.okhttp3.internal.connection.RealConnectionPool.lambda$new$0(RealConnectionPool.java:62)
    - locked <0x000000061cd9a6d0> (a org.testcontainers.shaded.okhttp3.internal.connection.RealConnectionPool)
    at org.testcontainers.shaded.okhttp3.internal.connection.RealConnectionPool$$Lambda$129/0x0000000800db4438.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.1/ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.1/ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"testcontainers-ryuk" #31 daemon prio=5 os_prio=0 cpu=25.04ms elapsed=1161.40s tid=0x00007feca92711d0 nid=0x22f3f0 in Object.wait()  [0x00007fec5a585000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(java.base@17.0.1/Native Method)
    - waiting on <no object reference available>
    at org.testcontainers.utility.ResourceReaper.lambda$null$1(ResourceReaper.java:170)
    - locked <0x000000061e4602e0> (a java.util.ArrayList)
    at org.testcontainers.utility.ResourceReaper$$Lambda$188/0x0000000800ec8228.run(Unknown Source)
    at org.rnorth.ducttape.ratelimits.RateLimiter.doWhenReady(RateLimiter.java:27)
    at org.testcontainers.utility.ResourceReaper.lambda$start$2(ResourceReaper.java:159)
    at org.testcontainers.utility.ResourceReaper$$Lambda$187/0x0000000800ec8000.run(Unknown Source)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"docker-java-stream-2041008728" #33 daemon prio=5 os_prio=0 cpu=23.42ms elapsed=1160.63s tid=0x00007feca927f250 nid=0x22f564 runnable  [0x00007fec5b488000]
   java.lang.Thread.State: RUNNABLE
    at org.testcontainers.shaded.com.github.dockerjava.okhttp.UnixDomainSocket.recv(Native Method)
    at org.testcontainers.shaded.com.github.dockerjava.okhttp.UnixDomainSocket$UnixSocketInputStream.read(UnixDomainSocket.java:246)
    at java.io.FilterInputStream.read(java.base@17.0.1/FilterInputStream.java:132)
    at org.testcontainers.shaded.com.github.dockerjava.okhttp.UnixSocketFactory$1$1.read(UnixSocketFactory.java:45)
    at org.testcontainers.shaded.okio.Okio$2.read(Okio.java:140)
    at org.testcontainers.shaded.okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
    at org.testcontainers.shaded.okio.RealBufferedSource.request(RealBufferedSource.java:72)
    at org.testcontainers.shaded.okio.RealBufferedSource.require(RealBufferedSource.java:65)
    at org.testcontainers.shaded.okio.RealBufferedSource.readHexadecimalUnsignedLong(RealBufferedSource.java:307)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.readChunkSize(Http1ExchangeCodec.java:492)
### Reason analyze (If you can)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.java:471)
    at org.testcontainers.shaded.okhttp3.internal.connection.Exchange$ResponseBodySource.read(Exchange.java:286)
    at org.testcontainers.shaded.okio.RealBufferedSource$1.read(RealBufferedSource.java:436)
    at org.testcontainers.shaded.com.github.dockerjava.core.FramedInputStreamConsumer.accept(FramedInputStreamConsumer.java:30)
    at org.testcontainers.shaded.com.github.dockerjava.core.FramedInputStreamConsumer.accept(FramedInputStreamConsumer.java:12)
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:275)
    at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder$$Lambda$177/0x0000000800ec0558.run(Unknown Source)
    at java.lang.Thread.run(java.base@17.0.1/Thread.java:833)

"Attach Listener" #35 daemon prio=9 os_prio=0 cpu=0.60ms elapsed=693.45s tid=0x00007fec54001550 nid=0x22fa58 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=130.82ms elapsed=1164.12s tid=0x00007feca8160df0 nid=0x22f2b4 runnable  

"GC Thread#0" os_prio=0 cpu=30.00ms elapsed=1164.13s tid=0x00007feca805adb0 nid=0x22f2af runnable  

"GC Thread#1" os_prio=0 cpu=27.36ms elapsed=1163.95s tid=0x00007fec680068c0 nid=0x22f2c1 runnable  

"GC Thread#2" os_prio=0 cpu=30.48ms elapsed=1163.95s tid=0x00007fec680072f0 nid=0x22f2c2 runnable  

"GC Thread#3" os_prio=0 cpu=25.00ms elapsed=1163.95s tid=0x00007fec68007d20 nid=0x22f2c3 runnable  

"GC Thread#4" os_prio=0 cpu=32.47ms elapsed=1163.95s tid=0x00007fec68008b40 nid=0x22f2c4 runnable  

"GC Thread#5" os_prio=0 cpu=31.29ms elapsed=1163.95s tid=0x00007fec68009980 nid=0x22f2c5 runnable  

"GC Thread#6" os_prio=0 cpu=28.47ms elapsed=1163.95s tid=0x00007fec6800a7e0 nid=0x22f2c6 runnable  

"GC Thread#7" os_prio=0 cpu=34.28ms elapsed=1163.95s tid=0x00007fec6800b640 nid=0x22f2c7 runnable  

"GC Thread#8" os_prio=0 cpu=35.57ms elapsed=1163.95s tid=0x00007fec6800c4a0 nid=0x22f2c8 runnable  

"GC Thread#9" os_prio=0 cpu=34.38ms elapsed=1163.95s tid=0x00007fec6800d300 nid=0x22f2c9 runnable  

"GC Thread#10" os_prio=0 cpu=30.56ms elapsed=1163.95s tid=0x00007fec6800e160 nid=0x22f2ca runnable  

"GC Thread#11" os_prio=0 cpu=30.47ms elapsed=1162.22s tid=0x00007fec680186d0 nid=0x22f30b runnable  

"GC Thread#12" os_prio=0 cpu=25.04ms elapsed=1162.22s tid=0x00007fec68018d80 nid=0x22f30c runnable  

"G1 Main Marker" os_prio=0 cpu=0.41ms elapsed=1164.13s tid=0x00007feca806b5d0 nid=0x22f2b0 runnable  

"G1 Conc#0" os_prio=0 cpu=3.12ms elapsed=1164.13s tid=0x00007feca806c530 nid=0x22f2b1 runnable  

"G1 Conc#1" os_prio=0 cpu=3.06ms elapsed=1162.53s tid=0x00007fec74000d30 nid=0x22f2da runnable  

"G1 Conc#2" os_prio=0 cpu=1.45ms elapsed=1162.53s tid=0x00007fec74001770 nid=0x22f2dc runnable  

"G1 Refine#0" os_prio=0 cpu=0.10ms elapsed=1164.13s tid=0x00007feca8131e10 nid=0x22f2b2 runnable  

"G1 Service" os_prio=0 cpu=110.02ms elapsed=1164.13s tid=0x00007feca8132d00 nid=0x22f2b3 runnable  

"VM Periodic Task Thread" os_prio=0 cpu=308.21ms elapsed=1163.86s tid=0x00007feca80273a0 nid=0x22f2cc waiting on condition  

JNI global refs: 33, weak refs: 45

Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.

The scenario may be changed.

             ./mvnw -B install -f shardingsphere-test/shardingsphere-integration-test/shardingsphere-integration-test-suite/pom.xml \
                 -Dit.run.modes=Cluster \
                 -Dit.cluster.databases=MySQL \
                 -Dit.cluster.adapters=proxy \
                 -Dit.scenarios=dbtbl_with_readwrite_splitting \
                 -Dit.cluster.env.type=DOCKER
TeslaCN commented 2 years ago

It seems that the thread is stuck in a dead loop.

image

image

I don't know if this is normal that the pos is greater than limit. image

TeslaCN commented 2 years ago

https://github.com/square/okio/issues/1133

github-actions[bot] commented 1 year ago

Hello , this issue has not received a reply for several days. This issue is supposed to be closed.

TeslaCN commented 1 year ago

This issue hasn't happened recently.