request help: failed to receive RPC_HTTP_REQ_CALL: timeout

[Cheol-Soon-Choi]

Issue description

At some point during apisix operation, an error suddenly occurs and the client call does not proceed at all. For all client calls, can check the error log as below.

phase_func(): failed to receive RPC_HTTP_REQ_CALL: timeout

run_plugin(): ext-plugin-pre-req exits with http status code 503

ERROR 57658 --- [tLoopGroup-34-6] o.a.a.p.r.h.ExceptionCaughtHandler       : handle request error:
io.netty.channel.StacklessClosedChannelException: null
        at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source) ~[netty-transport-4.1.78.Final.jar!/:4.1.78.Final]
Caused by: io.netty.channel.unix.Errors$NativeIoException: writeAddress(..) failed: Broken pipe

It was confirmed that the client call resumed normally after 20 to 30 minutes without taking any action.

I'd like to know under what circumstances this problem occurs and prevent it from recurring. Is it a linux or apisix or java runner resource issue? I wish I could give you some advice on this matter. Thanks in advance. @spacewander

Environment

linux RAM 8G / java runner cmd option is '-Xmx2g', '-Xms2g' / RPS is about 200~600(tends to happen when rps are high, but doesn't always happen)

• your apisix-java-plugin-runner version 0.4.0 apisix 3.2.1