search

swisspost / vertx-redisques

A highly scalable redis-persistent queuing system for vertx
Other
13 stars 8 forks source link

Is Vertx Redis API blocking? #165

Open hiddenalpha opened 7 months ago

hiddenalpha commented 7 months ago

Why do I have the impression that vertx redis API seems to perform blocking IO on the eventloop? See attached vertx-redis-client-blocking-EventLoop.log [6.7MiB].

I already opened https://github.com/swisspost/vertx-redisques/pull/161 to fix one specific case. But still, I'm confused that redisAPI seems to be blocking.

Question 1: In which extent is my assumption correct here? Or do I see some other effect I'm not aware of?

Question 2: If my assumption is correct, does this mean ALL our calls to redis API throughout our codebase are potentially blocking the eventloop?

hiddenalpha commented 6 months ago

Another example:

2024-05-28T12:04:52,327 test houston WARN BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-1,5,main] has been blocked for 75 ms
io.vertx.core.VertxException: Thread blocked
    at io.vertx.redis.client.impl.types.BulkType.create(BulkType.java:30) ~[vertx-redis-client-4.5.1.jar:4.5.1]
    at io.vertx.redis.client.impl.RESPParser.handle(RESPParser.java:128) ~[vertx-redis-client-4.5.1.jar:4.5.1]
    at io.vertx.redis.client.impl.RESPParser.handle(RESPParser.java:24) ~[vertx-redis-client-4.5.1.jar:4.5.1]
    at io.vertx.core.net.impl.NetSocketImpl.lambda$new$1(NetSocketImpl.java:101) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.vertx.core.net.impl.NetSocketImpl$$Lambda$1590/0x00000008406b8840.handle(Unknown Source) ~[?:?]
    at io.vertx.core.streams.impl.InboundBuffer.handleEvent(InboundBuffer.java:255) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.vertx.core.streams.impl.InboundBuffer.write(InboundBuffer.java:134) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.vertx.core.net.impl.NetSocketImpl$DataMessageHandler.handle(NetSocketImpl.java:402) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:335) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:328) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.vertx.core.net.impl.NetSocketImpl.handleMessage(NetSocketImpl.java:378) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.vertx.core.net.impl.ConnectionBase.read(ConnectionBase.java:159) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:153) ~[vertx-core-4.5.1.jar:4.5.1]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.103.Final.jar:4.1.103.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.103.Final.jar:4.1.103.Final]
    at java.lang.Thread.run(Thread.java:829) ~[?:?]