Setting maxInboundMessageSize didn't increase message size on server

[bnowakow]

The context

I'd like to increase maxInboundMessageSize on server.

The bug

As suggested in #163 (if I understood it right) in application.properties I'm setting grpc.server.max-inbound-message-size: 5983493 but this variable isn't read/used. This variable is exposed to be set: InteliJ screenshot. Smaller messages are sent fine.

Steps to Reproduce

When I request bigger gRPC message than 4MB default value I get { "error": "8 RESOURCE_EXHAUSTED: Received message larger than max (5983493 vs. 4194304)" } gRPC message from server.

The application's environment

• Spring (boot): 2.2.5.RELEASE
• grpc-java:1.21.0
• grpc-spring-boot-starter: 2.7.0.RELEASE
• java: version + architecture (64bit?): jdk1.8.0_171.jdk 64bit

Exception

2020-03-18 12:02:58.269  INFO 17290 --- [ault-executor-1] p.b.p.p.controller.PrimeNumberService    : Received gRPC request to generate prime numbers up to: 2000000
2020-03-18 12:02:58.390  WARN 17290 --- [-worker-ELG-3-2] i.g.n.s.i.grpc.netty.NettyServerHandler  : Stream Error

io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception$StreamException: Stream closed before write could take place
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:167) ~[grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.cancel(DefaultHttp2RemoteFlowController.java:481) ~[grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$1.onStreamClosed(DefaultHttp2RemoteFlowController.java:105) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection.notifyClosed(DefaultHttp2Connection.java:356) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.removeFromActiveStreams(DefaultHttp2Connection.java:1000) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.deactivate(DefaultHttp2Connection.java:956) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:512) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:518) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.closeStream(Http2ConnectionHandler.java:613) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onRstStreamRead(DefaultHttp2ConnectionDecoder.java:408) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onRstStreamRead(Http2InboundFrameLogger.java:80) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readRstStreamFrame(DefaultHttp2FrameReader.java:516) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:260) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:160) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:174) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:378) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]

2020-03-18 12:02:58.391  WARN 17290 --- [-worker-ELG-3-2] i.g.n.s.i.grpc.netty.NettyServerHandler  : Stream Error

io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception$StreamException: Stream closed before write could take place
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:167) ~[grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.cancel(DefaultHttp2RemoteFlowController.java:481) ~[grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$1.onStreamClosed(DefaultHttp2RemoteFlowController.java:105) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection.notifyClosed(DefaultHttp2Connection.java:356) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.removeFromActiveStreams(DefaultHttp2Connection.java:1000) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.deactivate(DefaultHttp2Connection.java:956) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:512) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:518) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.closeStream(Http2ConnectionHandler.java:613) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onRstStreamRead(DefaultHttp2ConnectionDecoder.java:408) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onRstStreamRead(Http2InboundFrameLogger.java:80) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readRstStreamFrame(DefaultHttp2FrameReader.java:516) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:260) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:160) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:174) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:378) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [grpc-netty-shaded-1.27.1.jar:1.27.1]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]

[bnowakow]

I'm sorry it's not a bug, it was my poor debug.

I was using 2 clients:

• BloomRPC client v1.4.1.145 which has client side limitation of receiving messages no bigger than 4MB (exception was probably it used http1 instead of http2 for larger message - that's only my unverified assumption)
• io.github.lognet.grpc-spring-boot-starter in which I needed increase timeout for bigger messages by calling keepAliveTimeout on ManagedChannel (I had maxInboundMessageSize already set on this).

How can I change label from bug to question?

[ST-DDT]

io.github.lognet.grpc-spring-boot-starter

That is a different library, or was that a typo?

[bnowakow]

I was using "yours" server and "theirs" client previously. So it wasn't a bug or configuration issue in "yours" server but a configuration in "theirs" client.

Btw. I've already migrated from "their" client to "yours" one, it's much more usable, thank you.

[ST-DDT]

I was using "yours" server and "theirs" client previously. So it wasn't a bug or configuration issue in "yours" server but a configuration in "theirs" client.

Btw. I've already migrated from "their" client to "yours" one, it's much more usable, thank you.

I'm still confused. Which library do you use:

• net.devh:grpc-spring-boot-starter (this library)
• or io.github.lognet:grpc-spring-boot-starter (a different library)
• or both?

[bnowakow]

Both, I was using

• net.devh:grpc-server-spring-boot-starter on server
• io.github.lognet:grpc-spring-boot-starter on client

[ST-DDT]

Thanks for the clarification.

Do you have any questions? How can we help you?

[bnowakow]

No, other than congrats for very good library and very useful FAQ

[ST-DDT]

I'd like to increase maxInboundMessageSize on server.

You can use grpc.server.maxInboundMessageSize=15MB in your config.

No, other than congrats for very good library and very useful FAQ

Thanks!