GRPC python client can not read big response from BRPC server

[dashjay]

Describe the bug (描述bug) When server response a big value(more than 4096K), client will get error.

I think this can be resolve by add some grpc options, but I don't know detail

To Reproduce (复现方法)

I edited the example echo client/server, client send 8M message to server and receive response result success.

https://github.com/dashjay/brpc/commit/923555a205288ebe1e0c251a4e83506e2f7a1bb4

Then when I generated a python grpc client and write a small case

from math import exp
import grpc
import echo_pb2
import echo_pb2_grpc
import random
import time
import string

keySize = 4096
size = keySize * 1024
v_large = ''.join(random.choices(string.ascii_lowercase, k=size))
bv_large = str.encode(v_large)

options = [
    ('grpc.max_receive_message_length', 83804160),
    ('grpc.max_send_message_length', 83804160),
]

channel = grpc.insecure_channel('0.0.0.0:8000', [])
client = echo_pb2_grpc.EchoServiceStub(channel)
req = echo_pb2.EchoRequest(message = bv_large)

def send():
    resp = client.Echo(req, timeout=10)
    print(len(resp.message))

if __name__ == '__main__':
    while True:
        try:
            send()
        except KeyboardInterrupt as e:
            break

server run normally:

I0703 12:02:46.774291  5379 755914244353 server.cpp:60] Received request[log_id=0] from 127.0.0.1:53151 to 127.0.0.1:8000: length(4194304) (attached=)
I0703 12:02:46.787413  5379 755914244353 server.cpp:89] req: length(4194318) res: length(4194318)

but grpc python client:

bytedance@FY2MY2TWDK ~/D/c/b/e/echo_c++ (issue/server-no-buffer-for-grpc-client)> /Users/bytedance/.pyenv/versions/3.10.13/bin/python3 python_client.py
Traceback (most recent call last):
  File "/codes/brpc/example/echo_c++/python_client.py", line 31, in <module>
    send()
  File "/codes/brpc/example/echo_c++/python_client.py", line 24, in send
    resp = client.Echo(req, timeout=10)
  File "/usr/lib/python3.10/site-packages/grpc/_channel.py", line 1181, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/python3.10/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "Received RST_STREAM with error code 3"
        debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-07-03T12:02:46.789109+08:00", grpc_status:13, grpc_message:"Received RST_STREAM with error code 3"}"
>

Expected behavior (期望行为)

Get the echo value.

I found on web and get this https://groups.google.com/g/grpc-io/c/3h-Q5Vjq5aU ,I suspect this has something to do with it

Versions (各种版本) OS: Darwin FY2MY2TWDK 23.3.0 Darwin Kernel Version 23.3.0 Compiler: Apple clang version 15.0.0 (clang-1500.3.9.4) brpc: master protobuf: protobuf: stable 27.1 (bottled)

Additional context/screenshots (更多上下文/截图)

BRPC server interact with BRPC client good

[dashjay]

You can easily use command like this to access BRPC server with http proto

dd if=/dev/random of=/tmp/test bs=4M count=1
echo "{\"message\": \"$(cat /tmp/test | base64)\"}" > test.1
curl -H 'Content-Type: application/json' --data-binary "@./test.1" 0.0.0.0:8000/EchoService/Echo > /tmp/test.2

But you can not use grpc client to do this.

[wlleiiwang]

We encountered the same problem, how was it solved?

[dashjay]

We encountered the same problem, how was it solved?

@wlleiiwang Hi, finally we solve this problem by invoking the HTTP interface directly, brpc server implement http interface automaticly.