hashgraph / hedera-services

Crypto, token, consensus, file, and smart contract services for the Hedera public ledger
Apache License 2.0
281 stars 125 forks source link

Create some GRPC PublishStreamRequest benchmarks #15251

Open derektriley opened 2 weeks ago

derektriley commented 2 weeks ago

https://github.com/hashgraph/hedera-services/tree/grpc-benchy

Create a few GRPC benchmarks for the bidi stream rpc publishBlockStream (stream PublishStreamRequest) returns (stream PublishStreamResponse);

I would like to see the throughput in requests/sec (PublishStreamRequest) that can be made before the GRPC server on the other side has some threshold of a delay. I.e. We're sending requests so fast that the GRPC server (Block Node) may process it 5 seconds later, 10, 15 etc.

derektriley commented 2 weeks ago

Ran an initial test (on my windows machine), it starts out at 10k requests/s and increases the requests/s by 5000 every 10 seconds until the delay between the GRPC server receiving the request exceeds 10 seconds. Plotted the results using matplotlib in python. Going to think about this approach and if the test is accurate in different ways.

Also going to look into https://github.com/grpc/grpc-java/blob/master/benchmarks/README.md and understand the context of grpc's queries per second benchmark. Image

derektriley commented 2 weeks ago

The gRPC QPS benchmark in grpc-java is designed to measure the maximum number of queries that can be handled per second, and it does this by sending requests and waiting for responses. Modifying it to not wait for responses would change the behavior of the benchmark and would be comparable to our use case. I am working on this modification to have a more comparable test to the one above.

I ran the qps client test with the following parameters to test 1 channel being open (to the GRPC server/block node) and 1 outstanding rpc in flight to simulate that the order we are submitting requests is critical. The payload size needs to be configured slightly more (going to get the size in bytes of the BlockItem (CryptoTransferTransactionBody) I am sending on the wire.

./qps_client --address=localhost:50051 --channels=1 --outstanding_rpcs=1 --duration=120

Channels: 1 Outstanding RPCs per Channel: 1 Server Payload Size: 0 Client Payload Size: 4 50%ile Latency (in micros): 129 90%ile Latency (in micros): 198 95%ile Latency (in micros): 239 99%ile Latency (in micros): 347 99.9%ile Latency (in micros): 515 Maximum Latency (in micros): 2847 QPS: 6848