nats-io / nats-architecture-and-design

Architecture and Design Docs
Apache License 2.0
177 stars 20 forks source link

Create client latency benchmark documentation / ADR #37

Closed scottf closed 1 year ago

scottf commented 2 years ago

Overview

It would be useful to define a benchmarking standard that all clients can implement. This means that we should define things like what constitutes a round trip, where timing points are what payload sizes are, what type of consumers to use etc.

The idea being that all clients could then implement something to this standard so the test can legitimately compare performance between languages.

AFAIK, the Java client and the CLI have significant benchmarking suites.

bruth commented 1 year ago

Recently I brought up the same rough idea since it seems to be a fairly common question. Were you thinking something akin to gRPC's a dashboard?

scottf commented 1 year ago

I was just thinking that it would be useful to have a benchmark algorithm defined so it could be implemented in all languages and we could compare apples to apples, even if the apples were different varieties.

aricart commented 1 year ago

The rtt, is simply a server ping, the amount of time it takes for the client to get a response back - so using the clients flush() and timing it should be sufficient.

scottf commented 1 year ago

This is not for RTT.

scottf commented 1 year ago

Closing due to lack of interest