kgiusti / ombt

oslo messaging benchmarking tool
Apache License 2.0
3 stars 5 forks source link

Throughput vs message rate #15

Open msimonin opened 6 years ago

msimonin commented 6 years ago

Currently in the statistics, throughput is measured in terms of number of messages per second. As mentionned by @ajssmith this may confuse people since throughput is usually measured in byte/s.

By design, ombt compute the message rate. Getting the throughput may be more difficult since one should take into account how the payload is actually encoded on the wire. This encoding is dependent on the transport used (amqp 0.9 or amqp 1.0 for instance). Note that the throughput is maybe accessible using the various monitoring tools of the bus itself (rabbitmq management console, qdstat ?).

kgiusti commented 6 years ago

I've run the rpc tests against rabbit and qdrouterd and captured the network traffic via wireshark. The 'on the wire' size of the message did vary based on the type of backend as well as the message (request vs. response). In addition the sizes were not exactly constant - varying a few bytes occasionally.

Unfortunately oslo.messaging does not give us access to the size actual encoded data - it's a high level API that only deals in procedure callers and handlers.

Another point I noticed is that the protocol 'overhead' - the non-data transfer protocol messages (like, ack, flow control, etc) - varied over time.

So in summation I don't think it is likely that we can get an accurate throughput measurement that takes into account all of the bytes that compose a message transfer.

However we can accurately measure the throughput of the payload data. This would provide the throughput from the application's point of view. And the payload size is configurable via the --length argument available to all of the tests (defaults to 1024 bytes).

Thoughts?

msimonin commented 6 years ago

Thanks @kgiusti for diving into this. I think we shouldn't try from the application to estimate the actual throuput. As you mention there are many sources of variability (transport, ack management ...). Throughput of the payload data is something we can have deduce from the stats returned by ombt. Maybe there is a chance to get the throughput from dedicated monitoring tool of the bus (rabbitmq management plugin). Do you know if qdstat has it ?