Benchmarks

[troian]

TBD

[X-Ryl669]

Here are some I've just done in the previous days: The benchmark tools run use this command line:

$ ./emqtt_bench pub -h 127.0.0.1 -p 1883 -V 5 -c 300 -I 3 -u username -P password -t /testme -s 512
$ ./mqtt-bench  -action=p -broker="tcp://localhost:1883" -broker-password="password" -broker-username="username" -clients=300 -size=512 -count=1000

Note: Removing docker-proxy

Docker userland proxy as it's consuming most of the CPU in such benchmarks In /etc/docker/daemon.json:

{
    "userland-proxy": false
}

Without this, docker is using the kernel's netif to bridge the packets in kernel space instead of user space. Don't forget to restart docker: systemctl restart docker A solution without Docker would probably give even higher results.

Result of benchmarks:

The machine is a 8 core Intel Xeon W3530 with 16GB of RAM. All the tests are not using TLS and runs locally on the server (no network interface limitation here)	Benchmark	MQTT version	Comment	Result
VolantMQ (running in Docker, since that's the only way to get it)
mqtt-bench	v3.1.1	publish size 512bytes/msg	18k msg/s
mqtt-bench	v3.1.1	publish size 256bytes/msg	20k msg/s
emqtt-bench	v5	publish size 512bytes/msg	110k msg/s
emqtt-bench	v5	publish size 256bytes/msg	90k msg/s
CPU usage: 150% (bench tool: 650%)
Memory: 0.9% (bench tool: 0.6%)
VerneMQ default configuration with file based authentication
mqtt-bench	v3.1.1	publish size 512bytes/msg	18k msg/s
mqtt-bench	v3.1.1	publish size 256bytes/msg	20k msg/s
emqtt-bench	v5	publish size 512bytes/msg	85k msg/s
emqtt-bench	v5	publish size 256bytes/msg	114k msg/s
CPU usage: 200% (bench tool: 600%)
Memory: 0.9% (bench tool: 1%)
Mosquitto
mqtt-bench	v3.1.1	publish size 512bytes/msg	25k msg/s
mqtt-bench	v3.1.1	publish size 256bytes/msg	22k msg/s
emqtt-bench	v5	publish size 512bytes/msg	100k msg/s
emqtt-bench	v5	publish size 256bytes/msg	100k msg/s
CPU usage: 100% (bench tool: 700%)
Memory: 0% (bench tool: 0.5%)

[troian]

@X-Ryl669 Appreciate your efforts of getting into this. Would you mind do some profiling as well to help understand bottlenecks?

[X-Ryl669]

If you tell me how, I can do that for sure...

[troian]

I would start with pprof. VolantMQ docker image contains debug plugin which exposes pprof over http so you could catch cpu profile and see who is taking longest