Mainflux Benchmarks

[dborovcanin]

Improve Mainflux benchmarks and load tests. We have a repo for load tests, but it is outdated. Some of the topics worth exploring:

• [ ] Add benchmark tests in Go

• [ ] Explore tools for distributed load testing and use them on Mainflux deployment in the cloud

• [ ] Explore different scenarios (check the linked repo for more info):

  • Fan-in
  • Fan-out
  • Experiment with message size and frequency
  • Compare protocols
  • Explore different brokers and brokers' configs

• [ ] Create reports that can be used for marketing and showcasing Mainflux capabilities

[dborovcanin]

Worth investigation: https://github.com/ultravioletrs/issues/issues/291#issuecomment-1666640602

[rodneyosodo]

Add benchmark tests in Go

Which tool will we use to visualize and act on the results from the benchmark? For example I wrote benchmark tests for ws/dapater.go

When I used

go tool pprof -http=:8080 cpu.prof

I get this output

On top of my head is we need to used a better strings concatenation

[rodneyosodo]

References

• https://arxiv.org/pdf/2305.13893.pdf
• https://www.altoroslabs.com/blog/a-collection-of-mqtt-broker-performance-benchmarks-2020-2023/
• https://www.koziolek.de/docs/Koziolek2020-ECSA-preprint.pdf?ref=learn.umh.app
• https://github.com/emqx/mqttbs
• https://www.emqx.com/en/blog/open-mqtt-benchmark-suite-the-ultimate-guide-to-mqtt-performance-testing
• https://re.public.polimi.it/retrieve/059afd8c-20fb-4752-adbe-96b514e16b8e/longo2022_border.pdf
• https://www.scalagent.com/IMG/pdf/Benchmark_MQTT_servers-v1-1.pdf