Running benchmarks against Aiven instance

[jeqo]

Progress: https://github.com/jeqo/openmessaging-benchmark/pull/2

Issues:

• driver-kop seem to be broken
• Has issues with Terraform, but created instances manually
• Using PEM files, there's a bug on the version used by the framework (3.2.1) and had to bump to 3.3.2
• Had to hardcode security configs on every driver config

[jeqo]

Command to build binaries:

 mvn install -pl "\!driver-kop" -am    

and avoid broken kop driver

[jeqo]

Command to run ansible on hardcoded client instances:

 ansible-playbook -i hosts.yaml deploy.yaml --limit client 

[jeqo]

AWS Security group needs:

• Access on port 8080 from within the security group.
• ICMP if test pings are needed
• SSH

[jeqo]

Command to run benchmark from one of the agents:

bin/benchmark --drivers driver-kafka/kafka-throughput.yaml workloads/1-topic-16-partitions-1kb.yaml

[jeqo]

Initial results for business 4 cluster on gcp eu-central-1: 1-topic-16-partitions-1kb-Kafka-2023-06-22-06-11-54.zip

test | avg | 50pct | 75pct | 95pct | 99pct | 999pct | 9999pct | max -- | -- | -- | -- | -- | -- | -- | -- | -- end-to-end-latency | 752.1609456 | 660.003 | 1065.007 | 1671.007 | 2125.007 | 2749.007 | 3620.015 | 4167.007 publish-latency | 679.4603813 | 584.427 | 994.995 | 1603.247 | 2060.215 | 2718.399 | 3476.687 | 4086.639