openmessaging / benchmark

OpenMessaging Benchmark Framework
Apache License 2.0
385 stars 237 forks source link

Kafka benchmark support cases not available #393

Closed HackToday closed 8 months ago

HackToday commented 12 months ago

Two common cases seems not possible with this benchmark:

Case1: I checked the workload samples https://github.com/openmessaging/benchmark/tree/master/workloads and https://openmessaging.cloud/docs/benchmarks/ it seems not have way to set topic name when benchmarking

As we know the kafka logs dir have topic name as prefix, if not have name set for workload, it may not quite easy to find the related log directory.

Case 2: Also, for the workload, it seems only support random case or same payload message over and over. Sometimes I have some payload, like production logs 1GB, how can make it 10Kb message iterate the 1GB logs. It seems right now, you can only pass 10Kb payload file, and the benchmark can only send 10Kb payload over and over. Not very common case.

plazma-prizma commented 11 months ago

Hi HackToday, would useRandomizedPayloads: true option be helpful for your case?

HackToday commented 11 months ago

hi @plazma-prizma I did use useRandomizedPayloads, it could not solve the above two cases. Thanks

stale[bot] commented 9 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 5 days if no further activity occurs. Thank you for your contributions.