search

commercetest / pepper-box

Pepper-Box is kafka load generator plugin for jmeter. It produces kafka messages of type plain text (JSON, XML, CSV or any other custom format) as well as java serialized objects.
http://pepperbox.gslab.com/
Apache License 2.0
2 stars 3 forks source link

Add facility to generate per-thread topic names automatically and predictably #5

Closed julianharty closed 6 years ago

julianharty commented 6 years ago

There are several use cases for testing Kafka at scale with jmeter. One is to see how lots of messages to a single topic will affect the behaviour. Another is to write to lots of topics, simulating many kafka clients. This issue is an enhancement to pepper-box to add the ability for the user to request pepper-box / jmeter to create a separate, predictable topic per thread.

We'll start with something simple, appending a suffix based on the thread number that's running. It'll be enough to test the proposed approach. We may then explore other approaches such as having each thread write to several topics, but that's likely to complicate the codebase and testing of pepper-box.

julianharty commented 6 years ago

We'll also add a gating flag that the user can use to enable this capability. By default it'll be disabled to maintain the current behaviour.

julianharty commented 6 years ago

The gating flag is: generate.per-thread.topics

The default value is NO as this enables the current tests to run without complaint. Set it to YES (or 'yes' as the check ignores the case of the text) to enable this feature.

julianharty commented 6 years ago

This works adequately based on my testing so I'll close this issue now so we can focus on what we still need to do.