Creating a Kafka consumer for jmeter

julianharty commented 6 years ago

When I test I would like to be able to test the consumption of the messages being generated and record when they were processed so we can assess the throughput and approximate latency.

From a testing perspective I'd like:

To post-process the results to calculate latency rather than risk slowing down the consumer
To design the data format so the results are easy to process automatically
To automatically pass in topic names and other configuration details automatically from the producer so they are kept in sync rather than require testers to manually edit changes in several places
To record test configuration and runtime data in the results so we don't lose valuable data or confuse one set of results with another, etc.
To maintain the script as code (note: the contents of the script may be loaded into jmeter's JSR223 sampler for convenience, however that might cause other complications).
To support similar security modes as the producer, in particular SASL_SSL

There are some encouraging examples particularly https://www.blazemeter.com/blog/apache-kafka-how-to-load-test-with-jmeter which uses a JSR223 sampler and https://www.blazemeter.com/blog/beanshell-vs-jsr223-vs-java-jmeter-scripting-its-performance which indicates performance might be adequate if we use groovy for the scripting language.

julianharty commented 6 years ago

Here are some useful reading material for groovy and for running groovy scripts with jmeter.

The Groovy Coding Style Guide
Tips for key differences between Java and Groovy vital for me where I'm used to Java...
jmeter version 3.2 includes groovy ./lib/groovy-all-2.4.13.jar so there's no need to install it separately.
Javadoc for JSR223Sampler
How to nest JSON objects which is the first data structure I'm planning to try using to format the results.

julianharty commented 6 years ago

Here are some reading material on JSR223

julianharty commented 6 years ago

The following has taken me hours to discover, so it's hard won information, for me at least.

Firstly the values can be set at a Test Plan level as User Defined Variables. They can also be set in a similar fashion to how Pepper-Box PlainText Config sets the placeholder MESSAGE

Java code, which would live in pepper-box sampler

JMeterVariables jMeterVariables = new JMeterVariables();
log.info("vars(\"zookeepers\":" + jMeterVariables.get("zookeepers")); // returns null

// in contrast the following is able to obtain the User Defined Value "zookeepers"
log.info("zk = " + JMeterContextService.getContext().getVariables().get("zookeepers")); 

try {
  JMeterUtils.setProperty("TOPIC", topic);
  } catch (NullPointerException npe) {
  // The try/catch is needed as the tests return a NPE when setProperty is called
  log.error("couldn't find JMeterUtils.");
}

I want to be able to read shared settings and properties created in the pepper-box sampler in groovy. Here's the appropriate groovy code

import org.apache.jmeter.util.JMeterUtils

log.info("TOPIC is set to " + JMeterUtils.getProperty("TOPIC"))
log.info("zk = " + vars.get("zookeepers"))

julianharty commented 6 years ago

Note: User Defined Variables can also be assigned as values in the tabular input for Java Request by using the ${zookeepers} style of entry. So, for instance after creating the relevant User Defined Variables at the Test Plan, I configure the following properties as follows:

Name | Value

bootstrap.servers   ${kafkabrokers}
zookeeper.servers   ${zookeepers}
kafka.topic.name    ${topic}

julianharty commented 6 years ago

Here's one way to get the hostname, by calling the relevant Java library directly

import java.net.InetAddress

log.info(InetAddress.getLocalHost().getHostName())

I'm still seeking a way to use jmeter's inbuilt functions, see https://stackoverflow.com/questions/48384721/how-can-we-use-built-in-jmeter-functions-in-groovy

julianharty commented 6 years ago

It's time to close this particular ticket as we've now got a Groovy script that seems to consume records per topic. There are still some issues related to specific changes and behaviours in the consumer code which we'll use to track those details, however this issue is complete in my view.

commercetest / pepper-box

Creating a Kafka consumer for jmeter #7