Pepper-Box is kafka load generator plugin for jmeter. It produces kafka messages of type plain text (JSON, XML, CSV or any other custom format) as well as java serialized objects.
During our testing we experience LEADER_NOT_AVAILABLE error messages for both the producer and consumer interactions with Kafka. The Kafka nodes are currently running with auto.create.topics.enable (see https://kafka.apache.org/documentation/#brokerconfigs for details on this effects of the setting for Kafka).
Leaving aside any debate on the merits of enabling and relying on auto-creation of topics, I'd like to focus on how it affects the behaviours of client code for both the producer and consumer aspects. And in particular I'd like to concentrate on the effects on this project.
One particular quirk and complication is where the server responds with a LEADER_NOT_AVAILABLE message. This seems to be reported as an error. Afterwards we sometimes seem to be able to send or receive messages, but often the code is unable to do so and the respective test eventually times-out.
The logging by/via/through jmeter doesn't provide many clues. What is apparent is that the pepper-box code treats any exception as a failure and reports it as such. It doesn't not explicitly try to interpret or address what's happened.
From searching online I've yet to find much clear, coherent and current information. This error message has been around for many releases of Kafka, including the old APIs. I'm hoping we'll at least find ways to mitigate and recover-from this response in both our producer and consumer code.
During our testing we experience LEADER_NOT_AVAILABLE error messages for both the producer and consumer interactions with Kafka. The Kafka nodes are currently running with
auto.create.topics.enable(see https://kafka.apache.org/documentation/#brokerconfigs for details on this effects of the setting for Kafka).Leaving aside any debate on the merits of enabling and relying on auto-creation of topics, I'd like to focus on how it affects the behaviours of client code for both the producer and consumer aspects. And in particular I'd like to concentrate on the effects on this project.
One particular quirk and complication is where the server responds with a
LEADER_NOT_AVAILABLEmessage. This seems to be reported as an error. Afterwards we sometimes seem to be able to send or receive messages, but often the code is unable to do so and the respective test eventually times-out.The logging by/via/through jmeter doesn't provide many clues. What is apparent is that the pepper-box code treats any exception as a failure and reports it as such. It doesn't not explicitly try to interpret or address what's happened.
From searching online I've yet to find much clear, coherent and current information. This error message has been around for many releases of Kafka, including the old APIs. I'm hoping we'll at least find ways to mitigate and recover-from this response in both our producer and consumer code.