search

sngular / kloadgen

KLoadGen is kafka load generator plugin for jmeter designed to work with AVRO, JSON and PROTOL-BUFFERS schema Registries.
http://sngular.com
Mozilla Public License 2.0
209 stars 42 forks source link

Auto registering schemas to SchemaRegistry #145

Closed harjis closed 2 years ago

harjis commented 2 years ago

Hi!

I was trying out how auto registering a schema to SchemaRegistry works but according my tests the schemas do not get sent to SchemaRegistry. My expectation was that if I have auto.register.schemas set to true the schema would be automatically sent to schema registry when I start producing messages with JMeter+Kloadgen. I'm also using Avro as the message format.

Is there something else I need to set in addition to auto.register.schema?

Thanks for your help!

The test plan I'm using is here:

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.4.1">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">1</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">1</stringProp>
        <stringProp name="ThreadGroup.ramp_time">5</stringProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
      </ThreadGroup>
      <hashTree>
        <net.coru.kloadgen.config.kafkaheaders.KafkaHeadersConfigElement guiclass="TestBeanGUI" testclass="net.coru.kloadgen.config.kafkaheaders.KafkaHeadersConfigElement" testname="Kafka Headers Config Element" enabled="true">
          <collectionProp name="kafkaHeaders">
            <elementProp name="" elementType="net.coru.kloadgen.model.HeaderMapping">
              <stringProp name="headerName">message_id</stringProp>
              <stringProp name="headerValue">uuid</stringProp>
            </elementProp>
            <elementProp name="" elementType="net.coru.kloadgen.model.HeaderMapping">
              <stringProp name="headerName">publication_timestamp</stringProp>
              <stringProp name="headerValue">stringTimestamp</stringProp>
            </elementProp>
            <elementProp name="" elementType="net.coru.kloadgen.model.HeaderMapping">
              <stringProp name="headerName">producer_id</stringProp>
              <stringProp name="headerValue">producer_id_1</stringProp>
            </elementProp>
            <elementProp name="" elementType="net.coru.kloadgen.model.HeaderMapping">
              <stringProp name="headerName">topic</stringProp>
              <stringProp name="headerValue">test.topic</stringProp>
            </elementProp>
            <elementProp name="" elementType="net.coru.kloadgen.model.HeaderMapping">
              <stringProp name="headerName">envelope_id</stringProp>
              <stringProp name="headerValue">envelope-id</stringProp>
            </elementProp>
            <elementProp name="" elementType="net.coru.kloadgen.model.HeaderMapping">
              <stringProp name="headerName">contentType</stringProp>
              <stringProp name="headerValue">application/*+avro</stringProp>
            </elementProp>
          </collectionProp>
        </net.coru.kloadgen.config.kafkaheaders.KafkaHeadersConfigElement>
        <hashTree/>
        <net.coru.kloadgen.config.valueserialized.ValueSerializedConfigElement guiclass="TestBeanGUI" testclass="net.coru.kloadgen.config.valueserialized.ValueSerializedConfigElement" testname="Value Schema Load Generator Config" enabled="true">
          <stringProp name="avroSubject"></stringProp>
          <collectionProp name="valueSchemaProperties">
            <elementProp name="" elementType="net.coru.kloadgen.model.FieldValueMapping">
              <stringProp name="fieldName">id</stringProp>
              <intProp name="valueLength">7</intProp>
              <stringProp name="fieldType">string</stringProp>
              <stringProp name="fieldValuesList">id_1,id_2,id_3</stringProp>
            </elementProp>
            <elementProp name="" elementType="net.coru.kloadgen.model.FieldValueMapping">
              <stringProp name="fieldName">name</stringProp>
              <stringProp name="fieldType">string</stringProp>
              <intProp name="valueLength">10</intProp>
              <stringProp name="fieldValuesList">name_1, name_2, name_3</stringProp>
            </elementProp>
          </collectionProp>
          <stringProp name="valueSubjectName">test</stringProp>
          <stringProp name="valueSchemaType">AVRO</stringProp>
          <stringProp name="valueNameStrategy">io.confluent.kafka.serializers.subject.RecordNameStrategy</stringProp>
          <stringProp name="valueSerializerConfiguration">net.coru.kloadgen.serializer.AvroSerializer</stringProp>
        </net.coru.kloadgen.config.valueserialized.ValueSerializedConfigElement>
        <hashTree/>
        <JavaSampler guiclass="JavaTestSamplerGui" testclass="JavaSampler" testname="Kafka Producer Properties" enabled="true">
          <elementProp name="arguments" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" enabled="true">
            <collectionProp name="Arguments.arguments">
              <elementProp name="bootstrap.servers" elementType="Argument">
                <stringProp name="Argument.name">bootstrap.servers</stringProp>
                <stringProp name="Argument.value">${__P(BootstrapServers)}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="zookeeper.servers" elementType="Argument">
                <stringProp name="Argument.name">zookeeper.servers</stringProp>
                <stringProp name="Argument.value"></stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="kafka.topic.name" elementType="Argument">
                <stringProp name="Argument.name">kafka.topic.name</stringProp>
                <stringProp name="Argument.value">test.topic</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="compression.type" elementType="Argument">
                <stringProp name="Argument.name">compression.type</stringProp>
                <stringProp name="Argument.value">none</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="batch.size" elementType="Argument">
                <stringProp name="Argument.name">batch.size</stringProp>
                <stringProp name="Argument.value">16384</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="linger.ms" elementType="Argument">
                <stringProp name="Argument.name">linger.ms</stringProp>
                <stringProp name="Argument.value">0</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="buffer.memory" elementType="Argument">
                <stringProp name="Argument.name">buffer.memory</stringProp>
                <stringProp name="Argument.value">33554432</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="acks" elementType="Argument">
                <stringProp name="Argument.name">acks</stringProp>
                <stringProp name="Argument.value">1</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="send.buffer.bytes" elementType="Argument">
                <stringProp name="Argument.name">send.buffer.bytes</stringProp>
                <stringProp name="Argument.value">131072</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="receive.buffer.bytes" elementType="Argument">
                <stringProp name="Argument.name">receive.buffer.bytes</stringProp>
                <stringProp name="Argument.value">32768</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="security.protocol" elementType="Argument">
                <stringProp name="Argument.name">security.protocol</stringProp>
                <stringProp name="Argument.value">SSL</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="kerberos.auth.enabled" elementType="Argument">
                <stringProp name="Argument.name">kerberos.auth.enabled</stringProp>
                <stringProp name="Argument.value">NO</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="jaas.enabled" elementType="Argument">
                <stringProp name="Argument.name">jaas.enabled</stringProp>
                <stringProp name="Argument.value">NO</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="java.security.auth.login.config" elementType="Argument">
                <stringProp name="Argument.name">java.security.auth.login.config</stringProp>
                <stringProp name="Argument.value">&lt;JAAS File Location&gt;</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="java.security.krb5.conf" elementType="Argument">
                <stringProp name="Argument.name">java.security.krb5.conf</stringProp>
                <stringProp name="Argument.value">&lt;krb5.conf location&gt;</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="sasl.kerberos.service.name" elementType="Argument">
                <stringProp name="Argument.name">sasl.kerberos.service.name</stringProp>
                <stringProp name="Argument.value">kafka</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="sasl.mechanism" elementType="Argument">
                <stringProp name="Argument.name">sasl.mechanism</stringProp>
                <stringProp name="Argument.value">GSSAPI</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.enabled" elementType="Argument">
                <stringProp name="Argument.name">ssl.enabled</stringProp>
                <stringProp name="Argument.value">YES</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.key.password" elementType="Argument">
                <stringProp name="Argument.name">ssl.key.password</stringProp>
                <stringProp name="Argument.value">${__P(SslKeyPassword)}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.keystore.location" elementType="Argument">
                <stringProp name="Argument.name">ssl.keystore.location</stringProp>
                <stringProp name="Argument.value">/secrets/jmeter.keystore.p12</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.keystore.password" elementType="Argument">
                <stringProp name="Argument.name">ssl.keystore.password</stringProp>
                <stringProp name="Argument.value">${__P(KeystorePassword)}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.truststore.location" elementType="Argument">
                <stringProp name="Argument.name">ssl.truststore.location</stringProp>
                <stringProp name="Argument.value">/secrets/jmeter.truststore.jks</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.truststore.password" elementType="Argument">
                <stringProp name="Argument.name">ssl.truststore.password</stringProp>
                <stringProp name="Argument.value">${__P(TruststorePassword)}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="client.id" elementType="Argument">
                <stringProp name="Argument.name">client.id</stringProp>
                <stringProp name="Argument.value"></stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="security.providers" elementType="Argument">
                <stringProp name="Argument.name">security.providers</stringProp>
                <stringProp name="Argument.value"></stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.enabled.protocols" elementType="Argument">
                <stringProp name="Argument.name">ssl.enabled.protocols</stringProp>
                <stringProp name="Argument.value">TLSv1.2,TLSv1.3</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.endpoint.identification.algorithm" elementType="Argument">
                <stringProp name="Argument.name">ssl.endpoint.identification.algorithm</stringProp>
                <stringProp name="Argument.value">&lt;Ssl identification algorithm&gt;</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.keymanager.algorithm" elementType="Argument">
                <stringProp name="Argument.name">ssl.keymanager.algorithm</stringProp>
                <stringProp name="Argument.value">SunX509</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.keystore.type" elementType="Argument">
                <stringProp name="Argument.name">ssl.keystore.type</stringProp>
                <stringProp name="Argument.value">PKCS12</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.provider" elementType="Argument">
                <stringProp name="Argument.name">ssl.provider</stringProp>
                <stringProp name="Argument.value"></stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="ssl.protocol" elementType="Argument">
                <stringProp name="Argument.name">ssl.protocol</stringProp>
                <stringProp name="Argument.value">TLSv1.3</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="auto.register.schemas" elementType="Argument">
                <stringProp name="Argument.name">auto.register.schemas</stringProp>
                <stringProp name="Argument.value">true</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>
          <stringProp name="classname">net.coru.kloadgen.sampler.KafkaSchemaSampler</stringProp>
        </JavaSampler>
        <hashTree/>
        <net.coru.kloadgen.config.schemaregistry.SchemaRegistryConfigElement guiclass="TestBeanGUI" testclass="net.coru.kloadgen.config.schemaregistry.SchemaRegistryConfigElement" testname="Schema Registry Config Element" enabled="true">
          <stringProp name="schemaRegistryUrl">${__P(SchemaRegistryUrl)}</stringProp>
          <collectionProp name="schemaRegistryProperties">
            <elementProp name="Config properties" elementType="net.coru.kloadgen.model.PropertyMapping" testname="Config properties">
              <stringProp name="propertyName">schema.registry.auth.enabled</stringProp>
              <stringProp name="propertyValue">${__P(SchemaRegistryAuthEnabled)}</stringProp>
            </elementProp>
            <elementProp name="Config properties" elementType="net.coru.kloadgen.model.PropertyMapping" testname="Config properties">
              <stringProp name="propertyName">schema.registry.auth.method</stringProp>
              <stringProp name="propertyValue">BASIC</stringProp>
            </elementProp>
            <elementProp name="Config properties" elementType="net.coru.kloadgen.model.PropertyMapping" testname="Config properties">
              <stringProp name="propertyName">schema.registry.username</stringProp>
              <stringProp name="propertyValue">${__P(SchemaRegistryUsername)}</stringProp>
            </elementProp>
            <elementProp name="Config properties" elementType="net.coru.kloadgen.model.PropertyMapping" testname="Config properties">
              <stringProp name="propertyName">schema.registry.password</stringProp>
              <stringProp name="propertyValue">${__P(SchemaRegistryPassword)}</stringProp>
            </elementProp>
            <elementProp name="Config properties" elementType="net.coru.kloadgen.model.PropertyMapping" testname="Config properties">
              <stringProp name="propertyName">schema.registry.bearer</stringProp>
              <stringProp name="propertyValue">&lt;bearer&gt;</stringProp>
            </elementProp>
          </collectionProp>
        </net.coru.kloadgen.config.schemaregistry.SchemaRegistryConfigElement>
        <hashTree/>
        <net.coru.kloadgen.config.keysimple.KeySimpleConfigElement guiclass="TestBeanGUI" testclass="net.coru.kloadgen.config.keysimple.KeySimpleConfigElement" testname="Key Plain Load Generator Config" enabled="true">
          <stringProp name="keySerializerConfiguration">org.apache.kafka.common.serialization.StringSerializer</stringProp>
          <stringProp name="keyType">string</stringProp>
          <stringProp name="keyValue">1</stringProp>
        </net.coru.kloadgen.config.keysimple.KeySimpleConfigElement>
        <hashTree/>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>
kszosze commented 2 years ago

Hi @harjis,

it should work as you expected. I'll check it out asap, may you hit a bug.

Cheers!!

kszosze commented 2 years ago

HI @harjis,

I found the problem and I have a solution in the branch associated to this issue. Do you mind testing it if it works for you?

Thanks!

harjis commented 2 years ago

Hi @kszosze!

Thanks for the quick response! I tried out the branch and on my computer it seems that it is not working as intended. I'll try to do some testing still to see if the problem is at my end. When I start producing messages I get the same error message as before.

Full log is this:

2021-12-27 09:45:20,788 ERROR n.c.k.l.i.AvroLoadGenerator: Please make sure that properties data type and expression function return type are compatible with each other
io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject 'test' not found.; error code: 40401
    at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:292) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:351) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at io.confluent.kafka.schemaregistry.client.rest.RestService.getLatestVersion(RestService.java:716) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at io.confluent.kafka.schemaregistry.client.rest.RestService.getLatestVersion(RestService.java:707) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getLatestSchemaMetadata(CachedSchemaRegistryClient.java:332) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.getSchemaBySubject(AvroLoadGenerator.java:69) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.retrieveSchema(AvroLoadGenerator.java:65) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.setUpGenerator(AvroLoadGenerator.java:42) [kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.sampler.SamplerUtil.configureValueGenerator(SamplerUtil.java:388) [kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.sampler.KafkaSchemaSampler.setupTest(KafkaSchemaSampler.java:92) [kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:194) [ApacheJMeter_java.jar:5.5-SNAPSHOT]
    at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:651) [ApacheJMeter_core.jar:5.5-SNAPSHOT]
    at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:570) [ApacheJMeter_core.jar:5.5-SNAPSHOT]
    at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:501) [ApacheJMeter_core.jar:5.5-SNAPSHOT]
    at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:268) [ApacheJMeter_core.jar:5.5-SNAPSHOT]
    at java.lang.Thread.run(Thread.java:829) [?:?]
2021-12-27 09:45:20,801 ERROR o.a.j.t.JMeterThread: Error while processing sampler: 'Kafka Producer Properties'.
net.coru.kloadgen.exception.KLoadGenException: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject 'test' not found.; error code: 40401
    at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.setUpGenerator(AvroLoadGenerator.java:45) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.sampler.SamplerUtil.configureValueGenerator(SamplerUtil.java:388) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.sampler.KafkaSchemaSampler.setupTest(KafkaSchemaSampler.java:92) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:194) ~[ApacheJMeter_java.jar:5.5-SNAPSHOT]
    at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:651) ~[ApacheJMeter_core.jar:5.5-SNAPSHOT]
    at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:570) ~[ApacheJMeter_core.jar:5.5-SNAPSHOT]
    at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:501) [ApacheJMeter_core.jar:5.5-SNAPSHOT]
    at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:268) [ApacheJMeter_core.jar:5.5-SNAPSHOT]
    at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject 'test' not found.; error code: 40401
    at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:292) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:351) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at io.confluent.kafka.schemaregistry.client.rest.RestService.getLatestVersion(RestService.java:716) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at io.confluent.kafka.schemaregistry.client.rest.RestService.getLatestVersion(RestService.java:707) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getLatestSchemaMetadata(CachedSchemaRegistryClient.java:332) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.getSchemaBySubject(AvroLoadGenerator.java:69) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.retrieveSchema(AvroLoadGenerator.java:65) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    at net.coru.kloadgen.loadgen.impl.AvroLoadGenerator.setUpGenerator(AvroLoadGenerator.java:42) ~[kloadgen-3.5.7-kloadgen-plugin-3.5.7.jar:?]
    ... 8 more
kszosze commented 2 years ago

Hi @harjis, I just made some adjustments in the branch. Do you min d to test it?

Cheers

harjis commented 2 years ago

Hi @kszosze!

I tested the branch and it seems to be working as intended! I got it working by using ValueFileSerializedConfigElement. Before I was using ValueSerializedConfigElement but is it so that this doesn't have the ability to send schema to schema registry if it doesn't exist?

I noticed that ValueFileSerializedConfigElement in Example-Test-Plan-2.jmx contains avroSchema, avroSubject and schemaProperties but if I read the code correctly, these are not used currently. If it is so I guess those could be removed to avoid confusion 🙂

Thanks for your help!

kszosze commented 2 years ago

HI @harjis, That's make sense, since ValueSerializedConfigElement depends on you have a Schema Registry up and running, while ValueFileSerializedConfigElement don't; you just upload the avsc file and work with it.

I'm going to check the example test plans and updated, may be some of them got a bit old with recent developments.

Thanks for the feedback!!