search

authorjapps / zerocode

A community-developed, free, opensource, automated testing framework for microservices APIs, Kafka(Data Streams) and Load testing. Zerocode Open Source enables you to create, change and maintain your automated test scenarios via simple JSON or YAML files. Visit documentation below:
https://zerocode-tdd.tddfy.com
Apache License 2.0
907 stars 398 forks source link

Is it possible to output messages to Kafka / Avro without using Kafka REST Proxy? #497

Closed erobertolima121 closed 3 years ago

erobertolima121 commented 3 years ago

Hello. Is it possible to send messages to Kafka / Avro without using Kafka REST Proxy? Directly on topic using AVRO?

sparrowV commented 3 years ago

Welcome @erobertolima121 . Can you elaborate more on what you are trying to achieve? You can send message to kafka in zerocode scenario step.

erobertolima121 commented 3 years ago

Welcome @erobertolima121 . Can you elaborate more on what you are trying to achieve? You can send message to kafka in zerocode scenario step.

Hello @sparrowV, follows a more elaborate explanation.

I have Kafka and Schema Registry Embedded, with the following AVRO schema registered. I don't have Kafka REST Proxy:

schema_avro

I performed the creation of the following configurations and test to post the message directly in the topic meeting the defined avro contract:

application_kafka.properties

kafka.bootstrap.servers=127.0.0.1:9092
kafka.producer.properties=application_kafka_producer.properties

application_kafka_producer.properties

kafka.acks=all
retries=10
key.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer
value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer
schema.registry.url=http://localhost:8081

KafkaAvroProducer.java

package br.com.robligo;

import org.jsmart.zerocode.core.domain.Scenario;
import org.jsmart.zerocode.core.domain.TargetEnv;
import org.jsmart.zerocode.core.runner.ZeroCodeUnitRunner;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(ZeroCodeUnitRunner.class)
@TargetEnv("application_kafka.properties")
public class KafkaAvroProducer {

    @Test
    @Scenario("rest/kafka_avro_test.json")
    public void test_post_message_kafka_avro() {
    }

}

kafka_avro_test.json

{
  "scenarioName": "test to validate kafka message production with avro",
  "steps": [
    {
      "name": "validate_production_user_success",
      "url": "kafka-topic:inbound-service",
      "operation": "PRODUCE",
      "request": {
        "records": [
          {
              "fields": [
                {
                  "mainName": "Name"
                },
                {
                  "hometown": "City"
                }
              ]
            }
        ]
      },
      "assertions": {
        "status": "Ok"
      }
    }
  ]
}

But when the test is run, the message production is performed with the null value:

consumer_null

I leave the test run log to help:

Log execution:

C:\Users\Public\Java\jdk-11.0.11\bin\java.exe -ea -Didea.test.cyclic.buffer.size=1048576 "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.1.1\lib\idea_rt.jar=58932:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.1.1\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.1.1\lib\idea_rt.jar;C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.1.1\plugins\junit\lib\junit5-rt.jar;C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2021.1.1\plugins\junit\lib\junit-rt.jar;G:\QA\zerocode\exemplo\target\test-classes;G:\QA\zerocode\exemplo\target\classes;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-starter-web\2.5.0\spring-boot-starter-web-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-starter\2.5.0\spring-boot-starter-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot\2.5.0\spring-boot-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-autoconfigure\2.5.0\spring-boot-autoconfigure-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-starter-logging\2.5.0\spring-boot-starter-logging-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\logging\log4j\log4j-to-slf4j\2.14.1\log4j-to-slf4j-2.14.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\logging\log4j\log4j-api\2.14.1\log4j-api-2.14.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\slf4j\jul-to-slf4j\1.7.30\jul-to-slf4j-1.7.30.jar;C:\Users\Roberto L Gonçalves\.m2\repository\jakarta\annotation\jakarta.annotation-api\1.3.5\jakarta.annotation-api-1.3.5.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\yaml\snakeyaml\1.28\snakeyaml-1.28.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-starter-json\2.5.0\spring-boot-starter-json-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.12.3\jackson-datatype-jsr310-2.12.3.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.12.3\jackson-module-parameter-names-2.12.3.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-starter-tomcat\2.5.0\spring-boot-starter-tomcat-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\tomcat\embed\tomcat-embed-core\9.0.46\tomcat-embed-core-9.0.46.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\tomcat\embed\tomcat-embed-el\9.0.46\tomcat-embed-el-9.0.46.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\tomcat\embed\tomcat-embed-websocket\9.0.46\tomcat-embed-websocket-9.0.46.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-web\5.3.7\spring-web-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-beans\5.3.7\spring-beans-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-webmvc\5.3.7\spring-webmvc-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-aop\5.3.7\spring-aop-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-context\5.3.7\spring-context-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-expression\5.3.7\spring-expression-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-starter-test\2.5.0\spring-boot-starter-test-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-test\2.5.0\spring-boot-test-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\boot\spring-boot-test-autoconfigure\2.5.0\spring-boot-test-autoconfigure-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\jayway\jsonpath\json-path\2.5.0\json-path-2.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\net\minidev\json-smart\2.3\json-smart-2.3.jar;C:\Users\Roberto L Gonçalves\.m2\repository\net\minidev\accessors-smart\1.2\accessors-smart-1.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\ow2\asm\asm\5.0.4\asm-5.0.4.jar;C:\Users\Roberto L Gonçalves\.m2\repository\jakarta\xml\bind\jakarta.xml.bind-api\2.3.3\jakarta.xml.bind-api-2.3.3.jar;C:\Users\Roberto L Gonçalves\.m2\repository\jakarta\activation\jakarta.activation-api\1.2.2\jakarta.activation-api-1.2.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\assertj\assertj-core\3.19.0\assertj-core-3.19.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\hamcrest\hamcrest\2.2\hamcrest-2.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\junit\jupiter\junit-jupiter\5.7.2\junit-jupiter-5.7.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\junit\jupiter\junit-jupiter-api\5.7.2\junit-jupiter-api-5.7.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apiguardian\apiguardian-api\1.1.0\apiguardian-api-1.1.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\opentest4j\opentest4j\1.2.0\opentest4j-1.2.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\junit\platform\junit-platform-commons\1.7.2\junit-platform-commons-1.7.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\junit\jupiter\junit-jupiter-params\5.7.2\junit-jupiter-params-5.7.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\junit\jupiter\junit-jupiter-engine\5.7.2\junit-jupiter-engine-5.7.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\junit\platform\junit-platform-engine\1.7.2\junit-platform-engine-1.7.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\mockito\mockito-core\3.9.0\mockito-core-3.9.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\net\bytebuddy\byte-buddy\1.10.20\byte-buddy-1.10.20.jar;C:\Users\Roberto L Gonçalves\.m2\repository\net\bytebuddy\byte-buddy-agent\1.10.20\byte-buddy-agent-1.10.20.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\objenesis\objenesis\3.2\objenesis-3.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\mockito\mockito-junit-jupiter\3.9.0\mockito-junit-jupiter-3.9.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\skyscreamer\jsonassert\1.5.0\jsonassert-1.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\vaadin\external\google\android-json\0.0.20131108.vaadin1\android-json-0.0.20131108.vaadin1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-core\5.3.7\spring-core-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-jcl\5.3.7\spring-jcl-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\springframework\spring-test\5.3.7\spring-test-5.3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\xmlunit\xmlunit-core\2.8.2\xmlunit-core-2.8.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\jsmart\zerocode-tdd\1.3.28\zerocode-tdd-1.3.28.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\fasterxml\jackson\dataformat\jackson-dataformat-yaml\2.9.8\jackson-dataformat-yaml-2.9.8.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\univocity\univocity-parsers\2.8.2\univocity-parsers-2.8.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\code\gson\gson\2.6.2\gson-2.6.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\json\json\20160810\json-20160810.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\velocity\velocity\1.7\velocity-1.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\commons-collections\commons-collections\3.2.1\commons-collections-3.2.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\fasterxml\jackson\dataformat\jackson-dataformat-csv\2.9.8\jackson-dataformat-csv-2.9.8.jar;C:\Users\Roberto L Gonçalves\.m2\repository\commons-io\commons-io\2.4\commons-io-2.4.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\github\tomakehurst\wiremock\2.19.0\wiremock-2.19.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-server\9.2.24.v20180105\jetty-server-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\javax\servlet\javax.servlet-api\3.1.0\javax.servlet-api-3.1.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-http\9.2.24.v20180105\jetty-http-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-io\9.2.24.v20180105\jetty-io-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-servlet\9.2.24.v20180105\jetty-servlet-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-security\9.2.24.v20180105\jetty-security-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-servlets\9.2.24.v20180105\jetty-servlets-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-continuation\9.2.24.v20180105\jetty-continuation-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-util\9.2.24.v20180105\jetty-util-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-webapp\9.2.24.v20180105\jetty-webapp-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\eclipse\jetty\jetty-xml\9.2.24.v20180105\jetty-xml-9.2.24.v20180105.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\xmlunit\xmlunit-legacy\2.5.1\xmlunit-legacy-2.5.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\net\sf\jopt-simple\jopt-simple\5.0.3\jopt-simple-5.0.3.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\commons\commons-lang3\3.7\commons-lang3-3.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\flipkart\zjsonpatch\zjsonpatch\0.4.4\zjsonpatch-0.4.4.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\github\jknack\handlebars\4.0.7\handlebars-4.0.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\antlr\antlr4-runtime\4.7.1\antlr4-runtime-4.7.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\github\jknack\handlebars-helpers\4.0.7\handlebars-helpers-4.0.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\ch\qos\logback\logback-classic\1.0.7\logback-classic-1.0.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\ch\qos\logback\logback-core\1.0.7\logback-core-1.0.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\jboss\resteasy\resteasy-jaxrs\2.2.1.GA\resteasy-jaxrs-2.2.1.GA.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\jboss\resteasy\jaxrs-api\2.2.1.GA\jaxrs-api-2.2.1.GA.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\scannotation\scannotation\1.0.3\scannotation-1.0.3.jar;C:\Users\Roberto L Gonçalves\.m2\repository\javassist\javassist\3.12.1.GA\javassist-3.12.1.GA.jar;C:\Users\Roberto L Gonçalves\.m2\repository\javax\annotation\jsr250-api\1.0\jsr250-api-1.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\javax\activation\activation\1.1\activation-1.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\commons-httpclient\commons-httpclient\3.1\commons-httpclient-3.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\net\jcip\jcip-annotations\1.0\jcip-annotations-1.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\commons-lang\commons-lang\2.6\commons-lang-2.6.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\classpath-explorer\classpath-explorer\1.0\classpath-explorer-1.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\jukito\jukito\1.4.1\jukito-1.4.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\inject\extensions\guice-assistedinject\3.0\guice-assistedinject-3.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\inject\guice\4.0\guice-4.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\javax\inject\javax.inject\1\javax.inject-1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\aopalliance\aopalliance\1.0\aopalliance-1.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\guava\guava\23.0\guava-23.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\errorprone\error_prone_annotations\2.0.18\error_prone_annotations-2.0.18.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\j2objc\j2objc-annotations\1.1\j2objc-annotations-1.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\codehaus\mojo\animal-sniffer-annotations\1.14\animal-sniffer-annotations-1.14.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.10.0\jackson-annotations-2.10.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\fasterxml\jackson\core\jackson-core\2.10.0\jackson-core-2.10.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.10.0\jackson-databind-2.10.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.10.0\jackson-datatype-jdk8-2.10.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\httpcomponents\httpclient\4.5\httpclient-4.5.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\httpcomponents\httpcore\4.4.1\httpcore-4.4.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\commons-codec\commons-codec\1.9\commons-codec-1.9.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\aventstack\extentreports\4.0.9\extentreports-4.0.9.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\freemarker\freemarker\2.3.23\freemarker-2.3.23.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\mongodb\mongodb-driver\3.3.0\mongodb-driver-3.3.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\mongodb\mongodb-driver-core\3.3.0\mongodb-driver-core-3.3.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\mongodb\bson\3.3.0\bson-3.3.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\httpcomponents\httpmime\4.5.2\httpmime-4.5.2.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\protobuf\protobuf-java\3.13.0\protobuf-java-3.13.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\google\protobuf\protobuf-java-util\3.13.0\protobuf-java-util-3.13.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\junit\junit\4.12\junit-4.12.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\kafka\kafka-clients\2.8.0\kafka-clients-2.8.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\github\luben\zstd-jni\1.4.9-1\zstd-jni-1.4.9-1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\lz4\lz4-java\1.7.1\lz4-java-1.7.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\xerial\snappy\snappy-java\1.1.8.1\snappy-java-1.1.8.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\slf4j\slf4j-api\1.7.30\slf4j-api-1.7.30.jar;C:\Users\Roberto L Gonçalves\.m2\repository\io\confluent\kafka-avro-serializer\5.3.0\kafka-avro-serializer-5.3.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\avro\avro\1.8.1\avro-1.8.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\codehaus\jackson\jackson-core-asl\1.9.13\jackson-core-asl-1.9.13.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\codehaus\jackson\jackson-mapper-asl\1.9.13\jackson-mapper-asl-1.9.13.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\thoughtworks\paranamer\paranamer\2.7\paranamer-2.7.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\commons\commons-compress\1.8.1\commons-compress-1.8.1.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\tukaani\xz\1.5\xz-1.5.jar;C:\Users\Roberto L Gonçalves\.m2\repository\io\confluent\kafka-schema-registry-client\5.3.0\kafka-schema-registry-client-5.3.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\io\confluent\common-config\5.3.0\common-config-5.3.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\io\confluent\common-utils\5.3.0\common-utils-5.3.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\zookeeper\zookeeper\3.4.14\zookeeper-3.4.14.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\github\spotbugs\spotbugs-annotations\3.1.9\spotbugs-annotations-3.1.9.jar;C:\Users\Roberto L Gonçalves\.m2\repository\jline\jline\0.9.94\jline-0.9.94.jar;C:\Users\Roberto L Gonçalves\.m2\repository\org\apache\yetus\audience-annotations\0.5.0\audience-annotations-0.5.0.jar;C:\Users\Roberto L Gonçalves\.m2\repository\io\netty\netty\3.10.6.Final\netty-3.10.6.Final.jar;C:\Users\Roberto L Gonçalves\.m2\repository\com\101tec\zkclient\0.10\zkclient-0.10.jar" com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit4 br.com.robligo.KafkaAvroProducer,test_post_message_kafka_avro
21:01:26,902 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
21:01:26,903 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
21:01:26,904 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/C:/Users/Roberto%20L%20Gon%c3%a7alves/.m2/repository/org/jsmart/zerocode-tdd/1.3.28/zerocode-tdd-1.3.28.jar!/logback.xml]
21:01:26,920 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList@68e5eea7 - URL [jar:file:/C:/Users/Roberto%20L%20Gon%c3%a7alves/.m2/repository/org/jsmart/zerocode-tdd/1.3.28/zerocode-tdd-1.3.28.jar!/logback.xml] is not of type file
21:01:27,013 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
21:01:27,027 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.FileAppender]
21:01:27,036 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [fileAppender]
21:01:27,082 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:01:27,162 |-INFO in ch.qos.logback.core.FileAppender[fileAppender] - File property is set to [target/logs/zerocode_rest_bdd_logs.log]
21:01:27,163 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
21:01:27,165 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [consoleAppender]
21:01:27,175 |-WARN in ch.qos.logback.core.ConsoleAppender[consoleAppender] - This appender no longer admits a layout as a sub-component, set an encoder instead.
21:01:27,175 |-WARN in ch.qos.logback.core.ConsoleAppender[consoleAppender] - To ensure compatibility, wrapping your layout in LayoutWrappingEncoder.
21:01:27,175 |-WARN in ch.qos.logback.core.ConsoleAppender[consoleAppender] - See also http://logback.qos.ch/codes.html#layoutInsteadOfEncoder for details
21:01:27,175 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
21:01:27,175 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [fileAppender] to Logger[ROOT]
21:01:27,176 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [consoleAppender] to Logger[ROOT]
21:01:27,176 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
21:01:27,177 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@291b4bf5 - Registering current configuration as safe fallback point

2021-06-07 21:01:27,216 [main] INFO org.jsmart.zerocode.core.utils.RunnerUtils - ### testClass : class br.com.robligo.KafkaAvroProducer
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/C:/Users/Roberto%20L%20Gon%c3%a7alves/.m2/repository/com/google/inject/guice/4.0/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2021-06-07 21:01:27,990 [main] INFO org.jsmart.zerocode.core.utils.RunnerUtils - ### testClass : class br.com.robligo.KafkaAvroProducer
2021-06-07 21:01:28,075 [main] INFO org.jsmart.zerocode.core.utils.RunnerUtils - ### testClass : class br.com.robligo.KafkaAvroProducer
2021-06-07 21:01:28,093 [main] INFO org.jsmart.zerocode.core.runner.ZeroCodeUnitRunner - System property zerocode.junit=null
2021-06-07 21:01:28,209 [main] INFO org.jsmart.zerocode.core.runner.ZeroCodeMultiStepsScenarioRunnerImpl - 
-------------------------- BDD: Scenario:test to validate kafka message production with avro -------------------------

2021-06-07 21:01:28,218 [main] INFO org.jsmart.zerocode.core.runner.ZeroCodeMultiStepsScenarioRunnerImpl - 
-------------------------------------------------------------------------
     Executing Scenario Count No. or parameter No. or Row No. | 0 | 
-------------------------------------------------------------------------
2021-06-07 21:01:28,546 [main] INFO org.jsmart.zerocode.core.kafka.helper.KafkaConsumerHelper - 
---------------------------------------------------------
kafka.bootstrap.servers - 127.0.0.1:9092
---------------------------------------------------------
2021-06-07 21:01:28,547 [main] INFO org.jsmart.zerocode.core.kafka.client.BasicKafkaClient - brokers:127.0.0.1:9092, topicName:inbound-service, operation:PRODUCE, requestJson:{"records":[{"fields":[{"mainName":"Name"},{"hometown":"City"}]}]}
2021-06-07 21:01:28,580 [main] INFO org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values: 
    acks = 1
    batch.size = 16384
    bootstrap.servers = [127.0.0.1:9092]
    buffer.memory = 33554432
    client.dns.lookup = use_all_dns_ips
    client.id = producer-1
    compression.type = none
    connections.max.idle.ms = 540000
    delivery.timeout.ms = 120000
    enable.idempotence = false
    interceptor.classes = []
    internal.auto.downgrade.txn.commit = false
    key.serializer = class io.confluent.kafka.serializers.KafkaAvroSerializer
    linger.ms = 0
    max.block.ms = 60000
    max.in.flight.requests.per.connection = 5
    max.request.size = 1048576
    metadata.max.age.ms = 300000
    metadata.max.idle.ms = 300000
    metric.reporters = []
    metrics.num.samples = 2
    metrics.recording.level = INFO
    metrics.sample.window.ms = 30000
    partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
    receive.buffer.bytes = 32768
    reconnect.backoff.max.ms = 1000
    reconnect.backoff.ms = 50
    request.timeout.ms = 30000
    retries = 10
    retry.backoff.ms = 100
    sasl.client.callback.handler.class = null
    sasl.jaas.config = null
    sasl.kerberos.kinit.cmd = /usr/bin/kinit
    sasl.kerberos.min.time.before.relogin = 60000
    sasl.kerberos.service.name = null
    sasl.kerberos.ticket.renew.jitter = 0.05
    sasl.kerberos.ticket.renew.window.factor = 0.8
    sasl.login.callback.handler.class = null
    sasl.login.class = null
    sasl.login.refresh.buffer.seconds = 300
    sasl.login.refresh.min.period.seconds = 60
    sasl.login.refresh.window.factor = 0.8
    sasl.login.refresh.window.jitter = 0.05
    sasl.mechanism = GSSAPI
    security.protocol = PLAINTEXT
    security.providers = null
    send.buffer.bytes = 131072
    socket.connection.setup.timeout.max.ms = 30000
    socket.connection.setup.timeout.ms = 10000
    ssl.cipher.suites = null
    ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
    ssl.endpoint.identification.algorithm = https
    ssl.engine.factory.class = null
    ssl.key.password = null
    ssl.keymanager.algorithm = SunX509
    ssl.keystore.certificate.chain = null
    ssl.keystore.key = null
    ssl.keystore.location = null
    ssl.keystore.password = null
    ssl.keystore.type = JKS
    ssl.protocol = TLSv1.3
    ssl.provider = null
    ssl.secure.random.implementation = null
    ssl.trustmanager.algorithm = PKIX
    ssl.truststore.certificates = null
    ssl.truststore.location = null
    ssl.truststore.password = null
    ssl.truststore.type = JKS
    transaction.timeout.ms = 60000
    transactional.id = null
    value.serializer = class io.confluent.kafka.serializers.KafkaAvroSerializer

2021-06-07 21:01:28,675 [main] INFO io.confluent.kafka.serializers.KafkaAvroSerializerConfig - KafkaAvroSerializerConfig values: 
    bearer.auth.token = [hidden]
    schema.registry.url = [http://localhost:8081]
    basic.auth.user.info = [hidden]
    auto.register.schemas = true
    max.schemas.per.subject = 1000
    basic.auth.credentials.source = URL
    schema.registry.basic.auth.user.info = [hidden]
    bearer.auth.credentials.source = STATIC_TOKEN
    value.subject.name.strategy = class io.confluent.kafka.serializers.subject.TopicNameStrategy
    key.subject.name.strategy = class io.confluent.kafka.serializers.subject.TopicNameStrategy

2021-06-07 21:01:28,685 [main] INFO io.confluent.kafka.serializers.KafkaAvroSerializerConfig - KafkaAvroSerializerConfig values: 
    bearer.auth.token = [hidden]
    schema.registry.url = [http://localhost:8081]
    basic.auth.user.info = [hidden]
    auto.register.schemas = true
    max.schemas.per.subject = 1000
    basic.auth.credentials.source = URL
    schema.registry.basic.auth.user.info = [hidden]
    bearer.auth.credentials.source = STATIC_TOKEN
    value.subject.name.strategy = class io.confluent.kafka.serializers.subject.TopicNameStrategy
    key.subject.name.strategy = class io.confluent.kafka.serializers.subject.TopicNameStrategy

2021-06-07 21:01:28,750 [main] WARN org.apache.kafka.clients.producer.ProducerConfig - The configuration 'kafka.acks' was supplied but isn't a known config.
2021-06-07 21:01:28,758 [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 2.8.0
2021-06-07 21:01:28,758 [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: ebb1d6e21cc92130
2021-06-07 21:01:28,758 [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1623110488751
2021-06-07 21:01:28,813 [main] WARN org.jsmart.zerocode.core.kafka.helper.KafkaProducerHelper - Could not find path '$.recordType' in the request. returned default type 'RAW'.
2021-06-07 21:01:28,827 [main] INFO org.jsmart.zerocode.core.kafka.send.KafkaSender - Sending record number: 0

2021-06-07 21:01:28,832 [main] INFO org.jsmart.zerocode.core.kafka.send.KafkaSender - Synchronous Producer sending record - ProducerRecord(topic=inbound-service, partition=null, headers=RecordHeaders(headers = [], isReadOnly = false), key=null, value=null, timestamp=null)
2021-06-07 21:01:29,237 [kafka-producer-network-thread | producer-1] INFO org.apache.kafka.clients.Metadata - [Producer clientId=producer-1] Cluster ID: U5UXNvkrRcqwMof4MbzPuQ
2021-06-07 21:01:29,447 [main] INFO org.jsmart.zerocode.core.kafka.send.KafkaSender - Record was sent to partition- 0, with offset- 1 
2021-06-07 21:01:29,464 [main] INFO org.jsmart.zerocode.core.kafka.send.KafkaSender - deliveryDetails- {"status":"Ok","recordMetadata":{"offset":1,"timestamp":1623110489238,"serializedKeySize":-1,"serializedValueSize":-1,"topicPartition":{"hash":874692642,"partition":0,"topic":"inbound-service"}}}
2021-06-07 21:01:29,465 [main] INFO org.apache.kafka.clients.producer.KafkaProducer - [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
2021-06-07 21:01:29,471 [main] INFO org.apache.kafka.common.metrics.Metrics - Metrics scheduler closed
2021-06-07 21:01:29,471 [main] INFO org.apache.kafka.common.metrics.Metrics - Closing reporter org.apache.kafka.common.metrics.JmxReporter
2021-06-07 21:01:29,472 [main] INFO org.apache.kafka.common.metrics.Metrics - Metrics reporters closed
2021-06-07 21:01:29,473 [main] INFO org.apache.kafka.common.utils.AppInfoParser - App info kafka.producer for producer-1 unregistered
2021-06-07 21:01:29,478 [main] INFO org.jsmart.zerocode.core.engine.validators.ZeroCodeValidatorImpl - Comparing results via LENIENT matchers
2021-06-07 21:01:29,487 [main] INFO org.jsmart.zerocode.core.runner.StepNotificationHandler - 
***Step PASSED - Scenario:test to validate kafka message production with avro -> validate_production_user_success
2021-06-07 21:01:29,488 [main] WARN org.jsmart.zerocode.core.runner.ZeroCodeMultiStepsScenarioRunnerImpl - 
--------- TEST-STEP-CORRELATION-ID: 55b54a43-1ea6-4853-96df-3d791b08c09d ---------
*requestTimeStamp:2021-06-07T21:01:28.542502200
step:validate_production_user_success
id:null
url:kafka-topic:inbound-service
method:PRODUCE
request:
{
  "records" : [ {
    "fields" : [ {
      "mainName" : "Name"
    }, {
      "hometown" : "City"
    } ]
  } ]
} 
--------- TEST-STEP-CORRELATION-ID: 55b54a43-1ea6-4853-96df-3d791b08c09d ---------
Response:
{
  "status" : "Ok",
  "recordMetadata" : {
    "offset" : 1,
    "timestamp" : 1623110489238,
    "serializedKeySize" : -1,
    "serializedValueSize" : -1,
    "topicPartition" : {
      "hash" : 874692642,
      "partition" : 0,
      "topic" : "inbound-service"
    }
  }
}
*responseTimeStamp:2021-06-07T21:01:29.478502800 
*Response delay:936.0 milli-secs 
---------> Expected Response: <----------
{
  "status" : "Ok"
} 

-done-

2021-06-07 21:01:29,618 [main] INFO org.jsmart.zerocode.core.runner.ZeroCodeUnitRunner - 
**FINISHED executing all Steps for [test to validate kafka message production with avro] **.
Steps were:[validate_production_user_success]
2021-06-07 21:01:29,620 [main] INFO org.jsmart.zerocode.core.engine.listener.ZeroCodeTestReportListener - #ZeroCode: Test run completed for this runner. Generating test reports and charts. 
* For more examples and help on automated Kafka data stream testing and Load testing visit https://zerocode.io
2021-06-07 21:01:29,729 [main] INFO org.jsmart.zerocode.core.domain.builders.ExtentReportsFactory - Where were the tests fired? Ans: OS:Windows 10, Architecture:amd64, Java:11.0.11, Vendor:Oracle Corporation

Process finished with exit code 0

If there is a need for more information, I am available, Thanks a lot.

sparrowV commented 3 years ago

@erobertolima121 Your zerocode producer is incorrect. You need to have key/value pairs in the records array. Take a look at the sample producer example here

sparrowV commented 3 years ago

@erobertolima121 , if you don't face the issue anymore, we can close this ticket

erobertolima121 commented 3 years ago

Hi, @sparrowV

I performed the suggested configuration and the message was posted correctly.

Thanks a lot.