pegasystems / dataset-integrations

Kafka custom serializer/deserializer implementations
Apache License 2.0
5 stars 6 forks source link

Issue while consuming messages #1

Closed ilyasahmed24 closed 4 years ago

ilyasahmed24 commented 4 years ago

Hi,

I tried doing a small POC on this project and when I try to consume a message it gives me the error I have set it up according to the instructions. Also when I had done the import of libs it did not take so I had to store it in the lib folder of the prweb and it worked after that. But then got this error. kafka schema

Error line in java code -- delegateValueSerializer.configure(additionalConfiguration, false);

Error message

Caused by: java.lang.NoClassDefFoundError: Could not initialize class io.confluent.kafka.schemaregistry.client.rest.RestService at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.(CachedSchemaRegistryClient.java:104) ~[kafka-schema-registry-client-5.2.1.jar:?] at io.confluent.kafka.serializers.AbstractKafkaAvroSerDe.configureClientProperties(AbstractKafkaAvroSerDe.java:62) ~[kafka-avro-serializer-5.3.1.jar:?] at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.configure(AbstractKafkaAvroSerializer.java:42) ~[kafka-avro-serializer-5.3.1.jar:?] at io.confluent.kafka.serializers.KafkaAvroSerializer.configure(KafkaAvroSerializer.java:48) ~[kafka-avro-serializer-5.3.1.jar:?] at com.pega.integration.kafka.AvroSchemaRegistrySerde.configure(AvroSchemaRegistrySerde.java:90) ~[PegaAvroSerde.jar:?] at com.pega.dsm.dnode.api.dataset.kafka.KafkaDataSet$KafkaDataSetBuilder.build(KafkaDataSet.java:305) ~[d-node.jar:?] at com.pega.dsm.dnode.api.dataset.kafka.KafkaDataSetFactory.createFromPage(KafkaDataSetFactory.java:94) ~[d-node.jar:?] at com.pega.dsm.dnode.api.dataset.DataSetFactory.createFromKey(DataSetFactory.java:39) ~[d-node.jar:?]

andreiadamian commented 4 years ago

@ilyasahmed24 Can you attach the full exception? Copying the libs into prweb/lib folder is not the right way to do that. You need to import the comonent as a RAP or Exchange component.

Which Pega platform version is it?

ilyasahmed24 commented 4 years ago

i did an import and still it does not take it and gives me exception, but now I have got a different exception. I am using 8.3 version personal edition.

java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to org.apache.avro.generic.GenericRecord Will attach the whole exception in sometimes.

ilyasahmed24 commented 4 years ago

om.pega.pegarules.pub.PRRuntimeException: Exception during data set execution at com.pegarules.generated.dataset.ra_action_customer_af207ed1c20973f6f91ab15a13aad8cf.perform(ra_action_customer_af207ed1c20973f6f91ab15a13aad8cf.java:122) ~[?:?] at com.pega.pegarules.session.internal.mgmt.Executable.doAction(Executable.java:2220) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.mgmt.Executable.doAction(Executable.java:2128) ~[prprivate-session.jar:?] at com.pegarules.generated.activity.ra_action_pyrunrecordexecute_9a1f2b369bd1517abdb95cb401e69517.step4_circum0(ra_action_pyrunrecordexecute_9a1f2b369bd1517abdb95cb401e69517.java:458) ~[?:?] at com.pegarules.generated.activity.ra_action_pyrunrecordexecute_9a1f2b369bd1517abdb95cb401e69517.perform(ra_action_pyrunrecordexecute_9a1f2b369bd1517abdb95cb401e69517.java:126) ~[?:?] at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:2744) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.mgmt.Executable.invokeActivity(Executable.java:11122) ~[prprivate-session.jar:?] at com.pegarules.generated.activity.ra_action_pzrunrecordexecute_46ae077cc50244070fb53e2948cb85e1.step11_circum0(ra_action_pzrunrecordexecute_46ae077cc50244070fb53e2948cb85e1.java:1036) ~[?:?] at com.pegarules.generated.activity.ra_action_pzrunrecordexecute_46ae077cc50244070fb53e2948cb85e1.perform(ra_action_pzrunrecordexecute_46ae077cc50244070fb53e2948cb85e1.java:248) ~[?:?] at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java:2744) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java:646) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivitiesAlt(PRThreadImpl.java:517) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.runActivities(HttpAPI.java:4002) ~[prprivate-session.jar:?] at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestInner(EngineAPI.java:438) ~[prenginext.jar:?] at sun.reflect.GeneratedMethodAccessor158.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121] at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1393) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1125) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:979) ~[prprivate-session.jar:?] at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequest(EngineAPI.java:361) ~[prenginext.jar:?] at com.pega.pegarules.session.internal.engineinterface.service.HttpAPI.invoke(HttpAPI.java:916) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privact(EngineImpl.java:331) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:274) ~[prprivate-session.jar:?] at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java:251) ~[prprivate-session.jar:?] at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngineInner(JNDIEnvironment.java:275) ~[prpublic.jar:?] at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngine(JNDIEnvironment.java:220) ~[prpublic.jar:?] at com.pega.pegarules.web.impl.WebStandardImpl.makeEtierRequest(WebStandardImpl.java:743) ~[prwebj2ee.jar:?] at com.pega.pegarules.web.impl.WebStandardImpl.doPost(WebStandardImpl.java:413) ~[prwebj2ee.jar:?] at sun.reflect.GeneratedMethodAccessor157.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121] at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:380) ~[prbootstrap-8.3.0-225.jar:8.3.0-225] at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:422) ~[prbootstrap-8.3.0-225.jar:8.3.0-225] at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:224) ~[prbootstrap-api-8.3.0-225.jar:8.3.0-225] at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethod(AppServerBridgeToPega.java:273) ~[prbootstrap-api-8.3.0-225.jar:8.3.0-225] at com.pega.pegarules.internal.web.servlet.WebStandardBoot.doPost(WebStandardBoot.java:141) ~[prbootstrap-api-8.3.0-225.jar:8.3.0-225] at com.pega.pegarules.internal.web.servlet.WebStandardBoot.doGet(WebStandardBoot.java:102) ~[prbootstrap-api-8.3.0-225.jar:8.3.0-225] at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) ~[servlet-api.jar:?] at com.pega.pegarules.internal.web.servlet.WebStandardBoot.service(WebStandardBoot.java:167) ~[prbootstrap-api-8.3.0-225.jar:8.3.0-225] at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) ~[servlet-api.jar:?] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[catalina.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[catalina.jar:8.5.14] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-websocket.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[catalina.jar:8.5.14] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[catalina.jar:8.5.14] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) ~[catalina.jar:8.5.14] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) ~[catalina.jar:8.5.14] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) ~[catalina.jar:8.5.14] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) ~[catalina.jar:8.5.14] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) ~[catalina.jar:8.5.14] at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624) ~[catalina.jar:8.5.14] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) ~[catalina.jar:8.5.14] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) ~[catalina.jar:8.5.14] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799) ~[tomcat-coyote.jar:8.5.14] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) ~[tomcat-coyote.jar:8.5.14] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861) ~[tomcat-coyote.jar:8.5.14] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) ~[tomcat-coyote.jar:8.5.14] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-coyote.jar:8.5.14] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-util.jar:8.5.14] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] Caused by: com.pega.dsm.dnode.api.ExceptionWithInputRecord: java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to org.apache.avro.generic.GenericRecord at com.pega.dsm.dnode.impl.dataset.kafka.KafkaBrowseOperation.processRecords(KafkaBrowseOperation.java:190) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.dataset.kafka.KafkaBrowseOperation.access$000(KafkaBrowseOperation.java:67) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.dataset.kafka.KafkaBrowseOperation$1.emit(KafkaBrowseOperation.java:114) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:338) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:40) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl$3.emit(DataObservableImpl.java:161) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:338) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:40) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:102) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl.toList(DataObservableImpl.java:81) ~[d-node.jar:?] at com.pegarules.generated.dataset.ra_action_customer_af207ed1c20973f6f91ab15a13aad8cf.perform(ra_action_customer_af207ed1c20973f6f91ab15a13aad8cf.java:92) ~[?:?] ... 62 more Caused by: java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to org.apache.avro.generic.GenericRecord at com.pega.integration.kafka.AvroSchemaRegistrySerde.deserialize(AvroSchemaRegistrySerde.java:173) ~[PegaAvroSerde.jar:?] at com.pega.dsm.dnode.impl.dataset.kafka.KafkaBrowseOperation.convertRecordToClipboardPage(KafkaBrowseOperation.java:281) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.dataset.kafka.KafkaBrowseOperation.processRecords(KafkaBrowseOperation.java:180) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.dataset.kafka.KafkaBrowseOperation.access$000(KafkaBrowseOperation.java:67) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.dataset.kafka.KafkaBrowseOperation$1.emit(KafkaBrowseOperation.java:114) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:338) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:40) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl$3.emit(DataObservableImpl.java:161) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:338) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:40) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:102) ~[d-node.jar:?] at com.pega.dsm.dnode.impl.stream.DataObservableImpl.toList(DataObservableImpl.java:81) ~[d-node.jar:?] at com.pegarules.generated.dataset.ra_action_customer_af207ed1c20973f6f91ab15a13aad8cf.perform(ra_action_customer_af207ed1c20973f6f91ab15a13aad8cf.java:92) ~[?:?] ... 62 more

andreiadamian commented 4 years ago

@ilyasahmed24 We've tested compatibility with 8.2.x only. It can be that it is not working in 8.3. Give me few days to check it.

ilyasahmed24 commented 4 years ago

where can i get personal edition for 8.2 ?

andreiadamian commented 4 years ago

That is something I do not know.

ilyasahmed24 commented 4 years ago

thanks keep me posted.

ilyasahmed24 commented 4 years ago

It works with 8.3.0 i had issues with my environment and also the dependencies were giving me an issue and it works now. Thanks for the help