search

camunda-community-hub / zeebe-kafka-exporter

Export events from Zeebe to Kafka
Apache License 2.0
36 stars 27 forks source link

compatability with zeebe 8 #295

Open cameronbraid opened 2 years ago

cameronbraid commented 2 years ago

I am wanting to use zeebe-kafka-exporter-serde v3.1.1 with zeebe 8.0.2 which uses zeebe-protocol-jackson 8.0.2 but am getting an exception


Caused by: java.lang.TypeNotPresentException: Type io.camunda.zeebe.protocol.jackson.record.AbstractRecord not present
    at java.base/sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:117)
CoreReflectionFactory.java:117
    at java.base/sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:125)
Reifier.java:125
    at java.base/sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
    at java.base/sun.reflect.generics.visitor.Reifier.reifyTypeArguments(Reifier.java:68)
Reifier.java:68
    at java.base/sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:138)
Reifier.java:138
    at java.base/sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
ClassTypeSignature.java:49
    at java.base/sun.reflect.generics.repository.ClassRepository.computeSuperclass(ClassRepository.java:104)
ClassRepository.java:104
    at java.base/sun.reflect.generics.repository.ClassRepository.getSuperclass(ClassRepository.java:86)
ClassRepository.java:86
    at java.base/java.lang.Class.getGenericSuperclass(Class.java:1022)
Class.java:1022
    at com.fasterxml.jackson.core.type.TypeReference.<init>(TypeReference.java:33)
TypeReference.java:33
    at io.zeebe.exporters.kafka.serde.RecordDeserializer$1.<init>(RecordDeserializer.java:38)
RecordDeserializer.java:38
    at io.zeebe.exporters.kafka.serde.RecordDeserializer.<init>(RecordDeserializer.java:38)
RecordDeserializer.java:38
    at io.zeebe.exporters.kafka.serde.RecordDeserializer.<init>(RecordDeserializer.java:34)
RecordDeserializer.java:34
    at io.zeebe.exporter.source.kafka.KafkaRecordSourceConfiguration.zeebeConsumerFactory(KafkaRecordSourceConfiguration.java:69)
KafkaRecordSourceConfiguration.java:69
    at io.zeebe.exporter.source.kafka.KafkaRecordSourceConfiguration$$EnhancerBySpringCGLIB$$996179f0.CGLIB$zeebeConsumerFactory$1(<generated>)
    at io.zeebe.exporter.source.kafka.KafkaRecordSourceConfiguration$$EnhancerBySpringCGLIB$$996179f0$$FastClassBySpringCGLIB$$c8592355.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
MethodProxy.java:244
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
ConfigurationClassEnhancer.java:331
    at io.zeebe.exporter.source.kafka.KafkaRecordSourceConfiguration$$EnhancerBySpringCGLIB$$996179f0.zeebeConsumerFactory(<generated>)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
NativeMethodAccessorImpl.java:78
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
DelegatingMethodAccessorImpl.java:43
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
Method.java:567
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
SimpleInstantiationStrategy.java:154
    ... 71 common frames omitted
Caused by: java.lang.ClassNotFoundException: io.camunda.zeebe.protocol.jackson.record.AbstractRecord
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
BuiltinClassLoader.java:636
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
ClassLoaders.java:182
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
ClassLoader.java:519
    at java.base/java.lang.Class.forName0(Native Method)
    at java.base/java.lang.Class.forName(Class.java:466)
Class.java:466
    at java.base/sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:114)
CoreReflectionFactory.java:114
    ... 94 common frames omitted

(edited to paste the correct stacktrace)

Should this work ? or does zeebe-kafka-exporter-serde need an update to make it compatible ?

cameronbraid commented 2 years ago

I also tried the exporter in a zeebe 8 broker and got 

Caused by: java.lang.UnsupportedOperationException: Failed to serialize value to JSON; this implementation does not support it out of the box.  You may want to try with the protocol-jackson module.
    at io.camunda.zeebe.protocol.record.JsonSerializable.toJson(JsonSerializable.java:29)
JsonSerializable.java:29
    at io.zeebe.exporter.source.KafkaRecordSourceIT$RecordCollector.lambda$connectTo$0(KafkaRecordSourceIT.java:107)
KafkaRecordSourceIT.java:107
    at io.zeebe.exporter.source.kafka.AbstractRecordSource.lambda$handleRecord$0(AbstractRecordSource.java:53)
AbstractRecordSource.java:53
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
ArrayList.java:1511
    at io.zeebe.exporter.source.kafka.AbstractRecordSource.handleRecord(AbstractRecordSource.java:53)
AbstractRecordSource.java:53
    at io.zeebe.exporter.source.kafka.KafkaListenerRecordSource.handleRecord(KafkaListenerRecordSource.java:37)
KafkaListenerRecordSource.java:37
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
NativeMethodAccessorImpl.java:78
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
DelegatingMethodAccessorImpl.java:43
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
Method.java:567
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:169)
InvocableHandlerMethod.java:169
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:119)
InvocableHandlerMethod.java:119
    at org.springframework.kafka.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:56)
HandlerAdapter.java:56
    at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:347)
MessagingMessageListenerAdapter.java:347
    at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:92)
RecordMessagingMessageListenerAdapter.java:92
    at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:53)
RecordMessagingMessageListenerAdapter.java:53
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:2645)
KafkaMessageListenerContainer.java:2645
    ... 11 common frames omitted

So I guess its not zeebe 8 compatible

Any idea how much work the upgrade to v8 compatibility would be ?

rusterholz commented 1 year ago

@npepinpe Any progress on a version of zeebe-kafka-exporter which is compatible with Zeebe 8.x?

weiye-Lee commented 1 year ago

@cameronbraid @rusterholz See also #375 #346