tchiotludo / akhq

Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more...
https://akhq.io/
Apache License 2.0
3.4k stars 655 forks source link

unable to configure protobuf deserialization , error ,".google.protobuf.Timestamp" is not defined. #1055

Open hari819 opened 2 years ago

hari819 commented 2 years ago

I have converted my proto files to desc files and followed the link https://akhq.io/docs/configuration/protobuf.html , but could not get it working ,

syntax = "proto3";

import "google/protobuf/timestamp.proto";

message ForElastic {
  google.protobuf.Timestamp eventDate = 2;

i am continuously getting this timestamp error , please can someone guide me on this ,

2022-03-22 15:21:05,494 ERROR r-thread-3 o.a.c.ErrorController  Cannot build Protobuf descriptors for the topics regex [ForElastic.*]
java.lang.RuntimeException: Cannot build Protobuf descriptors for the topics regex [ForElastic.*]
at org.akhq.utils.ProtobufToJsonDeserializer.buildAllDescriptors(ProtobufToJsonDeserializer.java:63)
at org.akhq.utils.ProtobufToJsonDeserializer.<init>(ProtobufToJsonDeserializer.java:40)
at org.akhq.repositories.CustomDeserializerRepository.getProtobufToJsonDeserializer(CustomDeserializerRepository.java:26)
at org.akhq.repositories.RecordRepository.newRecord(RecordRepository.java:446)
at org.akhq.repositories.RecordRepository.lambda$getLastRecord$3(RecordRepository.java:105)
at java.base/java.lang.Iterable.forEach(Unknown Source)
at org.akhq.repositories.RecordRepository.getLastRecord(RecordRepository.java:103)
at org.akhq.controllers.TopicController.lastRecord(TopicController.java:223)
at org.akhq.controllers.$TopicController$Definition$Exec.dispatch(Unknown Source)
at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:351)
at io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:583)
at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:303)
at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:111)
at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:103)
at io.micronaut.http.server.RouteExecutor.lambda$executeRoute$14(RouteExecutor.java:656)
at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:49)
at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62)
at reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)
at io.micronaut.reactive.reactor.instrument.ReactorInstrumentation.lambda$null$0(ReactorInstrumentation.java:62)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
at io.micrometer.core.instrument.composite.CompositeTimer.recordCallable(CompositeTimer.java:68)
at io.micrometer.core.instrument.Timer.lambda$wrap$1(Timer.java:171)
at io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedCallable.call(InvocationInstrumenterWrappedCallable.java:53)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.google.protobuf.Descriptors$DescriptorValidationException: ForElastic.eventDate: **".google.protobuf.Timestamp" is not defined.**
at com.google.protobuf.Descriptors$DescriptorPool.lookupSymbol(Descriptors.java:2563)
at com.google.protobuf.Descriptors$FieldDescriptor.crossLink(Descriptors.java:1559)
at com.google.protobuf.Descriptors$FieldDescriptor.access$1400(Descriptors.java:1057)
at com.google.protobuf.Descriptors$Descriptor.crossLink(Descriptors.java:1000)
at com.google.protobuf.Descriptors$Descriptor.access$1200(Descriptors.java:648)
at com.google.protobuf.Descriptors$FileDescriptor.crossLink(Descriptors.java:600)
at com.google.protobuf.Descriptors$FileDescriptor.buildFrom(Descriptors.java:321)
at com.google.protobuf.Descriptors$FileDescriptor.buildFrom(Descriptors.java:290)
at org.akhq.utils.ProtobufToJsonDeserializer.buildAllDescriptorsForDescriptorFile(ProtobufToJsonDeserializer.java:94)
at org.akhq.utils.ProtobufToJsonDeserializer.buildAllDescriptors(ProtobufToJsonDeserializer.java:61)
... 27 common frames omitted
Logs from Mar 22, 202
tchiotludo commented 2 years ago

Seems that a } is missing on the proto desc ...

hari819 commented 2 years ago

Seems that a } is missing on the proto desc ...

no i did not copy the full proto file here, the actual file has the brackets closed

tchiotludo commented 2 years ago

I'm a noob on protobuf but the implementation is talking about that here

hari819 commented 2 years ago

i can see the alhq.jar file has all the proto files including timestamp.proto , but no clue why it is saying timestamp not found,

hari819 commented 2 years ago

i will try to dig more on this , but would like to know what value to pass here into "value schema" field while pushing data into topic , image

please could you help me on this,

MilkyWay-core commented 2 years ago

got the same problem

tchiotludo commented 2 years ago

The PR is here: https://github.com/tchiotludo/akhq/pull/543 but won't merged due to conflict, PR are welcome