Closed asalhani closed 2 months ago
As per README.md,
exporters: redis: className: io.zeebe.redis.exporter.RedisExporter jarPath: exporters/zeebe-redis-exporter-1.0.0-jar-with-dependencies.jar args: cleanupCycleInSeconds: -1 maxTimeToLiveInSeconds: -1 format: "json"
Format property can take either of 2 values [protobuf|json] When setting to
protobuf
the listner is working fine.However, when setting it to "json", the following excption is being thrown when listner activating:
2024-08-17T18:53:22.173+03:00 ERROR 22244 --- [tasklistapp] [pool-5-thread-1] io.zeebe.redis.connect.java.ZeebeRedis : Consumer[group=USER_TASK_CONSUMER_GROUP, id=consumer-1] failed to read from streams 'zeebe:*' com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag. at com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:114) ~[protobuf-java-3.25.1.jar:na] at com.google.protobuf.CodedInputStream$ArrayDecoder.checkLastTagWas(CodedInputStream.java:615) ~[protobuf-java-3.25.1.jar:na] at com.google.protobuf.CodedInputStream$ArrayDecoder.readGroup(CodedInputStream.java:811) ~[protobuf-java-3.25.1.jar:na] at com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:494) ~[protobuf-java-3.25.1.jar:na] at com.google.protobuf.GeneratedMessageV3$Builder.parseUnknownField(GeneratedMessageV3.java:864) ~[protobuf-java-3.25.1.jar:na] at io.zeebe.exporter.proto.Schema$Record$Builder.mergeFrom(Schema.java:995) ~[zeebe-exporter-protobuf-1.6.1.jar:1.6.1] at io.zeebe.exporter.proto.Schema$Record$1.parsePartialFrom(Schema.java:1166) ~[zeebe-exporter-protobuf-1.6.1.jar:1.6.1] at io.zeebe.exporter.proto.Schema$Record$1.parsePartialFrom(Schema.java:1158) ~[zeebe-exporter-protobuf-1.6.1.jar:1.6.1] at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:135) ~[protobuf-java-3.25.1.jar:na] at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:168) ~[protobuf-java-3.25.1.jar:na] at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:180) ~[protobuf-java-3.25.1.jar:na] at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:185) ~[protobuf-java-3.25.1.jar:na] at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:25) ~[protobuf-java-3.25.1.jar:na] at io.zeebe.exporter.proto.Schema$Record.parseFrom(Schema.java:754) ~[zeebe-exporter-protobuf-1.6.1.jar:1.6.1] at io.zeebe.redis.connect.java.ZeebeRedis.handleRecord(ZeebeRedis.java:341) ~[zeebe-redis-connector-1.0.0.jar:1.0.0] at io.zeebe.redis.connect.java.ZeebeRedis.readNext(ZeebeRedis.java:300) ~[zeebe-redis-connector-1.0.0.jar:1.0.0] at io.zeebe.redis.connect.java.ZeebeRedis.readFromStream(ZeebeRedis.java:270) ~[zeebe-redis-connector-1.0.0.jar:1.0.0] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[na:na] at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na] at java.base/java.lang.Thread.run(Thread.java:842) ~[na:na]
I need to set it to JSON because "protobuf" does not render the value correctly when checking record in Redis Insight
Format= json ==> value can be inspected in Redis Insight
Format= protobuf ==> value can't retrived in Redis insight
@VonDerBeck
Hi @asalhani,
even if the exporter itself is able to (optionally) export events as JSON, the connector is designed to use the default which is protobuf. So this is not an error.
@VonDerBeck Is there is a way to configure both exporter and connector to transmit data as JSON instead of protobuf?
@asalhani Currently the connector has no option to read JSON. Provided connectors are based on protobuf.
While I understand that it could be a nice feature especially during development to have a look into Redis itself and have readable messages on the other hand you are able to log messages after receiving them via the connector as well. So of course this is nice-to-have, but currently it's not there. Maybe something for the future.
As per README.md,
exporters: redis: className: io.zeebe.redis.exporter.RedisExporter jarPath: exporters/zeebe-redis-exporter-1.0.0-jar-with-dependencies.jar args: cleanupCycleInSeconds: -1 maxTimeToLiveInSeconds: -1 format: "json"
Format property can take either of 2 values [protobuf|json] When setting to
protobuf
the listner is working fine.However, when setting it to "json", the following excption is being thrown when listner activating:
I need to set it to JSON because "protobuf" does not render the value correctly when checking record in Redis Insight
Format= json ==> value can be inspected in Redis Insight
Format= protobuf ==> value can't retrived in Redis insight