Open ericsun2 opened 3 years ago
Any update on this
Poke
Hello, thanks for starting this conversation.
kafka-go exposes the content of messages as an opaque byte array, the package currently does not unmarshal nor interpret the data.
From where we stand as maintainers of kafka-go, this functionality appears to deserve a dedicated package since it does not have a tight dependency to kafka-go itself, any byte array could be decoded from schemas fetched from remote locations.
We would happily provide visibility to such package in kafka-go, or potentially host it as a submodule if we have a convincing case for it. At this time such a change will need to come from a community contributor, the core maintainer team will not be able to allocate development time to build this feature.
Let me know what you think!
Support for Protobuf with Confluent Schema Registry If the Kafka event is serialized in Protobuf and a 4-byte integer schema_id is put in front of the Protobuf binary stream, can
kafka-go
invoke the REST API of Confluent Schema Registry to fetch the.proto
schema text from Schema Registry, and then form a decoder/deserializer on-the-fly to deserialize the Protobuf binary stream?Supporting documentation