confluentinc / confluent-kafka-go

Confluent's Apache Kafka Golang client
Apache License 2.0
4.58k stars 651 forks source link

Use protocompile instead of protoparse #954

Open james-johnston-thumbtack opened 1 year ago

james-johnston-thumbtack commented 1 year ago

Description

Currently the protobuf deserializer makes use of the protoparse package to parse protobuf IDL to descriptors.

According to the project website, this package is effectively deprecated (emphasis mine):

Later this year (2023), we expect to cut a v2 of this whole repo. The V2 protobuf API now includes support for descriptors and dynamic messages, and it also exposes numerous low-level aspects like the binary wire format. So a lot of what's in this repo is no longer necessary. But some features still are, such as the desc/builder, desc/protoprint, dynamic/grpcdynamic, dynamic/msgregistry, and grpcreflect packages. These will remain in the V2 form of the repo (though possibly rearranged a little). Noticeably absent from the list above is desc/protoparse, whose V2 API is already available in a brand new module named protocompile. It has a much improved API, provides better performance, and directly uses the reflection APIs in the v2 Protobuf runtime.

It sounds like the protoparse package is not going to be migrated to V2. Instead, the author recommends the linked protocompile, which looks like it has a nice, easy API for parsing IDL and getting protobuf V2 descriptors from it. Therefore at some point we might want to revise confluent-kafka-go to use this newer protocompile package.

PrasanthV454 commented 1 year ago

Hey @james-johnston-thumbtack, thanks for bringing this up. We will take a look into it and decide when to take this up.