search

confluentinc / schema-registry

Confluent Schema Registry for Kafka
https://docs.confluent.io/current/schema-registry/docs/index.html
Other
2.21k stars 1.11k forks source link

Update protobuf for V4 - serde to work with v4 of protobuf #3047

Open krickert opened 6 months ago

krickert commented 6 months ago

Within the schema registry code for protocol buffers needs to be updated to work with the latest version 4.26

So far I see that GeneratedMessageV3 has been moved to GeneratedMessage. I'm sure there might be many others.

krickert commented 6 months ago

GeneratedMessageV3 is all in the tests. I'm trying to compile it locally, then I can make a patch

krickert commented 4 months ago

Has anyone looked into this? It's recommended never to use the GeneratedMessageV3 in code. It's all over the testing suite, but I can't get this to complie on my osx machine. I'd be glad to update the code, it should just work if you change the protocol buffer to the latest, 4.26, and use GeneratedMesasage instead of GeneratedMessageV3

ljj7kr commented 1 month ago

In my case

  1. protobuf-java: 4.x version without GeneratedMessageV3
  2. latest(7.7.0) kafka-protobuf libraries uses GeneratedMessageV3 ex) kafka-protobuf-serializer > kafka-protobuf-provider.ProtobufSchemaUtils > ProtobufSchema uses GeneratedMessageV3
  3. ClassNotFondException: GeneratedMessageV3

Conclusion: Protobuf 4.x version cannot be used in the latest(7.7.0) version of kafka-protobuf.

krickert commented 1 month ago

In my case

  1. protobuf-java: 4.x version without GeneratedMessageV3
  2. latest kafka-protobuf libraries uses GeneratedMessageV3 ex) kafka-protobuf-serializer > kafka-protobuf-provider.ProtobufSchemaUtils > ProtobufSchema uses GeneratedMessageV3
  3. ClassNotFondException: GeneratedMessageV3

Conclusion: Protobuf 4.x version cannot be used in the latest version of kafka-protobuf.

You are correct. They have to get rid of the code that references V3

It's slightly more than that right now, I went ahead and changed the references but I got some serialization issues.

It's a pretty big update, probably going to take a bit of time for this and gRPC to catch up