search

awslabs / aws-glue-schema-registry

AWS Glue Schema Registry Client library provides serializers / de-serializers for applications to integrate with AWS Glue Schema Registry Service. The library currently supports Avro, JSON and Protobuf data formats. See https://docs.aws.amazon.com/glue/latest/dg/schema-registry.html to get started.
Apache License 2.0
127 stars 95 forks source link

Provides library version compatible with kafka client 3 ? #301

Open yannrouillard opened 1 year ago

yannrouillard commented 1 year ago

Hi,

We are currently considering switching from Kafka Registry to Glue Schema Registry. As part of that work, we tested updating some of our kakfa stream application to GlueSchemaRegistryKafkaStreamsSerde.

We noticed that the software.amazon.glue:schema-registry-serde depends on org.apache.kafka:kafka-clients:2.8.2 whereas all our kafka streams apps were using org.apache.kafka:kafka-streams:3.x which depends on org.apache.kafka:kafka-clients:3.x. As the result the org.apache.kafka:kafka-streams:3.x is used and org.apache.kafka:kafka-clients:2.8.2 is evicted despite the risk of binary incompatibility with GlueSchemaRegistryKafkaStreamsSerde

Is there some plan to provide a version of this library that is built with org.apache.kafka:kafka-clients:3.x ? Or is it know that there is no risk of binary incompatibility in this case ?

blacktooth commented 11 months ago

Hello,

When you say binary incompatibility, are you talking about the schema header version format? We cannot think of reasons for it but recommend testing it to ensure it doesn't affect your use-cases.

As of now, we welcome PRs to upgrade as long as it doesn't break our unit / integration tests.

Thanks.