Support for avro schema for kafka sink

ArroyoSystems / arroyo

Distributed stream processing engine in Rust

https://arroyo.dev

Apache License 2.0

3.63k stars 204 forks source link

Support for avro schema for kafka sink #679

Open hazelnut-99 opened 2 months ago

hazelnut-99 commented 2 months ago

As mentioned in the docs, "Currently, Kafka sinks only support writing JSON data, with the structure determined by the schema of the data being written."

It will be a really nice and useful feature to support avro schema for kafka sink. For example, currently in our use case, all data written to and read from kafka are avro-based.

hazelnut-99 commented 2 months ago

Oh, it seems that it's already supported! I managed to make it work by specifying this parameter 'avro.confluent_schema_registry' = 'true' in the sink definition.