edenhill / kcat

Generic command line non-JVM Apache Kafka producer and consumer
Other
5.4k stars 483 forks source link

Support to be able to produce avro schema messages #226

Open IlyaBSR opened 4 years ago

IlyaBSR commented 4 years ago

Currently there is support to read from kafka topics using avro schema, but there is no capability to publish messages in AVRO schema.

docker run -t edenhill/kafkacat:20190711 -t topic-avro -b localhost:9093 -s avro -r https://${KAFKA_USERNAME}:${KAFKA_PASSWORD}@schema-registry.test.com -P % ERROR: -s serdes only available in the consumer

rmechler commented 4 years ago

It would be great to also support TLS client auth for the schema registry, it seems that is not currently supported even for consuming (schema.registry.ssl.* configs seem to not be supported). I don't know if this warrants a separate ticket.

edenhill commented 4 years ago

This is handled by libserdes, can you please file an issue here: https://github.com/confluentinc/libserdes/issues/new

tomerpe commented 3 years ago

Any status on the ability to produce Avro schema messages with schema registry?

ahawtho commented 2 years ago

For our use case, it would be useful to be able to say something like:

kcat -b broker -P -s avro -r https://schema-registry.local -Xserializer.framing=cp1 -Xkey.schema.subject=my-topic-key -Xvalue.schema.subject=my-topic-value -k '{"id":101}' '{myfield1: true, myfield2: "hello"}'

and

kcat -b broker -P -s avro -r https://schema-registry.local -Xserializer.framing=cp1 -Xkey.schema.id=23 -Xvalue.schema.id=94 -k '{"id":101}' '{myfield1: true, myfield2: "hello"}'

It seems pretty difficult to generate anything with avro from the cli using any tool.

0xleizhang commented 1 year ago

@ahawtho does the command work?

bedilbek commented 1 week ago

It's been 4 years, and this issue is still open. @edenhill is it still not possible to produce message with a schema?