search

edenhill / kcat

Generic command line non-JVM Apache Kafka producer and consumer
Other
5.39k stars 483 forks source link

Decimal Support in avro format #316

Open maver1ck opened 3 years ago

maver1ck commented 3 years ago

Is it possible to add Decimal support in Avro format to kafkacat ? Right now decimals are displayed as byte arrays.

edenhill commented 3 years ago

We rely on avro-c to generate JSON from Avro, maybe Avro-C 1.8.x doesn't support decimal?

krisajenkins commented 3 years ago

I did a bit of research on this (because I'd like decimal support too).

Avro-C doesn't support decimal because it doesn't support logical types. There's a PR open for it but it seems to be stuck in PR purgatory.

edenhill commented 3 years ago

Thanks @krisajenkins !

SahilKang commented 4 months ago

I opened https://github.com/apache/avro/pull/2891 as an alternative to add decimal support to avro-c

with that said though, I think there's a small misunderstanding why kcat currently displays decimal objects as a series of bytes: it's not because avro-c lacks decimal support, it's because the json representation you're seeing is actually specified that way in the spec

so even after avro-c adds decimal support, we'll probably have to add some logic here in kcat to print decimals more readably...that should be relatively straightforward to do though