search

confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
118 stars 1.04k forks source link

Missing KEY_FORMAT parameter #805

Closed maver1ck closed 6 years ago

maver1ck commented 6 years ago

Hi, I'm using KSQL for querying data where both key and value are serialized with Avro. How can I specify the format for KEY ?

apurvam commented 6 years ago

KSQL only accepts avro values. The keys themselves are always interpreted as strings. Do you have a use case for accessing fields within an avro encoded key?

maver1ck commented 6 years ago

No. I wanted to group by key. And don't know how. (whole key so it can by string)

apurvam commented 6 years ago

Have you seen our documentation on https://github.com/confluentinc/ksql? It has plenty of examples for grouping by key.

rmoff commented 6 years ago

@maver1ck whilst there's no support for Avro keys currently, you can use KSQL to re-key a topic (PARTITION BY). The resulting topic will have String keys and can then be correctly used for joining, grouping, etc. This blog posts demonstrates this feature in action: https://www.confluent.io/blog/ksql-in-action-real-time-streaming-etl-from-oracle-transactional-data

rmoff commented 6 years ago

Track support for avro keys in #824