Filter by header

[andrewthad]

It would be useful to, in a consuming context, be able to filter by message headers. I imagine the syntax being something like this:

kafkacat -C -b example.com:9092  -G myexamplegroup -H foo=bar -H baz=bang -t mytopic

The format for key-value pairs would be the same as for the -H flag in the producer context, and the only matching strategy would be exact byte-for-byte matching (no case sensitivity, no regex, no locale-sensitive matching, etc.) implemented by memcmp.

The argument against such a flag is that it's possible to accomplish something similar with grep. However, using grep to do this has these disadvantages:

• Grep requires clever anchoring of the pattern in order to get the desired behavior, especially if you want to include multiple keys or the kafka timestamp in the output
• Grep implies that kafkacat's deserialization will be applied even to messages that are filtered out.

[andrewthad]

I'm still interested in adding support for this if maintainers believe that kafkacat is a good place for this logic to live.

[amterp]

Would be very keen on this feature! I recently discovered kcat and was hoping this was already supported, but seems I'm not in luck.

I'm dealing with quite large messages so being able to filter by header and thus not needing to deserialize all the non-matching messages would be a big win.