search

twmb / kcl

Your one stop shop to do anything with Kafka. Producing, consuming, transacting, administrating; 0.8.0 through 3.2+
BSD 3-Clause "New" or "Revised" License
194 stars 19 forks source link

Feature Requests: "consume until" and produce tombstones #11

Closed Zach-Johnson closed 2 years ago

Zach-Johnson commented 2 years ago

Hello,

There are a couple of things that would be great to have on this CLI. I'd like to be able to consume from a topic with a "until" setting - this could be either a specific offset or the "end" of the topic partitions (determined by the LSO at the start of the command).

I would also like to be able to produce tombstone messages, which as far as I can tell isn't currently possible. It looks like producing an empty value does not result in a tombstone right now. For example, kafkacat handles this with a -Z producer flag which translates empty values into nulls.

Are those things you would consider adding or would you consider PRs for these two things?

twmb commented 2 years ago

I'm open to PRs! I probably won't be working on kcl much because some things are pretty large and I'm pretty busy and really just keep franz-go up to date now. I occasionally add things to rpk, and some of what I add to rpk is basically taken from "how could this be done better compared to how I did it in kcl?".

FWIW, I am adding enhanced consuming ranges to rpk here: https://github.com/redpanda-data/redpanda/pull/4091

Producing tombstones is interesting and a good idea ... I wonder how that experience could be best represented.

twmb commented 2 years ago

I've finally released this in v0.9.0: https://github.com/twmb/kcl/releases/tag/v0.9.0

Thanks!