Add `RETENTION` as an option for a `WITH` clause.

krisajenkins commented 2 years ago

Is your feature request related to a problem? Please describe.

It’s great that you can create a topic automatically as part of creating a stream—WITH ( KAFKA_TOPIC = ...—and it’s obviously necessary that you can set the PARTITIONS and REPLICAS when you do it. I'd also find it really useful to be able to set the retention times in the same way.

Describe the solution you'd like

Add RETENTION as a valid WITH clause, such that this would be valid:

) WITH (
  KAFKA_TOPIC = 'foo',
  RETENTION = -1,
  ...
);

Describe alternatives you've considered

At the moment you're forced out of KSQL and into another tool once the CREATE STREAM statement has run. Either the cloud GUI or one of the command line tools. It's a bit of a pain.

One alternative would be an ALTER TOPIC ... SET RETENTION ... command, but that feels like a big can of worms to open right now.

Additional context

This is a UX enhancement. It doesn't enable anything you can't already do, but it would make the task much simpler and allow one to get more work done directly via ksqlDB.

suhas-satish commented 2 years ago

@bvarghese1 is working on a similar issue to add RETENTION_MS . We can duplicate this

bvarghese1 commented 2 years ago

Duplicate of https://github.com/confluentinc/ksql/issues/5148

confluentinc / ksql

Add `RETENTION` as an option for a `WITH` clause. #9319