confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
41 stars 1.04k forks source link

CREATE STREAM AS SELECT should inherit topic retention from source #7794

Open vcrfxia opened 3 years ago

vcrfxia commented 3 years ago

Is your feature request related to a problem? Please describe.

Today, when a CREATE STREAM AS SELECT statement in ksqlDB results in a new topic being created, the new topic is created using the broker default for retention.ms, even if the source topic has a custom value for retention.ms. A more intuitive default behavior would be to inherit the retention.ms from the source topic when creating the sink topic.

Describe the solution you'd like

See above.

Describe alternatives you've considered

Some thought should be put into the set of topic configs that should be inherited by default. retention.ms may or may not be the only one.

Additional context

As a workaround, users can pre-create sink topics with their desired topic configs.

vcrfxia commented 3 years ago

Related questions:

Probably easiest to continue copying nothing (no configs/properties) from the source as we do today.

colinhicks commented 3 years ago

We should ensure the option to pre-create the sink topics (or alter configs later) with the desired configuration is well-documented.