confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
128 stars 1.04k forks source link

Not able to configure producer/consumer/admin client settings with Streams prefixes #7887

Open mikebin opened 3 years ago

mikebin commented 3 years ago

Describe the bug It doesn't seem to be possible to configure settings for specific types of clients used internally with the Kafka Streams naming prefixes in ksqlDB.

It might be helpful to be able to configure settings at this fine-grained level in some cases.

To Reproduce Version: ksqlDB 6.2.0

SET 'ksql.streams.main.consumer.max.poll.records'='300';

Expected behavior Successfully update property for the main consumers used in the query by Kafka Streams internally.

Actual behaviour

Not recognizable as ksql, streams, consumer, or producer property: 'ksql.streams.main.consumer.max.poll.records'

Additional context Also tried main.consumer.max.poll.records and main.max.poll.records, same error. max.poll.records works.

AlanConfluent commented 3 years ago

I think it just has to do with the prefix of the config name. If you set it in the properties file, I believe ksql.streams.main.consumer.max.poll.records would be correct, but when setting it on the CLI, you remove ksql I believe, so I think it would be SET 'streams.main.consumer.max.poll.records'='300';

mikebin commented 3 years ago

Removing the ksql prefix doesn't seem to work either in the CLI:

ksql> SET 'streams.main.consumer.max.poll.records'='300';
Not recognizable as ksql, streams, consumer, or producer property: 'streams.main.consumer.max.poll.records'