confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
124 stars 1.04k forks source link

Irrelevant properties shown in CLI `show properties` output #1939

Open blueedgenick opened 6 years ago

blueedgenick commented 6 years ago

The original intent of show properties, and the associated set mechanism, was to expose a handful of end-user-facing properties which may be relatively-common to be changed by an end-user to get the query results (or visual display of query results) that they want. Motivating examples being the auto.offset.reset property, along with those determining the defualt partition counts and replication factor for output topics. Somewhere along the way this morphed into showing all/most of the internal client properties that a kstreams app uses in it's initialization, which is overwhelming to most users and of limited utility - see for example the list of 15 ssl.* properties. This issue is to track cleanup/suppression of the 'unwanted' properties from the client-side display.

miguno commented 6 years ago

IIRC @big-andy-coates worked on these recent changes.

Perhaps we could address your request above by making a distinction between:

If yes to this idea, what would you expect to see in the shorter list? Only properties that were explicitly changed by the user? Or ...?

vcrfxia commented 5 years ago

Tried this just now on the latest master and I don't see a giant list of SSL properties but I do see some internal configs, including compatibility breaking ones that are irrelevant to the user such as ksql.functions.substring.legacy.args and ksql.query.fields.key.legacy and ksql.windowed.session.key.legacy so I'll leave this ticket open for now as a reminder to clean these up.

 Property                                               | Default override | Effective Value
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 ksql.access.validator.enable                           |                  | auto
 ksql.avro.maps.named                                   |                  | true
 ksql.connect.url                                       |                  | http://localhost:8083
 ksql.connect.worker.config                             |                  |
 ksql.extension.dir                                     |                  | ext
 ksql.functions.substring.legacy.args                   |                  | false
 ksql.insert.into.values.enabled                        |                  | true
 ksql.internal.topic.replicas                           |                  | 1
 ksql.metric.reporters                                  |                  |
 ksql.metrics.extension                                 |                  | NULL
 ksql.metrics.tags.custom                               |                  |
 ksql.named.internal.topics                             |                  | on
 ksql.output.topic.name.prefix                          |                  |
 ksql.persistence.wrap.single.values                    |                  | true
 ksql.persistent.prefix                                 |                  | query_
 ksql.query.fields.key.legacy                           |                  | false
 ksql.query.inject.legacy.map.values.node               |                  | false
 ksql.query.persistent.active.limit                     |                  | 2147483647
 ksql.query.stream.groupby.rowkey.repartition           |                  | false
 ksql.schema.registry.url                               |                  | http://localhost:8081
 ksql.security.extension.class                          |                  | NULL
 ksql.service.id                                        |                  | default_
 ksql.sink.partitions                                   |                  | NULL
 ksql.sink.replicas                                     |                  | NULL
 ksql.sink.window.change.log.additional.retention       |                  | 1000000
 ksql.streams.application.server                        | SERVER           | http://localhost:8088
 ksql.streams.auto.offset.reset                         |                  | latest
 ksql.streams.bootstrap.servers                         | SERVER           | localhost:9092
 ksql.streams.cache.max.bytes.buffering                 | SERVER           | 10000000
 ksql.streams.commit.interval.ms                        | SERVER           | 2000
 ksql.streams.default.deserialization.exception.handler | SERVER           | io.confluent.ksql.errors.LogMetricAndContinueExceptionHandler
 ksql.streams.default.production.exception.handler      | SERVER           | io.confluent.ksql.errors.ProductionExceptionHandlerUtil$LogAndFailProductionExceptionHandler
 ksql.streams.num.stream.threads                        | SERVER           | 4
 ksql.streams.topology.optimization                     | SERVER           | all
 ksql.transient.prefix                                  |                  | transient_
 ksql.udf.collect.metrics                               |                  | false
 ksql.udf.enable.security.manager                       |                  | true
 ksql.udfs.enabled                                      |                  | true
 ksql.windowed.session.key.legacy                       |                  | false
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
agavra commented 5 years ago

I'm with @miguno - we should have an option to see all the "irrelevant" configs in order to debug fringe errors. This may involve a SHOW PROPERTIES EXTENDED functionality