Provide smooth path for users to run Control Center Docker with any tutorial

[ybyzek]

Control Center provides nice UI for users who want to:

• inspect topic data
• use ksqlDB UI instead of CLI

cc: @SrWaves

[ybyzek]

Manual workaround: before running the harness runner, add a file docker-compose.override.yml (with control center) to the subdirectory for the KT (same location as where the docker-compose.yml file is)

version: "2"

services:

  control-center:
    image: confluentinc/cp-enterprise-control-center:5.5.0
    hostname: control-center
    container_name: control-center
    ports:
      - "9021:9021"
    environment:
      CONTROL_CENTER_BOOTSTRAP_SERVERS: 'broker:9092'
      CONTROL_CENTER_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      CONTROL_CENTER_CONNECT_CLUSTER: 'connect:8083'
      CONTROL_CENTER_KSQL_KSQLDB1_URL: "http://ksqldb-server:8088"
      CONTROL_CENTER_KSQL_KSQLDB1_ADVERTISED_URL: "http://localhost:8088"
      CONTROL_CENTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
      CONTROL_CENTER_REPLICATION_FACTOR: 1
      CONTROL_CENTER_INTERNAL_TOPICS_PARTITIONS: 1
      CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_PARTITIONS: 1
      CONFLUENT_METRICS_TOPIC_REPLICATION: 1
      PORT: 9021

If https://github.com/confluentinc/kafka-tutorials/issues/461 is implemented and broker image goes to cp-kafka instead of cp-server, then additional overrides are required for the broker:

  broker:
    image: confluentinc/cp-server:5.5.0
    environment:
      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: broker:9092
      CONFLUENT_METRICS_REPORTER_ZOOKEEPER_CONNECT: zookeeper:2181
      CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
      CONFLUENT_METRICS_ENABLE: 'true'
      CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'

[ybyzek]

If we choose to modify KT, one implementation option to consider is creating a new make target to differentiate validation runs (e.g. Semaphore, would would not use C3) from manual runs (which would use C3). Impact: this would change how manual runs are triggered: instead of current implementation of make SEQUENCE="dev, test", it could be run with make manual

[tlberglund]

Do we want this image in all tutorials, or only in those that use it? I'm 💯 in support of some C3 tutorials, but having the image everywhere adds a fairly heavy vendor footprint in a property whose nominal purpose is broad Kafka education under Confluent branding.

[rmoff]

Given that there's a workaround for where people do want C3, I would be against adding C3 by default, for the same reasons that @tlberglund describes.

[ybyzek]

Thanks for the feedback. Makes and agree that we want to emphasize broad Kafka tutorials without Confluent branding. Just thinking about UX for the workaround and if there's a middle ground that adheres to Apache Kafka learning but also makes it easy to layer on C3.

Painpoints with current workaround require users to:

• Create the docker-compose.override.yml file
• Copy it into each KT code folder where it is desired to be used

To keep the balance but also meet end user needs, what about this idea? Add a bash script in https://github.com/confluentinc/kafka-tutorials/tree/master/tools that takes care of the above? It would create the overrides file and copy it into all KT code folders on the user's local checked out copy (we can also add docker-compose.override.yml to gitignore to make sure it doesn't accidentally get checked in). Then all the user needs to do is run that script once, and they are all set, and it has no impact to other KT runs (e.g. does not increase footprint in Semaphore, etc)

[tlberglund]

So the people wanting to layer on C3 to their use of any given KT would just run that script and docker-compose up like they owned the place?

[ybyzek]

run that script and docker-compose up like they owned the place?

... like a baus.

If a user (human or Semaphore) doesn't run that script, they don't even know about C3. 💯 opt-in

[tlberglund]

Well, then IMO: :shipit:

[ybyzek]

Resolved by https://github.com/confluentinc/kafka-tutorials/pull/462