datagouv / data.gouv.fr

Ce dépôt rassemble les tickets techniques qui portent sur data.gouv.fr.
https://www.data.gouv.fr
76 stars 14 forks source link

Ajout d'un système de monitoring de Kafka #783

Closed maudetes closed 2 years ago

maudetes commented 2 years ago

Exemple avec ControlCenter

maudetes commented 2 years ago

Using docker-compose, adding a simple control center is as simple as adding:

  control-center:
    image: confluentinc/cp-enterprise-control-center:7.0.2
    hostname: control-center
    container_name: control-center
    depends_on:
      - broker
    ports:
      - "9021:9021"
    environment:
      CONTROL_CENTER_BOOTSTRAP_SERVERS: 'broker:29092'
      CONTROL_CENTER_REPLICATION_FACTOR: 1
      CONTROL_CENTER_INTERNAL_TOPICS_PARTITIONS: 1
      CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_PARTITIONS: 1
      CONFLUENT_METRICS_TOPIC_REPLICATION: 1
      PORT: 9021

You can now go to http://localhost:9021/. It works well :sparkles: without having all the confluent services. The control center gives us access to a lot of information on a dashboard, ex: :eyes:

It allows to produce message on a topic, create a topic, etc. It also allows to create alerts on cluster status, production/consumption throughput, etc.

maudetes commented 2 years ago

Installé de manière ansible-isé, reste à voir comment l'exposer ?

maudetes commented 2 years ago

Done by @jordanguedj