confluentinc / cp-helm-charts

The Confluent Platform Helm charts enable you to deploy Confluent Platform services on Kubernetes for development, test, and proof of concept environments.
https://cnfl.io/getting-started-kafka-kubernetes
Apache License 2.0
784 stars 842 forks source link

helm values for a Developer licensed cluster #545

Open tuxerrante opened 3 years ago

tuxerrante commented 3 years ago

Hi, I'm trying to setup a Confluent cluster in a non-enterprise configuration. I've tried different options, changing both the number of kafka broker replicas to 1 and the kafka image itself, but everything seems to end in a crashLoopBackoff of the pods.

What I did here was to apply the helm values taken from this repo, and

Applied with this command: helm install -f confluent-cp-helm-values.yaml confluent confluentinc/cp-helm-charts --namespace confluent

## ------------------------------------------------------
## Zookeeper
## ------------------------------------------------------
cp-zookeeper:
  enabled: true
  servers: 3
  image: confluentinc/cp-zookeeper
  imageTag: 6.2.0
  heapOptions: "-Xms512M -Xmx512M"
  persistence:
    enabled: true
    dataDirSize: 10Gi
    dataDirStorageClass: "oci-bv"
snaphots.
    dataLogDirSize: 10Gi
    dataLogDirStorageClass: "oci-bv"
  securityContext: 
    runAsUser: 0
  resources: {}
## ------------------------------------------------------
## Kafka
## ------------------------------------------------------
cp-kafka:
  enabled: true
  brokers: 1                        # NO MORE THAN ONE REPLICAS IS ALLOWED FOR DEVELOPER LICENSE
  image: confluentinc/cp-kafka
  imageTag: 6.2.0
  heapOptions: "-Xms512M -Xmx512M"
  persistence:
    enabled: true
    storageClass: "oci-bv"
    size: 5Gi
    disksPerBroker: 1
  resources: {}
  securityContext: 
    runAsUser: 0

## ------------------------------------------------------
## Schema Registry
## ------------------------------------------------------
cp-schema-registry:
  enabled: true
  image: confluentinc/cp-schema-registry
  imageTag: 6.2.0
  imagePullSecrets:
  #  - name: "regcred"
  heapOptions: "-Xms512M -Xmx512M"
  resources: {}

## ------------------------------------------------------
## REST Proxy
## ------------------------------------------------------
cp-kafka-rest:
  enabled: true
  image: confluentinc/cp-kafka-rest
  imageTag: 6.2.0
  heapOptions: "-Xms512M -Xmx512M"
  resources: {}

## ------------------------------------------------------
## Kafka Connect
## ------------------------------------------------------
cp-kafka-connect:
  enabled: true
  image: confluentinc/cp-kafka-connect
  imageTag: 6.2.0
  heapOptions: "-Xms512M -Xmx512M"
  resources: {}

## ------------------------------------------------------
## KSQL Server
## ------------------------------------------------------
cp-ksql-server:
  enabled: true
  image: confluentinc/cp-ksqldb-server
  imageTag: 6.2.0
  heapOptions: "-Xms512M -Xmx512M"
  ksql:
    headless: false

## ------------------------------------------------------
## Control Center
## ------------------------------------------------------
cp-control-center:
  enabled: false
  image: confluentinc/cp-enterprise-control-center
  imageTag: 6.2.0
  heapOptions: "-Xms512M -Xmx512M"
  resources: {}

It seems they can't see each other, for example trying to reach "confluent-cp-kafka-0.confluent-cp-kafka-headless.confluent:9092" the kafka broker fails.

immagine

Instead using cp-enterprise-kafka, the pods restarts seem lees frequent and due to timeouts: "" ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain) org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 30000ms ""

gAmUssA commented 2 years ago

cp-helm-charts are not designed to be used with Kafka nodes less than 3

tuxerrante commented 2 years ago

So how are we supposed to use the community licensed cluster?

Il giorno lun 26 lug 2021 alle ore 15:26 Viktor Gamov < @.***> ha scritto:

cp-helm-charts are not designed to be used with Kafka nodes less than 3

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/confluentinc/cp-helm-charts/issues/545#issuecomment-886702438, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB72DNNXPBYQA33LNBXO7OTTZVO77ANCNFSM5AC7ASSQ .

gAmUssA commented 2 years ago

@tuxerrante don;t use cp-enterprise-kafka, cp-server and cp-control-center images. for Kafka use cp-kafka. you don't need a license to use the confluent community. hope this helps.

tuxerrante commented 2 years ago

@tuxerrante don;t use cp-enterprise-kafka, cp-server and cp-control-center images. for Kafka use cp-kafka. you don't need a license to use the confluent community. hope this helps.

Sorry not sure I've understood, you've mentioned all the possible alternatives. How should be built a complete Community Confluent cluster?

This seems to work: confluentinc/cp-zookeeper: as many replicas as the nodes confluentinc/cp-enterprise-kafka: 1 broker with replication factor ovveride to 1 confluentinc/cp-schema-registry: enabled confluentinc/cp-kafka-rest: enabled confluentinc/cp-kafka-connect: with replication.factor = 1 confluentinc/cp-ksqldb-server: 1 replica confluentinc/cp-enterprise-control-center: disabled

OneCricketeer commented 2 years ago

confluentinc/cp-enterprise-kafka: 1 broker with replication factor ovveride to 1

Note: This is still an enterprise image, as mentioned