[BUG] 502 Bad Gateway using IBM Cloud default Ingress Controller

vctrmn commented 2 years ago

Describe the bug:

I am unable to open Operate, Tasklist and Optimize in the same browser with KeyCloak authentication. I have to use a different browser (private navigation is not enough), one for each component.

Actual behavior:

If I first open Operate in a window (https://operate.xxxxxxxxxxxxxx.com/), I will have a 502 Bad Gateway for Tasklist (https://tasklist.xxxxxxxxxxxxxx.com/) or Optimize (https://optimize.xxxxxxxxxxxxxx.com/).

Expected behavior:

I should be able to open Operate, Tasklist and Optimize in the same browser.

Environment:

Platform: IBM Cloud Kubernetes Service
Chart version: 8.0.14
Values file:

# Default values for Camunda Platform helm.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

# The values file follows helm best practices https://helm.sh/docs/chart_best_practices/values/
#
# This means:
#   * Variable names should begin with a lowercase letter, and words should be separated with camelcase.
#   * Every defined property in values.yaml should be documented. The documentation string should begin with the name of the property that it describes, and then give at least a one-sentence description
#
# Furthermore, we try to apply the following pattern: # [VarName] [conjunction] [definition]
#
# VarName:
#
#  * In the documentation the variable name is started with a big letter, similar to kubernetes resource documentation.
#  * If the variable is part of a subsection/object we use a json path expression (to make it more clear where the variable belongs to).
#    The root (chart name) is omitted (e.g. zeebe). This is useful for using --set in helm.
#
# Conjunction:
#   * [defines] for mandatory configuration
#   * [can be used] for optional configuration
#   * [if true] for toggles
#   * [configuration] for section/group of variables

# Global configuration for variables which can be accessed by all sub charts
global:

  # Annotations can be used to define common annotations, which should be applied to all deployments
  annotations: {}
  # Labels can be used to define common labels, which should be applied to all deployments
  labels:
    app: camunda-platform

  # Image configuration to be used in each sub chart
  image:
    # Image.tag defines the tag / version which should be used in the chart
    tag: 8.0.0
    # Image.pullPolicy defines the image pull policy which should be used https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy
    pullPolicy: IfNotPresent
    # Image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
    pullSecrets: []

  # Ingress configuration to configure the ingress resource
  ingress:
    # Ingress.enabled if true, an ingress resource is deployed. Only useful if an ingress controller is available, like Ingress-NGINX.
    enabled: true
    # Ingress.className defines the class or configuration of ingress which should be used by the controller
    className: public-iks-k8s-nginx
    # Ingress.annotations defines the ingress related annotations, consumed mostly by the ingress controller
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    # Ingress.host can be used to define the host of the ingress rule. https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    # If not specified the rules applies to all inbound http traffic, if specified the rule applies to that host.
    host: cwa.xxxxxxxxxxxxxx.com
    # Ingress.tls configuration for tls on the ingress resource https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: xxxxxxxxxxxxxx.com

  # Elasticsearch configuration which is shared between the sub charts
  elasticsearch:
    # Elasticsearch.disableExporter if true, disables the elastic exporter in zeebe
    disableExporter: false
    # Elasticsearch.url can be used to configure the URL to access elasticsearch, if not set services fallback to host and port configuration
    url:
    # Elasticsearch.host defines the elasticsearch host, ideally the service name inside the namespace
    host: "elasticsearch-master"
    # Elasticsearch.port defines the elasticsearch port, under which elasticsearch can be accessed
    port: 9200
    # Elasticsearch.clusterName defines the cluster name which is used by Elasticsearch
    clusterName: "elasticsearch"
    # Elasticsearch.prefix defines the prefix which is used by the Zeebe Elasticsearch Exporter to create Elasticsearch indexes
    prefix: zeebe-record
  # ZeebeClusterName defines the cluster name for the Zeebe cluster. All Zeebe pods get this prefix in their name and the brokers uses that as cluster name.
  zeebeClusterName: "{{ .Release.Name }}-zeebe"
  # ZeebePort defines the port which is used for the Zeebe Gateway. This port accepts the GRPC Client messages and forwards them to the Zeebe Brokers.
  zeebePort: 26500

  # Identity configuration to configure identity specifics on global level, which can be accessed by other sub-charts
  identity:
    keycloak:
      # Identity.keycloak.fullname can be used to change the referenced Keycloak service name inside the sub-charts, like operate, optimize, etc.
      # Subcharts can't access values from other sub-charts or the parent, global only.
      # This is useful if the identity.keycloak.fullnameOverride is set, and specifies a different name for the Keycloak service.
      fullname: ""
    # Identity.auth configuration, to configure Identity authentication setup
    auth:
      # Identity.auth.enabled if true, enables the Identity authentication otherwise basic-auth will be used on all services.
      enabled: true

      # Identity.auth.publicIssuerUrl defines the token issuer (Keycloak) URL, where the services can request JWT tokens.
      # Should be public accessible, per default we assume a port-forward to Keycloak (18080) is created before login.
      # Can be overwritten if, ingress is in use and an external IP is available.
      publicIssuerUrl: "https://keycloak.xxxxxxxxxxxxxx.com/auth/realms/camunda-platform"

      # Identity.auth.operate configuration to configure Operate authentication specifics on global level, which can be accessed by other sub-charts
      operate:
        # Identity.auth.operate.existingSecret can be used to reference an existing secret. If not set, a random secret is generated.
        # The existing secret should contain an `operate-secret` field, which will be used as secret for the Identity-Operate communication.
        existingSecret:
        # Identity.auth.operate.redirectUrl defines the redirect URL, which is used by Keycloak to access Operate.
        # Should be public accessible, the default value works if port-forward to Operate is created to 8081.
        # Can be overwritten if, ingress is in use and an external IP is available.
        redirectUrl: "https://operate.xxxxxxxxxxxxxx.com"

      # Identity.auth.tasklist configuration to configure Tasklist authentication specifics on global level, which can be accessed by other sub-charts
      tasklist:
        # Identity.auth.tasklist.existingSecret can be used to use an own existing secret. If not set a random secret is generated.
        # The existing secret should contain an `tasklist-secret` field, which will be used as secret for the Identity-Tasklist communication.
        existingSecret:
        # Identity.auth.tasklist.redirectUrl defines the root (or redirect) URL, which is used by Keycloak to access Tasklist.
        # Should be public accessible, the default value works if port-forward to Tasklist is created to 8082.
        # Can be overwritten if, ingress is in use and an external IP is available.
        redirectUrl: "https://tasklist.xxxxxxxxxxxxxx.com"

      # Identity.auth.optimize configuration to configure Optimize authentication specifics on global level, which can be accessed by other sub-charts
      optimize:
        # Identity.auth.optimize.existingSecret can be used to use an own existing secret. If not set a random secret is generated.
        # The existing secret should contain an `optimize-secret` field, which will be used as secret for the Identity-Optimize communication.
        existingSecret:
        # Identity.auth.optimize.redirectUrl defines the root (or redirect) URL, which is used by Keycloak to access Optimize.
        # Should be public accessible, the default value works if port-forward to Optimize is created to 8082.
        # Can be overwritten if, ingress is in use and an external IP is available.
        redirectUrl: "https://optimize.xxxxxxxxxxxxxx.com"

# Zeebe configuration for the Zeebe sub chart. Contains configuration for the Zeebe broker and related resources.
zeebe:
  # Enabled if true, all zeebe related resources are deployed via the helm release
  enabled: true

  # Image configuration to configure the zeebe image specifics
  image:
    # Image.repository defines which image repository to use
    repository: camunda/zeebe
    # Image.tag can be set to overwrite the global tag, which should be used in that chart
    tag:
    # Image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
    pullSecrets: []

  # ClusterSize defines the amount of brokers (=replicas), which are deployed via helm
  clusterSize: "1"
  # PartitionCount defines how many zeebe partitions are set up in the cluster
  partitionCount: "1"
  # ReplicationFactor defines how each partition is replicated, the value defines the number of nodes
  replicationFactor: "1"
  # Env can be used to set extra environment variables in each zeebe broker container
  env:
    - name: ZEEBE_BROKER_DATA_SNAPSHOTPERIOD
      value: "5m"
    - name: ZEEBE_BROKER_DATA_DISKUSAGECOMMANDWATERMARK
      value: "0.85"
    - name: ZEEBE_BROKER_DATA_DISKUSAGEREPLICATIONWATERMARK
      value: "0.87"
  # ConfigMap configuration which will be applied to the mounted config map.
  configMap:
    # ConfigMap.defaultMode can be used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511.
    # See https://github.com/kubernetes/api/blob/master/core/v1/types.go#L1615-L1623
    defaultMode: 0754
  # Command can be used to override the default command provided by the container image. See https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
  command: []

  # LogLevel defines the log level which is used by the zeebe brokers
  logLevel: info
  # Log4j2 can be used to overwrite the log4j2 configuration of the zeebe brokers
  log4j2: ''
  # JavaOpts can be used to set java options for the zeebe brokers
  javaOpts: >-
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:HeapDumpPath=/usr/local/zeebe/data
    -XX:ErrorFile=/usr/local/zeebe/data/zeebe_error%p.log
    -XX:+ExitOnOutOfMemoryError

 # Service configuration for the broker service
  service:
    # Service.type defines the type of the service https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
    type: ClusterIP
    # Service.httpPort defines the port of the http endpoint, where for example metrics are provided
    httpPort: 9600
    # Service.httpName defines the name of the http endpoint, where for example metrics are provided
    httpName: "http"
    # Service.commandPort defines the port of the command api endpoint, where the broker commands are sent to
    commandPort: 26501
    # Service.commandName defines the name of the command api endpoint, where the broker commands are sent to
    commandName: "command"
    # Service.internalPort defines the port of the internal api endpoint, which is used for internal communication
    internalPort: 26502
    # Service.internalName defines the name of the internal api endpoint, which is used for internal communication
    internalName: "internal"
    # extraPorts can be used to expose any other ports which are required. Can be useful for exporters
    extraPorts: []
      # - name: hazelcast
      #   protocol: TCP
      #   port: 5701
      #   targetPort: 5701

  # ServiceAccount configuration for the service account where the broker pods are assigned to
  serviceAccount:
    # ServiceAccount.enabled if true, enables the broker service account
    enabled: true
    # ServiceAccount.name can be used to set the name of the broker service account
    name: ""
    # ServiceAccount.annotations can be used to set the annotations of the broker service account
    annotations: {}

  # CpuThreadCount defines how many threads can be used for the processing on each broker pod
  cpuThreadCount: "3"
  # IoThreadCount defines how many threads can be used for the exporting on each broker pod
  ioThreadCount: "3"
  # Resources configuration to set request and limit configuration for the container https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
  resources:
    requests:
      cpu: 800m
      memory: 1200Mi
    limits:
      cpu: 960m
      memory: 1920Mi

  # PersistenceType defines the type of persistence which is used by Zeebe. Possible values are: disk, local and memory.
  #   disk  - means a persistence volume claim is configured and used
  #   local - means the data is stored into the container, no volumeMount nor volume nor claim is configured
  #   memory   - means zeebe uses a tmpfs for the data persistence, be aware that this takes the limits into account
  persistenceType: disk
  # PvcSize defines the persistent volume claim size, which is used by each broker pod https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims
  pvcSize: "16Gi"
  # PvcAccessModes can be used to configure the persistent volume claim access mode https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes
  pvcAccessModes: ["ReadWriteOnce"]
  # PvcStorageClassName can be used to set the storage class name which should be used by the persistent volume claim. It is recommended to use a storage class, which is backed with a SSD.
  pvcStorageClassName: ''

  # ExtraVolumes can be used to define extra volumes for the broker pods, useful for additional exporters
  extraVolumes: []
  # ExtraVolumeMounts can be used to mount extra volumes for the broker pods, useful for additional exporters
  extraVolumeMounts: []
  # ExtraInitContainers can be used to set up extra init containers for the broker pods, useful for additional exporters
  extraInitContainers: []

  # PodAnnotations can be used to define extra broker pod annotations
  podAnnotations: {}
  # PodLabels can be used to define extra broker pod labels
  podLabels: {}
  # PodDisruptionBudget configuration to configure a pod disruption budget for the broker pods https://kubernetes.io/docs/tasks/run-application/configure-pdb/
  podDisruptionBudget:
    # PodDisruptionBudget.enabled if true a pod disruption budget is defined for the brokers
    enabled: false
    # PodDisruptionBudget.minAvailable can be used to set how many pods should be available. Be aware that if minAvailable is set, maxUnavailable will not be set (they are mutually exclusive).
    minAvailable:
    # podDisruptionBudget.maxUnavailable can be used to set how many pods should be at max. unavailable
    maxUnavailable: 1

  # PodSecurityContext defines the security options the Zeebe broker pod should be run with
  podSecurityContext: {}

  # ContainerSecurityContext defines the security options the Zeebe broker container should be run with
  containerSecurityContext: {}

  # NodeSelector can be used to define on which nodes the broker pods should run
  nodeSelector: {}
  # Tolerations can be used to define pod toleration's https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  tolerations: []
  # Affinity can be used to define pod affinity or anti-affinity https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
  # The default defined PodAntiAffinity allows constraining on which nodes the Zeebe pods are scheduled on https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity
  # It uses a hard requirement for scheduling and works based on the Zeebe pod labels
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: "app.kubernetes.io/component"
                operator: In
                values:
                  - zeebe-broker
          topologyKey: "kubernetes.io/hostname"

  # PriorityClassName can be used to define the broker pods priority https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass
  priorityClassName: ""

  # ReadinessProbe configuration for the zeebe broker readiness probe
  readinessProbe:
    # ReadinessProbe.probePath defines the readiness probe route used on the zeebe brokers
    probePath: /ready
    # ReadinessProbe.periodSeconds defines how often the probe is executed
    periodSeconds: 10
    # ReadinessProbe.successThreshold defines how often it needs to be true to be marked as ready, after failure
    successThreshold: 1
    # ReadinessProbe.timeoutSeconds defines the seconds after the probe times out
    timeoutSeconds: 1

# Gateway configuration to define properties related to the standalone gateway
zeebe-gateway:
  # Replicas defines how many standalone gateways are deployed
  replicas: 1
  # Image configuration to configure the zeebe-gateway image specifics
  image:
    # Image.repository defines which image repository to use
    repository: camunda/zeebe
    # Image.tag can be set to overwrite the global tag, which should be used in that chart
    tag:
    # Image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
    pullSecrets: []
  # PodAnnotations can be used to define extra gateway pod annotations
  podAnnotations: {}
  # PodLabels can be used to define extra gateway pod labels
  podLabels: {}

  # LogLevel defines the log level which is used by the gateway
  logLevel: info
  # Log4j2 can be used to overwrite the log4j2 configuration of the gateway
  log4j2: ''
  # JavaOpts can be used to set java options for the zeebe gateways
  javaOpts: >-
    -XX:+ExitOnOutOfMemoryError

  # Env can be used to set extra environment variables in each gateway container
  env: []
  # ConfigMap configuration which will be applied to the mounted config map.
  configMap:
    # ConfigMap.defaultMode can be used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511.
    # See https://github.com/kubernetes/api/blob/master/core/v1/types.go#L1615-L1623
    defaultMode: 0744
  # Command can be used to override the default command provided by the container image. See https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
  command: []

  # PodDisruptionBudget configuration to configure a pod disruption budget for the gateway pods https://kubernetes.io/docs/tasks/run-application/configure-pdb/
  podDisruptionBudget:
    # PodDisruptionBudget.enabled if true a pod disruption budget is defined for the gateways
    enabled: false
    # PodDisruptionBudget.minAvailable can be used to set how many pods should be available. Be aware that if minAvailable is set, maxUnavailable will not be set (they are mutually exclusive).
    minAvailable: 1
    # PodDisruptionBudget.maxUnavailable can be used to set how many pods should be at max. unavailable
    maxUnavailable:

  # Resources configuration to set request and limit configuration for the container https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
  resources:
    requests:
      cpu: 400m
      memory: 450Mi
    limits:
      cpu: 400m
      memory: 450Mi

  # PriorityClassName can be used to define the gateway pods priority https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass
  priorityClassName: ""

  # PodSecurityContext defines the security options the gateway pod should be run with
  podSecurityContext: {}

  # ContainerSecurityContext defines the security options the gateway container should be run with
  containerSecurityContext: {}

  # NodeSelector can be used to define on which nodes the gateway pods should run
  nodeSelector: {}
  # Tolerations can be used to define pod toleration's https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  tolerations: []
  # Affinity can be used to define pod affinity or anti-affinity https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
  # The default defined PodAntiAffinity allows constraining on which nodes the Zeebe gateway pods are scheduled on https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity
  # It uses a hard requirement for scheduling and works based on the Zeebe gateway pod labels
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: "app.kubernetes.io/component"
                operator: In
                values:
                  - zeebe-gateway
          topologyKey: "kubernetes.io/hostname"

  # ExtraVolumeMounts can be used to mount extra volumes for the gateway pods, useful for enabling tls between gateway and broker
  extraVolumeMounts: []
  # ExtraVolumes can be used to define extra volumes for the gateway pods, useful for enabling tls between gateway and broker
  extraVolumes: []
  # ExtraInitContainers can be used to set up extra init containers for the gateway pods, useful for adding interceptors
  extraInitContainers: []

  # Service configuration for the gateway service
  service:
    # Service.type defines the type of the service https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
    type: ClusterIP
    # Service.loadBalancerIP defines public ip of the load balancer if the type is LoadBalancer
    loadBalancerIP: ""
    # Service.loadBalancerSourceRanges defines list of allowed source ip address ranges if the type is LoadBalancer
    loadBalancerSourceRanges: []
    # Service.httpPort defines the port of the http endpoint, where for example metrics are provided
    httpPort: 9600
    # Service.httpName defines the name of the http endpoint, where for example metrics are provided
    httpName: "http"
    # Service.gatewayPort defines the port of the gateway endpoint, where client commands (grpc) are sent to
    gatewayPort: 26500
    # Service.gatewayName defines the name of the gateway endpoint, where client commands (grpc) are sent to
    gatewayName: "gateway"
    # Service.internalPort defines the port of the internal api endpoint, which is used for internal communication
    internalPort: 26502
    # Service.internalName defines the name of the internal api endpoint, which is used for internal communication
    internalName: "internal"
    # Service.annotations can be used to define annotations, which will be applied to the zeebe-gateway service
    annotations: {}

  # ServiceAccount configuration for the service account where the gateway pods are assigned to
  serviceAccount:
    # ServiceAccount.enabled if true, enables the gateway service account
    enabled: true
    # ServiceAccount.name can be used to set the name of the gateway service account
    name: ""
    # ServiceAccount.annotations can be used to set the annotations of the gateway service account
    annotations: {}

  # Ingress configuration to configure the ingress resource
  ingress:
    # Ingress.enabled if true, an ingress resource is deployed with the Zeebe gateway deployment. Only useful if an ingress controller is available, like nginx.
    enabled: true
    # Ingress.className defines the class or configuration of ingress which should be used by the controller
    className: public-iks-k8s-nginx
    # Ingress.annotations defines the ingress related annotations, consumed mostly by the ingress controller
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
    # Ingress.path defines the path which is associated with the operate service and port https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    path: /
    # Ingress.host can be used to define the host of the ingress rule. https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    # If not specified the rules applies to all inbound http traffic, if specified the rule applies to that host.
    host: zeebe.xxxxxxxxxxxxxx.com
    # Ingress.tls configuration for tls on the ingress resource https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: xxxxxxxxxxxxxx.com

# Operate configuration for the Operate sub chart.
operate:
  # Enabled if true, the Operate deployment and its related resources are deployed via a helm release
  enabled: true

  # Image configuration to configure the Operate image specifics
  image:
    # Image.repository defines which image repository to use
    repository: camunda/operate
    # Image.tag can be set to overwrite the global tag, which should be used in that chart
    tag:
    # Image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
    pullSecrets: []

  # ContextPath can be used to make Operate web application works on a custom sub-path. This is mainly used to run Camunda Platform web applications under a single domain.
  # contextPath: "/operate"

  # PodAnnotations can be used to define extra Operate pod annotations
  podAnnotations: {}
  # PodLabels can be used to define extra Operate pod labels
  podLabels: {}

  # Logging configuration for the Operate logging. This template will be directly included in the Operate configuration yaml file
  logging:
    level:
      ROOT: INFO
      io.camunda.operate: DEBUG

  # Service configuration to configure the Operate service.
  service:
    # Service.type defines the type of the service https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
    type: ClusterIP
    # Service.port defines the port of the service, where the Operate web application will be available
    port: 80
    # Service.annotations can be used to define annotations, which will be applied to the Operate service
    annotations: {}

  # Resources configuration to set request and limit configuration for the container https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 2000m
      memory: 2Gi

  # Env can be used to set extra environment variables in each Operate container
  env: []
  # ConfigMap configuration which will be applied to the mounted config map.
  configMap:
    # ConfigMap.defaultMode can be used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511.
    # See https://github.com/kubernetes/api/blob/master/core/v1/types.go#L1615-L1623
    defaultMode: 0744
  # Command can be used to override the default command provided by the container image. See https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
  command: []
  # ExtraVolumes can be used to define extra volumes for the Operate pods, useful for tls and self-signed certificates
  extraVolumes: []
  # ExtraVolumeMounts can be used to mount extra volumes for the Operate pods, useful for tls and self-signed certificates
  extraVolumeMounts: []

  # ServiceAccount configuration for the service account where the Operate pods are assigned to
  serviceAccount:
    # ServiceAccount.enabled if true, enables the Operate service account
    enabled: true
    # ServiceAccount.name can be used to set the name of the Operate service account
    name: ""
    # ServiceAccount.annotations can be used to set the annotations of the Operate service account
    annotations: {}

  # Ingress configuration to configure the ingress resource
  ingress:
    # Ingress.enabled if true, an ingress resource is deployed with the Operate deployment. Only useful if an ingress controller is available, like nginx.
    enabled: true
    # Ingress.className defines the class or configuration of ingress which should be used by the controller
    className: public-iks-k8s-nginx
    # Ingress.annotations defines the ingress related annotations, consumed mostly by the ingress controller
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    # Ingress.path defines the path which is associated with the Operate service and port https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    path: /
    # Ingress.host can be used to define the host of the ingress rule. https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    # If not specified the rules applies to all inbound http traffic, if specified the rule applies to that host.
    host: operate.xxxxxxxxxxxxxx.com
    # Ingress.tls configuration for tls on the ingress resource https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: xxxxxxxxxxxxxx.com

  # PodSecurityContext defines the security options the Operate pod should be run with
  podSecurityContext: {}

  # ContainerSecurityContext defines the security options the Operate container should be run with
  containerSecurityContext: {}

  # NodeSelector can be used to define on which nodes the Operate pods should run
  nodeSelector: {}
  # Tolerations can be used to define pod toleration's https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  tolerations: []
  # Affinity can be used to define pod affinity or anti-affinity https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
  affinity: {}

# Tasklist configuration for the tasklist sub chart.
tasklist:
  # Enabled if true, the tasklist deployment and its related resources are deployed via a helm release
  enabled: true

  # Image configuration to configure the tasklist image specifics
  image:
    # Image.repository defines which image repository to use
    repository: camunda/tasklist
    # Image.tag can be set to overwrite the global tag, which should be used in that chart
    tag:
    # Image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
    pullSecrets: []

  # ContextPath can be used to make Tasklist web application works on a custom sub-path. This is mainly used to run Camunda Platform web applications under a single domain.
  # contextPath: "/tasklist"

  # Env can be used to set extra environment variables on each Tasklist container
  env: []

  # PodAnnotations can be used to define extra Tasklist pod annotations
  podAnnotations: {}
  # PodLabels can be used to define extra tasklist pod labels
  podLabels: {}

  # ConfigMap configuration which will be applied to the mounted config map.
  configMap:
    # ConfigMap.defaultMode can be used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511.
    # See https://github.com/kubernetes/api/blob/master/core/v1/types.go#L1615-L1623
    defaultMode: 0744
  # Command can be used to override the default command provided by the container image. See https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
  command: []
  # Service configuration to configure the tasklist service.
  service:
    # Service.type defines the type of the service https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
    type: ClusterIP
    # Service.port defines the port of the service, where the tasklist web application will be available
    port: 80

  # GraphqlPlaygroundEnabled if true, enables the graphql playground
  graphqlPlaygroundEnabled: ""
  # GraphqlPlaygroundEnabled can be set to include the credentials in each request, should be set to "include" if graphql playground is enabled
  graphqlPlaygroundRequestCredentials: ""

  # ExtraVolumes can be used to define extra volumes for the Tasklist pods, useful for tls and self-signed certificates
  extraVolumes: []
  # ExtraVolumeMounts can be used to mount extra volumes for the Tasklist pods, useful for tls and self-signed certificates
  extraVolumeMounts: []

  # PodSecurityContext defines the security options the Tasklist pod should be run with
  podSecurityContext: {}

  # ContainerSecurityContext defines the security options the Tasklist container should be run with
  containerSecurityContext: {}

  # NodeSelector can be used to define on which nodes the Tasklist pods should run
  nodeSelector: {}
  # Tolerations can be used to define pod toleration's https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  tolerations: []
  # Affinity can be used to define pod affinity or anti-affinity https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
  affinity: {}

  # Resources configuration to set request and limit configuration for the container https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
  resources:
    requests:
      cpu: 400m
      memory: 1Gi
    limits:
      cpu: 1000m
      memory: 2Gi

  # Ingress configuration to configure the ingress resource
  ingress:
    # Ingress.enabled if true, an ingress resource is deployed with the tasklist deployment. Only useful if an ingress controller is available, like nginx.
    enabled: true
    # Ingress.className defines the class or configuration of ingress which should be used by the controller
    className: public-iks-k8s-nginx
    # Ingress.annotations defines the ingress related annotations, consumed mostly by the ingress controller
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    # Ingress.path defines the path which is associated with the operate service and port https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    path: /
    # Ingress.host can be used to define the host of the ingress rule. https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    # If not specified the rules applies to all inbound http traffic, if specified the rule applies to that host.
    host: tasklist.xxxxxxxxxxxxxx.com
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: xxxxxxxxxxxxxx.com

# Optimize configuration for the Optimize sub chart.
optimize:
  # Enabled if true, the Optimize deployment and its related resources are deployed via a helm release
  enabled: true

  # Image configuration to configure the Optimize image specifics
  image:
    # Image.repository defines which image repository to use
    repository: camunda/optimize
    # Image.tag can be set to overwrite the global tag, which should be used in that chart
    tag: 3.9.0-preview-2
    # Image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
    pullSecrets: []

  # ContextPath can be used to make Optimize web application works on a custom sub-path. This is mainly used to run Camunda Platform web applications under a single domain.
  # contextPath: "/optimize"

  # PodAnnotations can be used to define extra Optimize pod annotations
  podAnnotations: {}
  # PodLabels can be used to define extra Optimize pod labels
  podLabels: {}

  # PartitionCount defines how many Zeebe partitions are set up in the cluster and which should be imported by Optimize
  partitionCount: "1"
  # Env can be used to set extra environment variables in each Optimize container
  env: []
  # Command can be used to override the default command provided by the container image. See https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
  command: []
  # ExtraVolumes can be used to define extra volumes for the Optimize pods, useful for tls and self-signed certificates
  extraVolumes: []
  # ExtraVolumeMounts can be used to mount extra volumes for the Optimize pods, useful for tls and self-signed certificates
  extraVolumeMounts: []

  # ServiceAccount configuration for the service account where the Optimize pods are assigned to
  serviceAccount:
    # ServiceAccount.enabled if true, enables the Optimize service account
    enabled: true
    # ServiceAccount.name can be used to set the name of the Optimize service account
    name: ""
    # ServiceAccount.annotations can be used to set the annotations of the Optimize service account
    annotations: {}

  # Service configuration to configure the Optimize service.
  service:
    # Service.type defines the type of the service https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
    type: ClusterIP
    # Service.port defines the port of the service, where the Optimize web application will be available
    port: 80
    # Service.annotations can be used to define annotations, which will be applied to the Optimize service
    annotations: {}

  # PodSecurityContext defines the security options the Optimize pod should be run with
  podSecurityContext: {}

  # ContainerSecurityContext defines the security options the Optimize container should be run with
  containerSecurityContext: {}

  # NodeSelector can be used to define on which nodes the Optimize pods should run
  nodeSelector: {}
  # Tolerations can be used to define pod toleration's https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  tolerations: []
  # Affinity can be used to define pod affinity or anti-affinity https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
  affinity: {}

  # Resources configuration to set request and limit configuration for the container https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
  resources:
    requests:
      cpu: 600m
      memory: 1Gi
    limits:
      cpu: 2000m
      memory: 2Gi

  # Ingress configuration to configure the ingress resource
  ingress:
    # Ingress.enabled if true, an ingress resource is deployed with the Optimize deployment. Only useful if an ingress controller is available, like nginx.
    enabled: true
    # Ingress.className defines the class or configuration of ingress which should be used by the controller
    className: public-iks-k8s-nginx
    # Ingress.annotations defines the ingress related annotations, consumed mostly by the ingress controller
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    # Ingress.path defines the path which is associated with the operate service and port https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    path: /
    # Ingress.host can be used to define the host of the ingress rule. https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    # If not specified the rules applies to all inbound http traffic, if specified the rule applies to that host.
    host: optimize.xxxxxxxxxxxxxx.com
    # Ingress.tls configuration for tls on the ingress resource https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: xxxxxxxxxxxxxx.com

# RetentionPolicy configuration to configure the elasticsearch index retention policies
retentionPolicy:
  # RetentionPolicy.enabled if true, elasticsearch curator cronjob and configuration will be deployed.
  enabled: false
  # RetentionPolicy.schedule defines how often/when the curator should run
  schedule: "0 0 * * *"
  # RetentionPolicy.zeebeIndexTTL defines after how many days a zeebe index can be deleted
  zeebeIndexTTL: 1
  # RetentionPolicy.zeebeIndexMaxSize can be set to configure the maximum allowed zeebe index size in gigabytes.
  # After reaching that size, curator will delete that corresponding index on the next run.
  # To benefit from that configuration the schedule needs to be configured small enough, like every 15 minutes.
  zeebeIndexMaxSize:
  # RetentionPolicy.operateIndexTTL defines after how many days an operate index can be deleted
  operateIndexTTL: 30
  # RetentionPolicy.tasklistIndexTTL defines after how many days a tasklist index can be deleted
  tasklistIndexTTL: 30

  # Image configuration for the elasticsearch curator cronjob
  image:
    # Image.repository defines which image repository to use
    repository: bitnami/elasticsearch-curator
    # Image.tag defines the tag / version which should be used in the chart
    tag: 5.8.4

# PrometheusServiceMonitor configuration to configure a prometheus service monitor
prometheusServiceMonitor:
  # PrometheusServiceMonitor.enabled if true then a service monitor will be deployed, which allows an installed prometheus controller to scrape metrics from the deployed pods
  enabled: false
  # PromotheuServiceMonitor.labels can be set to configure extra labels, which will be added to the servicemonitor and can be used on the prometheus controller for selecting the servicemonitors
  labels:
    release: metrics
  # PromotheuServiceMonitor.scrapeInterval can be set to configure the interval at which metrics should be scraped
  # Should be *less* than 60s if the provided grafana dashboard is used, which can be found here https://github.com/camunda/zeebe/tree/main/monitor/grafana,
  # otherwise it isn't able to show any metrics which is aggregated over 1 min.
  scrapeInterval: 10s

# Identity configuration for the identity sub chart.
identity:
  # Enabled if true, the identity deployment and its related resources are deployed via a helm release
  #
  # Note: Identity is required by Optimize. If Identity is disabled, then Optimize will be unusable.
  #       If you don't need Optimize, then make sure to disable both: set global.identity.auth.enabled=false AND optimize.enabled=false.
  enabled: true

  # FirstUser configuration to configure properties of the first Identity user, which can be used to access all
  # web applications
  firstUser:
    # FirstUser.username defines the username of the first user, needed to log in into the web applications
    username: demo
    # FirstUser.password defines the password of the first user, needed to log in into the web applications
    password: demo

  # Image configuration to configure the identity image specifics
  image:
    # Image.repository defines which image repository to use
    repository: camunda/identity
    # Image.tag can be set to overwrite the global tag, which should be used in that chart
    tag:
    # Image.pullSecrets can be used to configure image pull secrets https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
    pullSecrets: []

  # FullURL can be used when Ingress is configured (for both multi and single domain setup).
  # Note: If the `ContextPath` is configured, then value of `ContextPath` should be included in the URL too.
  # fullURL: "https://camunda.example.com/identity"

  # ContextPath can be used to make Identity web application works on a custom sub-path. This is mainly used to run Camunda Platform web applications under a single domain.
  # contextPath: "/identity"

  # PodAnnotations can be used to define extra Identity pod annotations
  podAnnotations: {}

  # Service configuration to configure the identity service.
  service:
    # Service.type defines the type of the service https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
    type: ClusterIP
    # Service.port defines the port of the service, where the identity web application will be available
    port: 80
    # Service.annotations can be used to define annotations, which will be applied to the identity service
    annotations: {}

  # PodSecurityContext defines the security options the Identity pod should be run with
  podSecurityContext: {}

  # ContainerSecurityContext defines the security options the Identity container should be run with
  containerSecurityContext: {}

  # NodeSelector can be used to define on which nodes the Identity pods should run
  nodeSelector: {}
  # Tolerations can be used to define pod toleration's https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  tolerations: []
  # Affinity can be used to define pod affinity or anti-affinity https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
  affinity: {}

  # Resources configuration to set request and limit configuration for the container https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 2000m
      memory: 2Gi

  # Env can be used to set extra environment variables in each identity container. See the documentation https://docs.camunda.io/docs/self-managed/identity/deployment/configuration-variables/ for more details.
  env: []
  # Command can be used to override the default command provided by the container image. See https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/
  command: []
  # ExtraVolumes can be used to define extra volumes for the identity pods, useful for tls and self-signed certificates
  extraVolumes: []
  # ExtraVolumeMounts can be used to mount extra volumes for the identity pods, useful for tls and self-signed certificates
  extraVolumeMounts: []

  # Keycloak configuration, for the keycloak dependency chart which is used by identity. See the chart documentation https://github.com/bitnami/charts/tree/master/bitnami/keycloak#parameters for more details.
  keycloak:
    # Keycloak.service configuration, to configure the service which is deployed along with keycloak
    service:
      # Keycloak.service.type can be set to change the service type.
      # We use clusterIP for keycloak service, since per default LoadBalancer is used, which is not supported on all cloud providers.
      # This might prevent scheduling of the service.
      type: ClusterIP
    ## Keycloak authentication parameters
    ## ref: https://github.com/bitnami/bitnami-docker-keycloak#admin-credentials
    ##
    ## Identity uses the secrets generated by keycloak, to access keycloak.
    auth:
      # Keycloak.auth.adminUser defines the keycloak administrator user
      adminUser: admin
      # Keycloak.auth.existingSecret can be used to reuse an existing secret containing authentication information.
      # See https://docs.bitnami.com/kubernetes/apps/keycloak/configuration/manage-passwords/ for more details.
      #
      # Example:
      #
      # Keycloak.auth.existingSecret:
      #   name: mySecret
      #   keyMapping:
      #     admin-password: myPasswordKey
      #     management-password: myManagementPasswordKey
      #     tls-keystore-password: myTlsKeystorePasswordKey
      #     tls-truestore-password: myTlsTruestorePasswordKey
      existingSecret: ""

  # ServiceAccount configuration for the service account where the identity pods are assigned to
  serviceAccount:
    # ServiceAccount.enabled if true, enables the identity service account
    enabled: true
    # ServiceAccount.name can be used to set the name of the identity service account
    name: ""
    # ServiceAccount.annotations can be used to set the annotations of the identity service account
    annotations: {}

  # Ingress configuration to configure the ingress resource
  ingress:
    # Ingress.enabled if true, an ingress resource is deployed with the identity deployment. Only useful if an ingress controller is available, like nginx.
    enabled: true
    # Ingress.className defines the class or configuration of ingress which should be used by the controller
    className: public-iks-k8s-nginx
    # Ingress.annotations defines the ingress related annotations, consumed mostly by the ingress controller
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    # Ingress.path defines the path which is associated with the operate service and port https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    path: /
    # Ingress.host can be used to define the host of the ingress rule. https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
    # If not specified the rules applies to all inbound http traffic, if specified the rule applies to that host.
    host: identity.xxxxxxxxxxxxxx.com
    # Ingress.tls configuration for tls on the ingress resource https://kubernetes.io/docs/concepts/services-networking/ingress/#tls
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: xxxxxxxxxxxxxx.com

elasticsearch:
  enabled: true
  extraEnvs:
    - name: "xpack.security.enabled"
      value: "false"

  replicas: 1

  persistence:
    labels:
      enabled: true

  volumeClaimTemplate:
    accessModes: ["ReadWriteOnce"]
    resources:
      requests:
        storage: 16Gi

  esJavaOpts: "-Xmx1g -Xms1g"

  resources:
    requests:
      cpu: 1
      memory: 1Gi
    limits:
      cpu: 2
      memory: 2Gi

aabouzaid commented 2 years ago

Before inspecting this issue, did you try to use the latest batch version for v8.0? 8.0.7 or the latest v8.1? 8.1.1

vctrmn commented 2 years ago

Hi @aabouzaid,

I tried both (8.0.7 and 8.1.1), same behaviour

Here is a simplified version of the values.yaml :

# Chart values for the Camunda Platform 8 Helm chart in combined Ingress setup.

# This file deliberately contains only the values that differ from the defaults.
# For changes and documentation, use your favorite diff tool to compare it with:
# https://github.com/camunda/camunda-platform-helm/blob/main/charts/camunda-platform

global:
  # Image configuration to be used in each sub chart
  image:
    # Image.tag defines the tag / version which should be used in the chart
    tag: 8.1.1

  identity:
    auth:
      publicIssuerUrl: "https://keycloak.my-domain.com/auth/realms/camunda-platform"
      operate:
        redirectUrl: "https://operate.my-domain.com"
      tasklist:
        redirectUrl: "https://tasklist.my-domain.com"
      optimize:
        redirectUrl: "https://optimize.my-domain.com"

zeebe:
  # ClusterSize defines the amount of brokers (=replicas), which are deployed via helm
  clusterSize: "1"
  # PartitionCount defines how many zeebe partitions are set up in the cluster
  partitionCount: "1"
  # ReplicationFactor defines how each partition is replicated, the value defines the number of nodes
  replicationFactor: "1"

  resources:
    requests:
      cpu: 800m
      memory: 1200Mi
    limits:
      cpu: 800m
      memory: 1200Mi

  # PvcSize defines the persistent volume claim size, which is used by each broker pod https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims
  pvcSize: "16Gi"

zeebe-gateway:
  # Replicas defines how many standalone gateways are deployed
  replicas: 1

  ingress:
    enabled: true
    className: public-iks-k8s-nginx
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
    host: "zeebe.my-domain.com"
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: my-domain.com

operate:
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 600m
      memory: 400Mi

  ingress:
    enabled: true
    className: public-iks-k8s-nginx
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    host: "operate.my-domain.com"
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: my-domain.com

optimize:
  # PartitionCount defines how many Zeebe partitions are set up in the cluster and which should be imported by Optimize
  partitionCount: "1"

  resources:
    requests:
      cpu: 600m
      memory: 1Gi
    limits:
      cpu: 600m
      memory: 1Gi

  ingress:
    enabled: true
    className: public-iks-k8s-nginx
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    host: "optimize.my-domain.com"
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: my-domain.com

tasklist:
  resources:
    requests:
      cpu: 400m
      memory: 1Gi
    limits:
      cpu: 400m
      memory: 1Gi

  ingress:
    enabled: true
    className: public-iks-k8s-nginx
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    host: "tasklist.my-domain.com"
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: my-domain.com

identity:
  ingress:
    enabled: true
    className: public-iks-k8s-nginx
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    host: "identity.my-domain.com"
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: my-domain.com
  fullURL: "https://identity.my-domain.com"
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 600m
      memory: 400Mi

  keycloak:
    ingress:
      enabled: true
      ingressClassName: public-iks-k8s-nginx
      hostname: "keycloak.my-domain.com"
    extraEnvVars:
      - name: KEYCLOAK_PROXY_ADDRESS_FORWARDING
        value: "true"
      - name: KEYCLOAK_FRONTEND_URL
        value: "https://keycloak.my-domain.com"

elasticsearch:
  enabled: true

  replicas: 1

  volumeClaimTemplate:
    accessModes: ["ReadWriteOnce"]
    resources:
      requests:
        storage: 16Gi

  esJavaOpts: "-Xmx1g -Xms1g"

  resources:
    requests:
      cpu: 1
      memory: 1Gi
    limits:
      cpu: 2
      memory: 2Gi

vctrmn commented 2 years ago

Same behaviour with a combined ingress :

# Chart values for the Camunda Platform 8 Helm chart in combined Ingress setup.

# This file deliberately contains only the values that differ from the defaults.
# For changes and documentation, use your favorite diff tool to compare it with:
# https://github.com/camunda/camunda-platform-helm/blob/main/charts/camunda-platform

global:
  # Image configuration to be used in each sub chart
  image:
    # Image.tag defines the tag / version which should be used in the chart
    tag: 8.1.1

  ingress:
    enabled: true
    className: public-iks-k8s-nginx
    host: "my-domain.com"
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: my-domain.com

  identity:
    auth:
      publicIssuerUrl: "https://my-domain.com/auth/realms/camunda-platform"
      operate:
        redirectUrl: "https://my-domain.com/operate"
      tasklist:
        redirectUrl: "https://my-domain.com/tasklist"
      optimize:
        redirectUrl: "https://my-domain.com/optimize"

zeebe:
  # ClusterSize defines the amount of brokers (=replicas), which are deployed via helm
  clusterSize: "1"
  # PartitionCount defines how many zeebe partitions are set up in the cluster
  partitionCount: "1"
  # ReplicationFactor defines how each partition is replicated, the value defines the number of nodes
  replicationFactor: "1"

  resources:
    requests:
      cpu: 800m
      memory: 1200Mi
    limits:
      cpu: 800m
      memory: 1200Mi

  # PvcSize defines the persistent volume claim size, which is used by each broker pod https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims
  pvcSize: "16Gi"

zeebe-gateway:
  # Replicas defines how many standalone gateways are deployed
  replicas: 1

  ingress:
    enabled: true
    className: public-iks-k8s-nginx
    annotations:
      ingress.kubernetes.io/rewrite-target: "/"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
    host: "zeebe.my-domain.com"
    tls:
      # Ingress.tls.enabled if true, then tls is configured on the ingress resource. If enabled the Ingress.host need to be defined.
      enabled: true
      # Ingress.tls.secretName defines the secret name which contains the TLS private key and certificate
      secretName: my-domain.com

operate:
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 600m
      memory: 400Mi

  contextPath: "/operate"

optimize:
  # PartitionCount defines how many Zeebe partitions are set up in the cluster and which should be imported by Optimize
  partitionCount: "1"

  resources:
    requests:
      cpu: 600m
      memory: 1Gi
    limits:
      cpu: 600m
      memory: 1Gi

  contextPath: "/optimize"

tasklist:
  resources:
    requests:
      cpu: 400m
      memory: 1Gi
    limits:
      cpu: 400m
      memory: 1Gi

  contextPath: "/tasklist"

identity:
  resources:
    requests:
      cpu: 600m
      memory: 400Mi
    limits:
      cpu: 600m
      memory: 400Mi

  contextPath: "/identity"
  fullURL: "https://my-domain.com/identity"

  keycloak:
    extraEnvVars:
      - name: KEYCLOAK_PROXY_ADDRESS_FORWARDING
        value: "true"
      - name: KEYCLOAK_FRONTEND_URL
        value: "https://my-domain.com/auth"

elasticsearch:
  enabled: true

  replicas: 1

  volumeClaimTemplate:
    accessModes: ["ReadWriteOnce"]
    resources:
      requests:
        storage: 16Gi

  esJavaOpts: "-Xmx1g -Xms1g"

  resources:
    requests:
      cpu: 1
      memory: 1Gi
    limits:
      cpu: 2
      memory: 2Gi

aabouzaid commented 2 years ago

@vctrmn could you please get the output of the following commands:

kubectl describe pod -l app.kubernetes.io/component=operate
kubectl describe pod -l app.kubernetes.io/component=optimize
kubectl describe pod -l app.kubernetes.io/component=operate

vctrmn commented 2 years ago

@aabouzaid

Operate :

Name:             cwa-operate-78d7f75f6b-7669t
Namespace:        default
Priority:         0
Service Account:  default
Node:             10.243.64.7/10.243.64.7
Start Time:       Tue, 18 Oct 2022 17:54:55 +0200
Labels:           app=camunda-platform
                  app.kubernetes.io/component=operate
                  app.kubernetes.io/instance=cwa
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=operate
                  app.kubernetes.io/part-of=camunda-platform
                  app.kubernetes.io/version=8.1.1
                  helm.sh/chart=operate-8.0.14
                  pod-template-hash=78d7f75f6b
Annotations:      cni.projectcalico.org/containerID: bb937c1975d303e14cdf6d5a23982d0bd1dd553401fdc0dc28b16af9a919415f
                  cni.projectcalico.org/podIP: 172.17.120.248/32
                  cni.projectcalico.org/podIPs: 172.17.120.248/32
                  kubernetes.io/psp: ibm-privileged-psp
Status:           Running
IP:               172.17.120.248
IPs:
  IP:           172.17.120.248
Controlled By:  ReplicaSet/cwa-operate-78d7f75f6b
Containers:
  operate:
    Container ID:   containerd://a8a9fb7429f15644f1f664132b37cd016ab74e0f18a3628869bddae49bb75518
    Image:          camunda/operate:8.1.1
    Image ID:       docker.io/camunda/operate@sha256:02319da8caef515fa76a14c4b269b949716832951e8109c803b7c0ba6d7b51af
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 18 Oct 2022 17:54:56 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     600m
      memory:  400Mi
    Requests:
      cpu:     600m
      memory:  400Mi
    Environment:
      SPRING_PROFILES_ACTIVE:                               identity-auth
      SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUERURI:  http://cwa-keycloak:80/auth/realms/camunda-platform
      SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_JWKSETURI:  http://cwa-keycloak:80/auth/realms/camunda-platform/protocol/openid-connect/certs
      CAMUNDA_OPERATE_IDENTITY_ISSUER_URL:                  https://keycloak.my-custom-url/auth/realms/camunda-platform
      CAMUNDA_OPERATE_IDENTITY_ISSUER_BACKEND_URL:          http://cwa-keycloak:80/auth/realms/camunda-platform
      CAMUNDA_OPERATE_IDENTITY_CLIENT_ID:                   operate
      CAMUNDA_OPERATE_IDENTITY_CLIENT_SECRET:               <set to the key 'operate-secret' in secret 'cwa-operate-identity-secret'>  Optional: false
      CAMUNDA_OPERATE_IDENTITY_AUDIENCE:                    operate-api
    Mounts:
      /usr/local/operate/config/application.yml from config (rw,path="application.yml")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nzcfm (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      cwa-operate
    Optional:  false
  kube-api-access-nzcfm:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Guaranteed
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 600s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 600s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  2m30s  default-scheduler  Successfully assigned default/cwa-operate-78d7f75f6b-7669t to 10.243.64.7
  Normal  Pulled     2m29s  kubelet            Container image "camunda/operate:8.1.1" already present on machine
  Normal  Created    2m29s  kubelet            Created container operate
  Normal  Started    2m29s  kubelet            Started container operate

Optimize :

Name:             cwa-optimize-6d67bf9978-jdh5d
Namespace:        default
Priority:         0
Service Account:  default
Node:             10.243.64.10/10.243.64.10
Start Time:       Tue, 18 Oct 2022 17:54:55 +0200
Labels:           app=camunda-platform
                  app.kubernetes.io/component=optimize
                  app.kubernetes.io/instance=cwa
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=optimize
                  app.kubernetes.io/part-of=camunda-platform
                  app.kubernetes.io/version=3.9.0-preview-2
                  helm.sh/chart=optimize-8.0.14
                  pod-template-hash=6d67bf9978
Annotations:      cni.projectcalico.org/containerID: 087092c833c0d80c38b892870c081bb94ac122111f500da5a6ff63f019c99b46
                  cni.projectcalico.org/podIP: 172.17.114.127/32
                  cni.projectcalico.org/podIPs: 172.17.114.127/32
                  kubernetes.io/psp: ibm-privileged-psp
Status:           Running
IP:               172.17.114.127
IPs:
  IP:           172.17.114.127
Controlled By:  ReplicaSet/cwa-optimize-6d67bf9978
Containers:
  optimize:
    Container ID:   containerd://9ef27fe2d4a221f5b93a372c79a7246221d68981c4a7f198409944a2bbc949aa
    Image:          camunda/optimize:3.9.0-preview-2
    Image ID:       docker.io/camunda/optimize@sha256:7280013da531f0ff39723730120daf3306719ce10b92d95cfe699bc6b25ac782
    Port:           8090/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 18 Oct 2022 17:54:57 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     600m
      memory:  1Gi
    Requests:
      cpu:     600m
      memory:  1Gi
    Environment:
      CAMUNDA_OPTIMIZE_ZEEBE_ENABLED:                           true
      CAMUNDA_OPTIMIZE_ZEEBE_PARTITION_COUNT:                   1
      OPTIMIZE_ELASTICSEARCH_HOST:                              elasticsearch-master
      OPTIMIZE_ELASTICSEARCH_HTTP_PORT:                         9200
      SPRING_PROFILES_ACTIVE:                                   ccsm
      CAMUNDA_OPTIMIZE_IDENTITY_ISSUER_URL:                     https://keycloak.my-custom-url/auth/realms/camunda-platform
      CAMUNDA_OPTIMIZE_IDENTITY_ISSUER_BACKEND_URL:             http://cwa-keycloak:80/auth/realms/camunda-platform
      CAMUNDA_OPTIMIZE_IDENTITY_CLIENTID:                       optimize
      CAMUNDA_OPTIMIZE_IDENTITY_CLIENTSECRET:                   <set to the key 'optimize-secret' in secret 'cwa-optimize-identity-secret'>  Optional: false
      CAMUNDA_OPTIMIZE_IDENTITY_AUDIENCE:                       optimize-api
      CAMUNDA_OPTIMIZE_API_AUDIENCE:                            optimize-api
      CAMUNDA_OPTIMIZE_API_JWTSETURI:                           http://cwa-keycloak:80/auth/realms/camunda-platform/protocol/openid-connect/certs
      CAMUNDA_OPTIMIZE_SECURITY_AUTH_COOKIE_SAME_SITE_ENABLED:  false
      CAMUNDA_OPTIMIZE_UI_LOGOUT_HIDDEN:                        true
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vm2xs (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-vm2xs:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Guaranteed
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 600s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 600s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  4m55s  default-scheduler  Successfully assigned default/cwa-optimize-6d67bf9978-jdh5d to 10.243.64.10
  Normal  Pulled     4m54s  kubelet            Container image "camunda/optimize:3.9.0-preview-2" already present on machine
  Normal  Created    4m54s  kubelet            Created container optimize
  Normal  Started    4m53s  kubelet            Started container optimize

Tasklist :

Name:             cwa-tasklist-7457bb54ff-rs8kb
Namespace:        default
Priority:         0
Service Account:  default
Node:             10.243.64.10/10.243.64.10
Start Time:       Tue, 18 Oct 2022 17:54:55 +0200
Labels:           app=camunda-platform
                  app.kubernetes.io/component=tasklist
                  app.kubernetes.io/instance=cwa
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=tasklist
                  app.kubernetes.io/part-of=camunda-platform
                  app.kubernetes.io/version=8.1.1
                  helm.sh/chart=tasklist-8.0.14
                  pod-template-hash=7457bb54ff
Annotations:      cni.projectcalico.org/containerID: 56824fa58b1bf005cbebebc641a2ff8ff3379885c156ac47ca8539a2de7e55bb
                  cni.projectcalico.org/podIP: 172.17.114.126/32
                  cni.projectcalico.org/podIPs: 172.17.114.126/32
                  kubernetes.io/psp: ibm-privileged-psp
Status:           Running
IP:               172.17.114.126
IPs:
  IP:           172.17.114.126
Controlled By:  ReplicaSet/cwa-tasklist-7457bb54ff
Containers:
  tasklist:
    Container ID:   containerd://4a5c8e186783553c9dbc07750dbaba02e60b560670c9363b6c772e77893a1d87
    Image:          camunda/tasklist:8.1.1
    Image ID:       docker.io/camunda/tasklist@sha256:398720e2991f68129ebe587583b239e72450cb1464eb79c5eb9f0d05eb7d3f6e
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 18 Oct 2022 17:54:56 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     400m
      memory:  1Gi
    Requests:
      cpu:     400m
      memory:  1Gi
    Environment:
      SPRING_PROFILES_ACTIVE:                               identity-auth
      SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUERURI:  http://cwa-keycloak:80/auth/realms/camunda-platform
      SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_JWKSETURI:  http://cwa-keycloak:80/auth/realms/camunda-platform/protocol/openid-connect/certs
      CAMUNDA_TASKLIST_IDENTITY_ISSUER_URL:                 https://keycloak.my-custom-url/auth/realms/camunda-platform
      CAMUNDA_TASKLIST_IDENTITY_ISSUER_BACKEND_URL:         http://cwa-keycloak:80/auth/realms/camunda-platform
      CAMUNDA_TASKLIST_IDENTITY_CLIENT_ID:                  tasklist
      CAMUNDA_TASKLIST_IDENTITY_CLIENT_SECRET:              <set to the key 'tasklist-secret' in secret 'cwa-tasklist-identity-secret'>  Optional: false
      CAMUNDA_TASKLIST_IDENTITY_AUDIENCE:                   tasklist-api
      GRAPHQL_PLAYGROUND_ENABLED:                           true
      GRAPHQL_PLAYGROUND_SETTINGS_REQUEST_CREDENTIALS:      include
    Mounts:
      /app/resources/application.yml from config (rw,path="application.yml")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gswwr (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      cwa-tasklist
    Optional:  false
  kube-api-access-gswwr:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Guaranteed
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 600s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 600s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  6m27s  default-scheduler  Successfully assigned default/cwa-tasklist-7457bb54ff-rs8kb to 10.243.64.10
  Normal  Pulled     6m27s  kubelet            Container image "camunda/tasklist:8.1.1" already present on machine
  Normal  Created    6m27s  kubelet            Created container tasklist
  Normal  Started    6m27s  kubelet            Started container tasklist

vctrmn commented 2 years ago

Hello @aabouzaid ,

I have news on this : there is no 502 Bad Gateway with the default Ingress Controller in OpenShift Container Platform 4.x (based on HAProxy)

I think it is related to the IKS (IBM Kubernetes Service) default Ingress Controller (based on nginx)

vctrmn commented 2 years ago

Hello @aabouzaid ,

You can close this issue, because this bug only appears with IBM Cloud default Ingress Controller 🙂

aabouzaid commented 2 years ago

@vctrmn thanks a lot for the info :rocket:

I'd say let's keep it open just for visibility in case someone tried to install the charts on IBM Cloud (which is currently not officially supported/tested).

aflansburg commented 1 year ago

This may be unrelated, but it sounds similar to when using the ingress-gce on Google Kubernetes Engine which automagically creates LB, backends, etc, and network endpoint groups (NEG) for you. The LB are looking for a 200 from a health check endpoint and if they do not get a ✅ they throw 502s. No matter how many annotations and tweaks I attempt, there is currently no way to configure the NEG or backends to forward all health checks to the metrics port (82) and the metrics service does not respond w/ 200 on anything from what I can tell. Port 80 should respond to a health check imo..... at least at /healthz if not /actuator/health.

Sidenote, I was able to prove this ^ was the case by setting a gcloud health check to TCP instead of HTTP and the LB processed traffic and was able to access the identity component. However, this is a hack, and the cluster will just update the health check back to HTTP within a few minutes.

jessesimpson36 commented 1 year ago

I'm 90% sure this issue was that you need to increase the nginx proxy-buffer-size. I've seen this 502 Bad Gateway a lot from that proxy-buffer-size option being set to something that's too low by default.

I'm closing this, since it's probably irrelevant.

camunda / camunda-platform-helm

[BUG] 502 Bad Gateway using IBM Cloud default Ingress Controller #442