Consul Chart in User-MLA Stack does not make use of proper Storage Class

[stroebitzer]

@stroebitzer commented on Mon Jul 04 2022

What happened?

On installing consul via

helm --namespace mla upgrade --install consul ~/user-mla/charts/consul --values ~/user-mla/config/consul/values.yaml --set consul.server.storageClass=kubermatic-fast

the pods are stuck in pending state

kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
consul-consul-server-0   0/1     Pending   0          20m
consul-consul-server-1   0/1     Pending   0          20m
consul-consul-server-2   0/1     Pending   0          20m

due to the PVCs are also stuck in pending state

kubectl get pvc
NAME                              STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
data-mla-consul-consul-server-0   Pending                                                                                          15m
data-mla-consul-consul-server-1   Pending                                                                                          15m
data-mla-consul-consul-server-2   Pending                                                                                          15m
storage-grafana-0                 Bound     pvc-932db24e-c6d5-4680-9376-ecf07e620798   10Gi       RWO            kubermatic-fast   18m

=> IMO the reason could be that on GCE PVCs with strategy Immediate instead of WaitForFirstConsumer will not get bound due to maybe the PV was created in a different zone.

I also gave it a try via

helm --namespace mla upgrade --atomic --install consul ~/user-mla/charts/consul --values ~/user-mla/config/consul/values.yaml --set consul.server.storageClass=kubermatic-fast

with the same outcome

Expected behavior

Consul gets installed

How to reproduce the issue?

Try to run User MLA on a KKP installation running on GCE

How is your environment configured?

• KKP version: 2.20.4
• Shared or separate master/seed clusters?: shared

Provide your KKP manifest here (if applicable)

# Copyright 2020 The Kubermatic Kubernetes Platform contributors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: kubermatic.k8c.io/v1
kind: KubermaticConfiguration
metadata:
  name: kubermatic
  namespace: kubermatic
spec:
  featureGates:
    UserClusterMLA: true

  ingress:
    # Domain is the base domain where the dashboard shall be available. Even with
    # a disabled Ingress, this must always be a valid hostname.
    # this domain must match what you configured as dex.ingress.host
    # in the values.yaml
    domain: SAFETY_FIRST
    certificateIssuer:
      # APIGroup is the group for the resource being referenced.
      # If APIGroup is not specified, the specified Kind must be in the core API group.
      # For any other third-party types, APIGroup is required.
      apiGroup: null
      # Kind is the type of resource being referenced
      kind: ClusterIssuer
      # Name is the name of resource being referenced
      # For generating a certificate signed by a trusted root authority replace
      # with "letsencrypt-prod".
      name: "letsencrypt-prod"
    className: nginx

  # These secret keys configure the way components communicate with Dex.
  auth:
    clientID: kubermatic
    issuerClientID: kubermaticIssuer
    # When using letsencrypt-prod replace with "false"
    skipTokenIssuerTLSVerify: false
    tokenIssuer: https://SAFETY_FIRST/dex

    # This must match the secret configured for the kubermaticIssuer client from
    # the dex clients in values.yaml.
    # Needed if the "enableOIDCKubeconfig: true" option is used in KubermaticSetting
    issuerClientSecret: SAFETY_FIRST

    # these need to be randomly generated. Those can be generated on the
    # shell using:
    # cat /dev/urandom | tr -dc A-Za-z0-9 | head -c32
    issuerCookieKey: SAFETY_FIRST
    serviceAccountKey: SAFETY_FIRST

  versions:
    versions:
      - v1.20.13
      - v1.20.14
      - v1.21.8
      - v1.22.5
      - v1.22.10
      - v1.22.11
    default: '1.21.8'    

  userCluster:
    addons:
      dockerRepository: gcr.io/student-01-kkp-admin-training/kkp-addons
      dockerTagSuffix: "kkp-admin"
  api:
    accessibleAddons:
      - my-addon    

What cloud provider are you running on?

GCE

What operating system are you running in your user cluster?

Ubuntu 20.04

Additional information

[stroebitzer]

This does only happen if you do not have a default Storage Class applied.