search

vmware-tanzu / community-edition

VMware Tanzu Community Edition is no longer an actively maintained project. Code is available for historical purposes only.
https://tanzucommunityedition.io/
Apache License 2.0
1.34k stars 308 forks source link

Cluster create failed but cluster created #3907

Closed mohitsharma-in closed 2 years ago

mohitsharma-in commented 2 years ago

Bug Report

Expected Behavior

Tanzu Cli create cluste cluste threw an error for validation.awscluster.infrastructure.cluster.x-k8s.io but at the same time in the backend cluster got created with required resources. 

tanzu cluster create mohit-wrkld -f ~/wkr.yaml
Validating configuration...
Warning: Pinniped configuration not found. Skipping pinniped configuration in workload cluster. Please refer to the documentation to check if you can configure pinniped on workload cluster manually
unable to override node size
Creating workload cluster 'mohit-wrkld'...
Error: unable to create cluster: unable to apply cluster configuration: kubectl apply failed, output: cluster.cluster.x-k8s.io/mohit-wrkld unchanged
kubeadmcontrolplane.controlplane.cluster.x-k8s.io/mohit-wrkld-control-plane unchanged
awsmachinetemplate.infrastructure.cluster.x-k8s.io/mohit-wrkld-md-0 unchanged
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/mohit-wrkld-md-0 unchanged
machinedeployment.cluster.x-k8s.io/mohit-wrkld-md-0 unchanged
machinedeployment.cluster.x-k8s.io/mohit-wrkld-md-1 unchanged
machinedeployment.cluster.x-k8s.io/mohit-wrkld-md-2 unchanged
awsmachinetemplate.infrastructure.cluster.x-k8s.io/mohit-wrkld-md-1 unchanged
awsmachinetemplate.infrastructure.cluster.x-k8s.io/mohit-wrkld-md-2 unchanged
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/mohit-wrkld-md-1 unchanged
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/mohit-wrkld-md-2 unchanged
secret/mohit-wrkld-antrea-addon configured
secret/mohit-wrkld-kapp-controller-addon configured
secret/mohit-wrkld-tkg-metadata-namespace-role configured
secret/mohit-wrkld-tkg-metadata-configmap configured
secret/mohit-wrkld-tkg-metadata-bom-configmap configured
clusterresourceset.addons.cluster.x-k8s.io/mohit-wrkld-tkg-metadata unchanged
secret/mohit-wrkld-metrics-server-addon configured
clusterresourceset.addons.cluster.x-k8s.io/mohit-wrkld-core-package-repository unchanged
secret/mohit-wrkld-core-package-repository-crs configured
secret/mohit-wrkld-secretgen-controller-addon configured
machinehealthcheck.cluster.x-k8s.io/mohit-wrkld configured
machinehealthcheck.cluster.x-k8s.io/mohit-wrkld-control-plane configured
clusterresourceset.addons.cluster.x-k8s.io/mohit-wrkld-default-storage-class unchanged
secret/mohit-wrkld-default-storage-class configured
clusterresourceset.addons.cluster.x-k8s.io/mohit-wrkld-capabilities unchanged
secret/mohit-wrkld-capabilities configured
secret/mohit-wrkld-config-values configured
Error from server (Invalid): error when applying patch:
{"spec":{"network":{"subnets":[{"availabilityZone":"us-east-1a","cidrBlock":"10.0.0.0/24","id":""},{"availabilityZone":"us-east-1a","cidrBlock":"10.0.1.0/24","id":"","isPublic":true},{"availabilityZone":"us-east-1b","cidrBlock":"10.0.3.0/24","id":"","isPublic":true},{"availabilityZone":"us-east-1b","cidrBlock":"10.0.2.0/24","id":""},{"availabilityZone":"us-east-1c","cidrBlock":"10.0.5.0/24","id":"","isPublic":true},{"availabilityZone":"us-east-1c","cidrBlock":"10.0.4.0/24","id":""}],"vpc":{"id":""}}}}
to:
Resource: "infrastructure.cluster.x-k8s.io/v1beta1, Resource=awsclusters", GroupVersionKind: "infrastructure.cluster.x-k8s.io/v1beta1, Kind=AWSCluster"
Name: "mohit-wrkld", Namespace: "default"
for: "/tmp/kubeapply-2582333891": admission webhook "validation.awscluster.infrastructure.cluster.x-k8s.io" denied the request: AWSCluster.infrastructure.cluster.x-k8s.io "mohit-wrkld" is invalid: spec.network.vpc.id: Invalid value: v1beta1.AWSIdentityReference{Name:"default", Kind:"AWSClusterControllerIdentity"}: field cannot be modified once set
Error from server (BadRequest): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta1\",\"kind\":\"AWSMachineTemplate\",\"metadata\":{\"annotations\":{},\"name\":\"mohit-wrkld-control-plane\",\"namespace\":\"default\"},\"spec\":{\"template\":{\"spec\":{\"ami\":{\"id\":\"ami-0a3064b99b7f3410f\"},\"iamInstanceProfile\":\"control-plane.tkg.cloud.vmware.com\",\"instanceType\":\"t3.small\",\"rootVolume\":{\"size\":80},\"sshKeyName\":\"mykp\"}}}}\n"}},"spec":{"template":{"spec":{"instanceType":"t3.small"}}}}
to:
Resource: "infrastructure.cluster.x-k8s.io/v1beta1, Resource=awsmachinetemplates", GroupVersionKind: "infrastructure.cluster.x-k8s.io/v1beta1, Kind=AWSMachineTemplate"
Name: "mohit-wrkld-control-plane", Namespace: "default"
for: "/tmp/kubeapply-2582333891": admission webhook "validation.awsmachinetemplate.infrastructure.x-k8s.io" denied the request: AWSMachineTemplate.Spec is immutable
: exit status 1
Usage:
  tanzu cluster create CLUSTER_NAME [flags]

Flags:
  -d, --dry-run       Does not create cluster, but show the deployment YAML instead
  -f, --file string   Configuration file from which to create a cluster
  -h, --help          help for create
      --tkr string    TanzuKubernetesRelease(TKr) to be used for creating the workload cluster. If TKr name prefix is provided, the latest compatible TKr matching the TKr name prefix would be used

Global Flags:
      --log-file string   Log file path
  -v, --verbose int32     Number for the log level verbosity(0-9)

Error: exit status 1

✖  exit status 1```

### Output for tanzu cluster get clustername during the creating process 

```ubuntu@ip-172-31-83-191:~$ tanzu cluster get  mohit-wrkld
  NAME         NAMESPACE  STATUS    CONTROLPLANE  WORKERS  KUBERNETES        ROLES
  mohit-wrkld  default    creating  0/3           0/3      v1.22.5+vmware.1  <none>
ℹ

Details:

NAME                                                            READY  SEVERITY  REASON                           SINCE  MESSAGE
/mohit-wrkld                                                    False  Info      WaitingForControlPlane           6s
├─ClusterInfrastructure - AWSCluster/mohit-wrkld                False  Info      BastionCreationStarted           6s     6 of 8 completed
├─ControlPlane - KubeadmControlPlane/mohit-wrkld-control-plane
└─Workers
  ├─MachineDeployment/mohit-wrkld-md-0                          False  Warning   WaitingForAvailableMachines      2m24s  Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-0-5d4557f6bc-7plrn                 False  Info      WaitingForClusterInfrastructure  2m24s  0 of 2 completed
  ├─MachineDeployment/mohit-wrkld-md-1                          False  Warning   WaitingForAvailableMachines      2m24s  Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-1-67967d69c7-fddrl                 False  Info      WaitingForClusterInfrastructure  2m24s  0 of 2 completed
  └─MachineDeployment/mohit-wrkld-md-2                          False  Warning   WaitingForAvailableMachines      2m24s  Minimum availability requires 1 replicas, current 0 available
    └─Machine/mohit-wrkld-md-2-fcbd6bdcf-62jkb                  False  Info      WaitingForClusterInfrastructure  2m24s  0 of 2 completed
ubuntu@ip-172-31-83-191:~$ tanzu cluster get  mohit-wrkld
  NAME         NAMESPACE  STATUS    CONTROLPLANE  WORKERS  KUBERNETES        ROLES
  mohit-wrkld  default    creating  0/3           0/3      v1.22.5+vmware.1  <none>
ℹ

Details:

NAME                                                            READY  SEVERITY  REASON                           SINCE  MESSAGE
/mohit-wrkld                                                    False  Info      WaitingForControlPlane           72s
├─ClusterInfrastructure - AWSCluster/mohit-wrkld                False  Info      WaitForDNSNameResolve            39s    7 of 8 completed
├─ControlPlane - KubeadmControlPlane/mohit-wrkld-control-plane
└─Workers
  ├─MachineDeployment/mohit-wrkld-md-0                          False  Warning   WaitingForAvailableMachines      3m30s  Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-0-5d4557f6bc-7plrn                 False  Info      WaitingForClusterInfrastructure  3m30s  0 of 2 completed
  ├─MachineDeployment/mohit-wrkld-md-1                          False  Warning   WaitingForAvailableMachines      3m30s  Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-1-67967d69c7-fddrl                 False  Info      WaitingForClusterInfrastructure  3m30s  0 of 2 completed
  └─MachineDeployment/mohit-wrkld-md-2                          False  Warning   WaitingForAvailableMachines      3m30s  Minimum availability requires 1 replicas, current 0 available
    └─Machine/mohit-wrkld-md-2-fcbd6bdcf-62jkb                  False  Info      WaitingForClusterInfrastructure  3m30s  0 of 2 completed
ubuntu@ip-172-31-83-191:~$ tanzu cluster get  mohit-wrkld
  NAME         NAMESPACE  STATUS    CONTROLPLANE  WORKERS  KUBERNETES        ROLES
  mohit-wrkld  default    creating  0/3           0/3      v1.22.5+vmware.1  <none>
ℹ

Details:

NAME                                                            READY  SEVERITY  REASON                           SINCE  MESSAGE
/mohit-wrkld                                                    False  Info      WaitingForControlPlane           2m1s
├─ClusterInfrastructure - AWSCluster/mohit-wrkld                False  Info      WaitForDNSNameResolve            88s    7 of 8 completed
├─ControlPlane - KubeadmControlPlane/mohit-wrkld-control-plane
└─Workers
  ├─MachineDeployment/mohit-wrkld-md-0                          False  Warning   WaitingForAvailableMachines      4m19s  Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-0-5d4557f6bc-7plrn                 False  Info      WaitingForClusterInfrastructure  4m19s  0 of 2 completed
  ├─MachineDeployment/mohit-wrkld-md-1                          False  Warning   WaitingForAvailableMachines      4m19s  Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-1-67967d69c7-fddrl                 False  Info      WaitingForClusterInfrastructure  4m19s  0 of 2 completed
  └─MachineDeployment/mohit-wrkld-md-2                          False  Warning   WaitingForAvailableMachines      4m19s  Minimum availability requires 1 replicas, current 0 available
    └─Machine/mohit-wrkld-md-2-fcbd6bdcf-62jkb                  False  Info      WaitingForClusterInfrastructure  4m19s  0 of 2 completed
ubuntu@ip-172-31-83-191:~$ tanzu cluster get  mohit-wrkld
  NAME         NAMESPACE  STATUS    CONTROLPLANE  WORKERS  KUBERNETES        ROLES
  mohit-wrkld  default    creating  0/3           0/3      v1.22.5+vmware.1  <none>
ℹ

Details:

NAME                                                            READY  SEVERITY  REASON                           SINCE  MESSAGE
/mohit-wrkld                                                    False  Warning   ScalingUp                        5s     Scaling up control plane to 3 replicas (actual 1)
├─ClusterInfrastructure - AWSCluster/mohit-wrkld                True                                              27s
├─ControlPlane - KubeadmControlPlane/mohit-wrkld-control-plane  False  Warning   ScalingUp                        5s     Scaling up control plane to 3 replicas (actual 1)
│ └─Machine/mohit-wrkld-control-plane-sg5r5                     False  Info      InstanceProvisionStarted         25s    1 of 2 completed
└─Workers
  ├─MachineDeployment/mohit-wrkld-md-0                          False  Warning   WaitingForAvailableMachines      5m26s  Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-0-5d4557f6bc-7plrn                 False  Info      WaitingForBootstrapData          26s    0 of 2 completed
  │   └─BootstrapConfig - KubeadmConfig/mohit-wrkld-md-0-n68sh  False  Info      WaitingForControlPlaneAvailable  26s
  ├─MachineDeployment/mohit-wrkld-md-1                          False  Warning   WaitingForAvailableMachines      5m26s  Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-1-67967d69c7-fddrl                 False  Info      WaitingForBootstrapData          26s    0 of 2 completed
  │   └─BootstrapConfig - KubeadmConfig/mohit-wrkld-md-1-r6c84  False  Info      WaitingForControlPlaneAvailable  26s
  └─MachineDeployment/mohit-wrkld-md-2                          False  Warning   WaitingForAvailableMachines      5m26s  Minimum availability requires 1 replicas, current 0 available
    └─Machine/mohit-wrkld-md-2-fcbd6bdcf-62jkb                  False  Info      WaitingForBootstrapData          26s    0 of 2 completed
      └─BootstrapConfig - KubeadmConfig/mohit-wrkld-md-2-nw9sj  False  Info      WaitingForControlPlaneAvailable  26s
ubuntu@ip-172-31-83-191:~$ tanzu cluster get  mohit-wrkld
  NAME         NAMESPACE  STATUS    CONTROLPLANE  WORKERS  KUBERNETES        ROLES
  mohit-wrkld  default    updating  1/3           1/3      v1.22.5+vmware.1  <none>
ℹ

Details:

NAME                                                            READY  SEVERITY  REASON                       SINCE  MESSAGE
/mohit-wrkld                                                    False  Warning   ScalingUp                    118s   Scaling up control plane to 3 replicas (actual 2)
├─ClusterInfrastructure - AWSCluster/mohit-wrkld                True                                          5m4s
├─ControlPlane - KubeadmControlPlane/mohit-wrkld-control-plane  False  Warning   ScalingUp                    118s   Scaling up control plane to 3 replicas (actual 2)
│ └─2 Machines...                                               True                                          4m28s  See mohit-wrkld-control-plane-sg5r5, mohit-wrkld-control-plane-zlv9g
└─Workers
  ├─MachineDeployment/mohit-wrkld-md-0                          True                                          40s
  │ └─Machine/mohit-wrkld-md-0-5d4557f6bc-7plrn                 True                                          105s
  ├─MachineDeployment/mohit-wrkld-md-1                          False  Warning   WaitingForAvailableMachines  10m    Minimum availability requires 1 replicas, current 0 available
  │ └─Machine/mohit-wrkld-md-1-67967d69c7-fddrl                 True                                          105s
  └─MachineDeployment/mohit-wrkld-md-2                          False  Warning   WaitingForAvailableMachines  10m    Minimum availability requires 1 replicas, current 0 available
    └─Machine/mohit-wrkld-md-2-fcbd6bdcf-62jkb                  True                                          105s
ubuntu@ip-172-31-83-191:~$ tanzu cluster get  mohit-wrkld^C
ubuntu@ip-172-31-83-191:~$ tanzu cluster get  mohit-wrkld
  NAME         NAMESPACE  STATUS    CONTROLPLANE  WORKERS  KUBERNETES        ROLES
  mohit-wrkld  default    updating  2/3           3/3      v1.22.5+vmware.1  <none>
ℹ

Details:

NAME                                                            READY  SEVERITY  REASON  SINCE  MESSAGE
/mohit-wrkld                                                    True                     26s
├─ClusterInfrastructure - AWSCluster/mohit-wrkld                True                     6m23s
├─ControlPlane - KubeadmControlPlane/mohit-wrkld-control-plane  True                     26s
│ └─3 Machines...                                               True                     5m47s  See mohit-wrkld-control-plane-526t7, mohit-wrkld-control-plane-sg5r5, ...
└─Workers
  ├─MachineDeployment/mohit-wrkld-md-0                          True                     119s
  │ └─Machine/mohit-wrkld-md-0-5d4557f6bc-7plrn                 True                     3m4s
  ├─MachineDeployment/mohit-wrkld-md-1                          True                     66s
  │ └─Machine/mohit-wrkld-md-1-67967d69c7-fddrl                 True                     3m4s
  └─MachineDeployment/mohit-wrkld-md-2                          True                     66s
    └─Machine/mohit-wrkld-md-2-fcbd6bdcf-62jkb                  True                     3m4s

Environment Details

Diagnostics and log bundle

management-cluster.mohitpoc.diagnostics.tar.gz

github-actions[bot] commented 2 years ago

Hey @mohit94614! Thanks for opening your first issue. We appreciate your contribution and welcome you to our community! We are glad to have you here and to have your input on Tanzu Community Edition.

github-actions[bot] commented 2 years ago

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.