ambiknai
commented
11 months ago Test with 23.7.0- stork
Closed ambiknai closed 10 months ago
ambiknai
commented
11 months ago Test with 23.7.0- stork
ambiknai
commented
10 months ago ambikanair@Ambikas-MBP demo % ibmcloud ks cluster config --admin --cluster amb-px-44-test
OK
The configuration for amb-px-44-test was downloaded successfully.
Added context for amb-px-44-test to the current kubeconfig file.
You can now execute 'kubectl' commands against your cluster. For example, run 'kubectl get nodes'.
If you are accessing the cluster for the first time, 'kubectl' commands might fail for a few seconds while RBAC synchronizes.
ambikanair@Ambikas-MBP demo % kubectl get ns
NAME STATUS AGE
default Active 26m
ibm-cert-store Active 15m
ibm-operators Active 25m
ibm-system Active 25m
kube-node-lease Active 26m
kube-public Active 26m
kube-system Active 26m
px-verify Active 11m
ambikanair@Ambikas-MBP demo % ibmcloud ks cluster addon ls -c amb-px-44-test
OK
Name Version Health State Health Status
vpc-block-csi-driver 5.0 normal Addon Ready. For more info: http://ibm.biz/addon-state (H1500)
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo % ibmcloud ks cluster addon disable vpc-block-csi-driver -c amb-px-44-test
Data and resources that you created for the add-on might be deleted when the add-on is disabled. Continue? [y/N]> y
Disabling add-on vpc-block-csi-driver for cluster amb-px-44-test...
OK
ambikanair@Ambikas-MBP demo % vi 4.4.yaml
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo % kubectl create -f 4.4.yaml
configmap/ibm-vpc-block-csi-configmap created
serviceaccount/ibm-vpc-block-node-sa created
clusterrole.rbac.authorization.k8s.io/vpc-block-driver-registrar-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-driver-registrar-binding created
clusterrole.rbac.authorization.k8s.io/vpc-block-external-resizer-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-external-resizer-binding created
serviceaccount/ibm-vpc-block-controller-sa created
clusterrole.rbac.authorization.k8s.io/vpc-block-provisioner-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-provisioner-binding created
clusterrole.rbac.authorization.k8s.io/vpc-block-external-attacher-role created
clusterrolebinding.rbac.authorization.k8s.io/vpc-block-external-attacher-binding created
daemonset.apps/ibm-vpc-block-csi-node created
statefulset.apps/ibm-vpc-block-csi-controller created
storageclass.storage.k8s.io/ibmc-vpc-block-10iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-5iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-custom created
storageclass.storage.k8s.io/ibmc-vpc-block-general-purpose created
storageclass.storage.k8s.io/ibmc-vpc-block-retain-10iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-retain-5iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-retain-custom created
storageclass.storage.k8s.io/ibmc-vpc-block-retain-general-purpose created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-10iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-5iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-custom created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-general-purpose created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-10iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-5iops-tier created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-custom created
storageclass.storage.k8s.io/ibmc-vpc-block-metro-retain-general-purpose created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-10iops created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-5iops created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-custom created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-general created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-10iops created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-5iops created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-custom created
storageclass.storage.k8s.io/ibmc-vpcblock-odf-ret-general created
Error from server (AlreadyExists): error when creating "4.4.yaml": configmaps "addon-vpc-block-csi-driver-configmap" already exists
Error from server (AlreadyExists): error when creating "4.4.yaml": csidrivers.storage.k8s.io "vpc.block.csi.ibm.io" already exists
ambikanair@Ambikas-MBP demo % kubectl get pods -n kube-system | grep block
ibm-vpc-block-csi-controller-0 6/6 Running 0 32s
ibm-vpc-block-csi-node-95r7n 4/4 Running 0 32s
ibm-vpc-block-csi-node-lgbvw 4/4 Running 0 32s
ibm-vpc-block-csi-node-mn7qx 4/4 Running 0 32s
ambikanair@Ambikas-MBP demo % ambikanair@Ambikas-MBP demo % kubectl describe deployment -n kube-system stork
Name: stork
Namespace: kube-system
CreationTimestamp: Wed, 16 Aug 2023 15:19:56 +0530
Labels: tier=control-plane
Annotations: deployment.kubernetes.io/revision: 1
scheduler.alpha.kubernetes.io/critical-pod:
Selector: name=stork
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 1 max unavailable, 1 max surge
Pod Template:
Labels: name=stork
tier=control-plane
Annotations: scheduler.alpha.kubernetes.io/critical-pod:
Service Account: stork-account
Containers:
stork:
Image: openstorage/stork:23.7.0
Port: <none>
Host Port: <none>
Command:
/stork
--verbose
--leader-elect=true
--health-monitor-interval=120
--webhook-controller=true
Requests:
cpu: 100m
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: stork-5c58cd675 (3/3 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 7m52s deployment-controller Scaled up replica set stork-5c58cd675 to 3
ambikanair@Ambikas-MBP demo % kubectl confgi current-context
error: unknown command "confgi" for "kubectl"
Did you mean this?
config
ambikanair@Ambikas-MBP demo % kubectl config current-context
amb-px-backup/cje8rbkf0e30enpcgr80/admin
ambikanair@Ambikas-MBP demo % DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 600s
node.kubernetes.io/unreachable:NoExecute op=Exists for 600s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 7m16s default-scheduler Successfully assigned kube-system/ibm-vpc-block-csi-controller-0 to 10.243.64.10
Normal Pulling 7m14s kubelet Pulling image "icr.io/ext/sig-storage/csi-provisioner:v3.2.1"
Normal Pulled 7m9s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-provisioner:v3.2.1" in 4.999618266s (4.999631898s including waiting)
Normal Created 7m9s kubelet Created container csi-provisioner
Normal Started 7m9s kubelet Started container csi-provisioner
Normal Pulling 7m9s kubelet Pulling image "icr.io/ext/sig-storage/csi-attacher:v3.5.0"
Normal Pulled 7m4s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-attacher:v3.5.0" in 5.527812241s (5.527825878s including waiting)
Normal Created 7m4s kubelet Created container csi-attacher
Normal Started 7m3s kubelet Started container csi-attacher
Normal Pulling 7m3s kubelet Pulling image "icr.io/ext/sig-storage/csi-resizer:v1.5.0"
Normal Pulled 6m56s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-resizer:v1.5.0" in 7.075681895s (7.075702549s including waiting)
Normal Created 6m56s kubelet Created container csi-resizer
Normal Started 6m56s kubelet Started container csi-resizer
Normal Pulling 6m56s kubelet Pulling image "icr.io/ext/sig-storage/livenessprobe:v2.6.0"
Normal Pulled 6m53s kubelet Successfully pulled image "icr.io/ext/sig-storage/livenessprobe:v2.6.0" in 3.364942957s (3.364954376s including waiting)
Normal Created 6m53s kubelet Created container liveness-probe
Normal Started 6m53s kubelet Started container liveness-probe
Normal Pulling 6m53s kubelet Pulling image "us.icr.io/armada-master/ibm-vpc-block-csi-driver:v4.4.17"
Normal Pulled 6m52s kubelet Successfully pulled image "us.icr.io/armada-master/ibm-vpc-block-csi-driver:v4.4.17" in 610.357333ms (610.381008ms including waiting)
Normal Created 6m52s kubelet Created container iks-vpc-block-driver
Normal Started 6m52s kubelet Started container iks-vpc-block-driver
Normal Pulling 6m52s kubelet Pulling image "icr.io/obs/armada-storage-secret:v1.2.17"
Normal Pulled 6m47s kubelet Successfully pulled image "icr.io/obs/armada-storage-secret:v1.2.17" in 4.500715732s (4.500726745s including waiting)
Normal Created 6m47s kubelet Created container storage-secret-sidecar
Normal Started 6m47s kubelet Started container storage-secret-sidecar
ambikanair@Ambikas-MBP demo %
testcustom-85fbf88b7d-dzjq2 0/1 ContainerCreating 0 30s
ambikanair@Ambikas-MBP demo % kubectl get pods
NAME READY STATUS RESTARTS AGE
testcustom-85fbf88b7d-dzjq2 0/1 ContainerCreating 0 34s
ambikanair@Ambikas-MBP demo % kubectl get pods
NAME READY STATUS RESTARTS AGE
testcustom-85fbf88b7d-dzjq2 0/1 ContainerCreating 0 36s
ambikanair@Ambikas-MBP demo % kubectl get pods
NAME READY STATUS RESTARTS AGE
testcustom-85fbf88b7d-dzjq2 0/1 ContainerCreating 0 38s
ambikanair@Ambikas-MBP demo % kubectl get pods
NAME READY STATUS RESTARTS AGE
testcustom-85fbf88b7d-dzjq2 1/1 Running 0 43s
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo % kubectl get vs
NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE
csi-block-pvc-custom-verify-20f50b7d false csi-block-pvc-custom-verify stork-csi-snapshot-class-vpc.block.csi.ibm.io snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1 4s
ambikanair@Ambikas-MBP demo % kubectl get vsc
NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT VOLUMESNAPSHOTNAMESPACE AGE
snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io csi-block-pvc-custom-verify-20f50b7d default 6s
ambikanair@Ambikas-MBP demo % kubectl get vsc
NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT VOLUMESNAPSHOTNAMESPACE AGE
snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io csi-block-pvc-custom-verify-20f50b7d default 28s
ambikanair@Ambikas-MBP demo % kubectl get vs
NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE
csi-block-pvc-custom-verify-20f50b7d false csi-block-pvc-custom-verify stork-csi-snapshot-class-vpc.block.csi.ibm.io snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1 30s
ambikanair@Ambikas-MBP demo % kubectl get pods -n kube-system | grep stork
stork-5c58cd675-9r2bd 1/1 Running 0 7m25s
stork-5c58cd675-d4txq 1/1 Running 0 7m25s
stork-5c58cd675-p95wh 1/1 Running 0 7m25s
ambikanair@Ambikas-MBP demo % kubectl get pods -n kube-system -o wide | grep stork
stork-5c58cd675-9r2bd 1/1 Running 0 7m31s 172.17.105.202 10.243.0.6 <none> <none>
stork-5c58cd675-d4txq 1/1 Running 0 7m31s 172.17.114.69 10.243.64.10 <none> <none>
stork-5c58cd675-p95wh 1/1 Running 0 7m31s 172.17.89.78 10.243.128.9 <none> <none>
ambikanair@Ambikas-MBP demo % kubectl get deployment -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
calico-kube-controllers 1/1 1 1 58m
calico-typha 3/3 3 3 58m
coredns 3/3 3 3 58m
coredns-autoscaler 1/1 1 1 58m
dashboard-metrics-scraper 1/1 1 1 47m
ingress-cluster-healthcheck 1/1 1 1 47m
kubernetes-dashboard 1/1 1 1 47m
metrics-server 2/2 2 2 47m
public-crcje8rbkf0e30enpcgr80-alb1 1/2 2 1 46m
public-crcje8rbkf0e30enpcgr80-alb2 1/2 2 1 40m
public-crcje8rbkf0e30enpcgr80-alb3 1/2 2 1 16m
snapshot-controller 3/3 3 3 58m
stork 3/3 3 3 7m40s
ambikanair@Ambikas-MBP demo % kubectl describe deployment -n kube-system stork
Name: stork
Namespace: kube-system
CreationTimestamp: Wed, 16 Aug 2023 15:19:56 +0530
Labels: tier=control-plane
Annotations: deployment.kubernetes.io/revision: 1
scheduler.alpha.kubernetes.io/critical-pod:
Selector: name=stork
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 1 max unavailable, 1 max surge
Pod Template:
Labels: name=stork
tier=control-plane
Annotations: scheduler.alpha.kubernetes.io/critical-pod:
Service Account: stork-account
Containers:
stork:
Image: openstorage/stork:23.7.0
Port: <none>
Host Port: <none>
Command:
/stork
--verbose
--leader-elect=true
--health-monitor-interval=120
--webhook-controller=true
Requests:
cpu: 100m
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: stork-5c58cd675 (3/3 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 7m52s deployment-controller Scaled up replica set stork-5c58cd675 to 3
ambikanair@Ambikas-MBP demo % kubectl confgi current-context
error: unknown command "confgi" for "kubectl"
Did you mean this?
config
ambikanair@Ambikas-MBP demo % kubectl config current-context
amb-px-backup/cje8rbkf0e30enpcgr80/admin
ambikanair@Ambikas-MBP demo % ®kubectl get
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo % kubectl get sts -n kube-system ibm-vpc-block-csi-controller -o jsonpath="{.metadata.annotations.version}{'\n'}"
(( grab $BUILD_VERSION || "dev" ))
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo % kubectl describe pod -n kube-system ibm-vpc-block-csi-controller-0
Name: ibm-vpc-block-csi-controller-0
Namespace: kube-system
Priority: 0
Service Account: ibm-vpc-block-controller-sa
Node: 10.243.64.10/10.243.64.10
Start Time: Wed, 16 Aug 2023 15:22:15 +0530
Labels: app=ibm-vpc-block-csi-driver
controller-revision-hash=ibm-vpc-block-csi-controller-7f57bd7dd5
statefulset.kubernetes.io/pod-name=ibm-vpc-block-csi-controller-0
Annotations: cni.projectcalico.org/containerID: 486875afb6dc29f3f62f437b6a862d1c4be85a1d02e1ede0e39e49a37aac373e
cni.projectcalico.org/podIP: 172.17.114.71/32
cni.projectcalico.org/podIPs: 172.17.114.71/32
Status: Running
IP: 172.17.114.71
IPs:
IP: 172.17.114.71
Controlled By: StatefulSet/ibm-vpc-block-csi-controller
Containers:
csi-provisioner:
Container ID: containerd://2484157fc6ca09e1a539942c3b4086d58c51e740c95f0df6c70ae44802a9c69c
Image: icr.io/ext/sig-storage/csi-provisioner:v3.2.1
Image ID: icr.io/ext/sig-storage/csi-provisioner@sha256:bbfd530c06400bdb5d14371fe5dac2a57db529b62feb103245fa6b806c05a000
Port: <none>
Host Port: <none>
Args:
--v=5
--csi-address=$(ADDRESS)
--timeout=600s
--feature-gates=Topology=true
State: Running
Started: Wed, 16 Aug 2023 15:22:22 +0530
Ready: True
Restart Count: 0
Limits:
cpu: 80m
memory: 160Mi
Requests:
cpu: 20m
memory: 40Mi
Environment:
ADDRESS: /csi/csi.sock
Mounts:
/csi from socket-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7hvnf (ro)
csi-attacher:
Container ID: containerd://559319bf26856146774625e0617d1601e5d456632dae334489c68e1a2e6f0004
Image: icr.io/ext/sig-storage/csi-attacher:v3.5.0
Image ID: icr.io/ext/sig-storage/csi-attacher@sha256:3a3335453e1b8d8b63fdf273e644a9473076012eba7a8e05d3d19801bf677227
Port: <none>
Host Port: <none>
Args:
--v=5
--csi-address=/csi/csi.sock
--timeout=900s
--worker-threads=15
State: Running
Started: Wed, 16 Aug 2023 15:22:28 +0530
Ready: True
Restart Count: 0
Limits:
cpu: 60m
memory: 120Mi
Requests:
cpu: 15m
memory: 30Mi
Environment: <none>
Mounts:
/csi from socket-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7hvnf (ro)
csi-resizer:
Container ID: containerd://879ec7b1164a3edf0b6bedeee8064b9bfc177b3504f9105fc5c7a9807d54b0db
Image: icr.io/ext/sig-storage/csi-resizer:v1.5.0
Image ID: icr.io/ext/sig-storage/csi-resizer@sha256:6db65e5226d4a13dd5a621ce8949c9206603b4ee906b9f4c3e096c2aa3301389
Port: <none>
Host Port: <none>
Args:
--v=5
--csi-address=/csi/csi.sock
--timeout=900s
State: Running
Started: Wed, 16 Aug 2023 15:22:35 +0530
Ready: True
Restart Count: 0
Limits:
cpu: 80m
memory: 160Mi
Requests:
cpu: 20m
memory: 40Mi
Environment: <none>
Mounts:
/csi from socket-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7hvnf (ro)
liveness-probe:
Container ID: containerd://40501346fb08b0ae7bf70463641b64315d1ebcf5a54559e889310822f71357d6
Image: icr.io/ext/sig-storage/livenessprobe:v2.6.0
Image ID: icr.io/ext/sig-storage/livenessprobe@sha256:ff67fd4ce939f74f60c88e0d843de65e2403bd49aa009fe04b328f8c1c3d071f
Port: <none>
Host Port: <none>
Args:
--csi-address=/csi/csi.sock
State: Running
Started: Wed, 16 Aug 2023 15:22:38 +0530
Ready: True
Restart Count: 0
Limits:
cpu: 20m
memory: 40Mi
Requests:
cpu: 5m
memory: 10Mi
Environment: <none>
Mounts:
/csi from socket-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7hvnf (ro)
iks-vpc-block-driver:
Container ID: containerd://4980ce55bb417b60857cd3858fbd8706daf33642dc8c500745b6f4f38d92e506
Image: us.icr.io/armada-master/ibm-vpc-block-csi-driver:v4.4.17
Image ID: us.icr.io/armada-master/ibm-vpc-block-csi-driver@sha256:9b9af82a28b9297788f514fc98b3176cc02d356afd1c30cfa9c2ac8b910ee29c
Port: 9808/TCP
Host Port: 0/TCP
Args:
--v=5
--endpoint=$(CSI_ENDPOINT)
--lock_enabled=false
--sidecarEndpoint=$(SIDECAREP)
State: Running
Started: Wed, 16 Aug 2023 15:22:39 +0530
Ready: True
Restart Count: 0
Limits:
cpu: 300m
memory: 600Mi
Requests:
cpu: 75m
memory: 150Mi
Liveness: http-get http://:healthz/healthz delay=10s timeout=3s period=10s #success=1 #failure=5
Environment Variables from:
ibm-vpc-block-csi-configmap ConfigMap Optional: false
Environment:
SIDECAREP: /csi/provider.sock
POD_NAME: ibm-vpc-block-csi-controller-0 (v1:metadata.name)
POD_NAMESPACE: kube-system (v1:metadata.namespace)
KUBE_NODE_NAME: (v1:spec.nodeName)
Mounts:
/csi from socket-dir (rw)
/etc/storage_ibmc from customer-auth (ro)
/etc/storage_ibmc/cluster_info from cluster-info (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7hvnf (ro)
storage-secret-sidecar:
Container ID: containerd://64dd7a1bf5c10b5fd71b915bf8538b8cd06314252c06cccea98d623d8beb3531
Image: icr.io/obs/armada-storage-secret:v1.2.17
Image ID: icr.io/obs/armada-storage-secret@sha256:777da3133acd4a14f1adacecf1acf1214aa38192ba832b24a962e9fbbdacac62
Port: <none>
Host Port: <none>
Args:
--endpoint=$(ENDPOINT)
State: Running
Started: Wed, 16 Aug 2023 15:22:44 +0530
Ready: True
Restart Count: 0
Limits:
cpu: 40m
memory: 80Mi
Requests:
cpu: 10m
memory: 20Mi
Environment:
ENDPOINT: unix:/sidecardir/provider.sock
TOKEN_EXPIRY_DIFF: 20m
PROFILE_CAPACITY: 1
Mounts:
/sidecardir from socket-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7hvnf (ro)
/var/run/secrets/tokens from vault-token (rw)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
vault-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 600
socket-dir:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
customer-auth:
Type: Secret (a volume populated by a Secret)
SecretName: storage-secret-store
Optional: false
cluster-info:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: cluster-info
Optional: false
kube-api-access-7hvnf:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 600s
node.kubernetes.io/unreachable:NoExecute op=Exists for 600s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 7m16s default-scheduler Successfully assigned kube-system/ibm-vpc-block-csi-controller-0 to 10.243.64.10
Normal Pulling 7m14s kubelet Pulling image "icr.io/ext/sig-storage/csi-provisioner:v3.2.1"
Normal Pulled 7m9s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-provisioner:v3.2.1" in 4.999618266s (4.999631898s including waiting)
Normal Created 7m9s kubelet Created container csi-provisioner
Normal Started 7m9s kubelet Started container csi-provisioner
Normal Pulling 7m9s kubelet Pulling image "icr.io/ext/sig-storage/csi-attacher:v3.5.0"
Normal Pulled 7m4s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-attacher:v3.5.0" in 5.527812241s (5.527825878s including waiting)
Normal Created 7m4s kubelet Created container csi-attacher
Normal Started 7m3s kubelet Started container csi-attacher
Normal Pulling 7m3s kubelet Pulling image "icr.io/ext/sig-storage/csi-resizer:v1.5.0"
Normal Pulled 6m56s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-resizer:v1.5.0" in 7.075681895s (7.075702549s including waiting)
Normal Created 6m56s kubelet Created container csi-resizer
Normal Started 6m56s kubelet Started container csi-resizer
Normal Pulling 6m56s kubelet Pulling image "icr.io/ext/sig-storage/livenessprobe:v2.6.0"
Normal Pulled 6m53s kubelet Successfully pulled image "icr.io/ext/sig-storage/livenessprobe:v2.6.0" in 3.364942957s (3.364954376s including waiting)
Normal Created 6m53s kubelet Created container liveness-probe
Normal Started 6m53s kubelet Started container liveness-probe
Normal Pulling 6m53s kubelet Pulling image "us.icr.io/armada-master/ibm-vpc-block-csi-driver:v4.4.17"
Normal Pulled 6m52s kubelet Successfully pulled image "us.icr.io/armada-master/ibm-vpc-block-csi-driver:v4.4.17" in 610.357333ms (610.381008ms including waiting)
Normal Created 6m52s kubelet Created container iks-vpc-block-driver
Normal Started 6m52s kubelet Started container iks-vpc-block-driver
Normal Pulling 6m52s kubelet Pulling image "icr.io/obs/armada-storage-secret:v1.2.17"
Normal Pulled 6m47s kubelet Successfully pulled image "icr.io/obs/armada-storage-secret:v1.2.17" in 4.500715732s (4.500726745s including waiting)
Normal Created 6m47s kubelet Created container storage-secret-sidecar
Normal Started 6m47s kubelet Started container storage-secret-sidecar
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo % kubectl get applicationbackup
NAME AGE
backup1-deb0838 3m51s
ambikanair@Ambikas-MBP demo % kubectl get applicationbackup backup1-deb0838
NAME AGE
backup1-deb0838 3m59s
ambikanair@Ambikas-MBP demo % kubectl describe applicationbackup backup1-deb0838
Name: backup1-deb0838
Namespace: default
Labels: <none>
Annotations: portworx.io/backup-name: backup1
portworx.io/backup-uid: deb08388-2ee2-4712-a2dd-ec5ef67089b6
portworx.io/backuplocation-name: cloudbackup1
portworx.io/cluster-name: amb-px-backup
portworx.io/cluster-uid: 111cba8d-9d6e-4d5d-a115-e105a026b76a
portworx.io/created-by: px-backup
portworx.io/last-update: 2023-08-16T09:56:01.749228635Z
portworx.io/object-lock-retention-period: 0
portworx.io/org-id: default
portworx.io/skip-backup-location-name-check: true
API Version: stork.libopenstorage.org/v1alpha1
Kind: ApplicationBackup
Metadata:
Creation Timestamp: 2023-08-16T09:56:01Z
Finalizers:
stork.libopenstorage.org/finalizer-cleanup
Generation: 37
Managed Fields:
API Version: stork.libopenstorage.org/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:portworx.io/backup-name:
f:portworx.io/backup-uid:
f:portworx.io/backuplocation-name:
f:portworx.io/cluster-name:
f:portworx.io/cluster-uid:
f:portworx.io/created-by:
f:portworx.io/last-update:
f:portworx.io/object-lock-retention-period:
f:portworx.io/org-id:
f:portworx.io/skip-backup-location-name-check:
f:spec:
.:
f:backupLocation:
f:backupType:
f:includeResources:
f:namespaceSelector:
f:namespaces:
f:options:
.:
f:stork.libopenstorage.org/csi-snapshot-class-name:
f:platformCredential:
f:postExecRule:
f:preExecRule:
f:rancherProjects:
f:reclaimPolicy:
f:resourceTypes:
f:selectors:
f:skipServiceUpdate:
f:status:
.:
f:backupPath:
f:finishTimestamp:
f:largeResourceEnabled:
f:resourceCount:
f:resources:
f:totalSize:
Manager: px-backup
Operation: Update
Time: 2023-08-16T09:56:01Z
API Version: stork.libopenstorage.org/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:finalizers:
.:
v:"stork.libopenstorage.org/finalizer-cleanup":
f:status:
f:lastUpdateTimestamp:
f:reason:
f:stage:
f:status:
f:triggerTimestamp:
f:volumes:
Manager: stork
Operation: Update
Time: 2023-08-16T10:00:03Z
Resource Version: 7123
UID: d50cfd15-1126-4584-83f9-b98a42523118
Spec:
Backup Location: cloudbackup1-deb0838
Backup Type: Generic
Include Resources: <nil>
Namespace Selector:
Namespaces:
default
Options:
stork.libopenstorage.org/csi-snapshot-class-name: stork-csi-snapshot-class-vpc.block.csi.ibm.io
Platform Credential:
Post Exec Rule:
Pre Exec Rule:
Rancher Projects: <nil>
Reclaim Policy: Retain
Resource Types: <nil>
Selectors: <nil>
Skip Service Update: false
Status:
Backup Path:
Finish Timestamp: <nil>
Large Resource Enabled: false
Last Update Timestamp: 2023-08-16T10:00:03Z
Reason: Volume backups are in progress
Resource Count: 0
Resources: <nil>
Stage: Volumes
Status: InProgress
Total Size: 0
Trigger Timestamp: 2023-08-16T09:56:01Z
Volumes:
Options: <nil>
Actual Size: 0
Backup ID:
Driver Name: kdmp
Namespace: default
Persistent Volume Claim: csi-block-pvc-custom-verify
Persistent Volume Claim UID: 1638f9ff-e8bd-4d7f-936f-ede07b2849bb
Provisioner: vpc.block.csi.ibm.io
Reason: Volume backup not started yet
Status:
Storage Class: ibmc-vpc-block-5iops-tier
Total Size: 0
Volume: pvc-1638f9ff-e8bd-4d7f-936f-ede07b2849bb
Volume Snapshot:
Zones: <nil>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Failed 4m6s stork error creating backup location path: BucketAlreadyExists: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.
status code: 409, request id: 8955ad83-86ad-468d-b535-c7f284e6c39d, host id:
ambikanair@Ambikas-MBP demo % kubectl describe applicationbackup backup1-deb0838
Error from server (NotFound): applicationbackups.stork.libopenstorage.org "backup1-deb0838" not found
ambikanair@Ambikas-MBP demo % kubectl get vs
NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE
backup-6b74de04f03e-ede07b2849bb false csi-block-pvc-custom-verify stork-csi-snapshot-class-vpc.block.csi.ibm.io snapcontent-1f8f825c-ba60-4d9f-aab0-5803d09ce133 77s
ambikanair@Ambikas-MBP demo % kubectl get vsc
NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT VOLUMESNAPSHOTNAMESPACE AGE
snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io csi-block-pvc-custom-verify-20f50b7d default 8m19s
snapcontent-1f8f825c-ba60-4d9f-aab0-5803d09ce133 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io backup-6b74de04f03e-ede07b2849bb default 81s
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo % kubectl get vsc
kuNAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT VOLUMESNAPSHOTNAMESPACE AGE
snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io csi-block-pvc-custom-verify-20f50b7d default 12m
snapcontent-1f8f825c-ba60-4d9f-aab0-5803d09ce133 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io backup-6b74de04f03e-ede07b2849bb default 5m49s
ambikanair@Ambikas-MBP demo % kubectl describe vsc snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1
Name: snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1
Namespace:
Labels: <none>
Annotations: snapshot.storage.kubernetes.io/volumesnapshot-being-deleted: yes
API Version: snapshot.storage.k8s.io/v1
Kind: VolumeSnapshotContent
Metadata:
Creation Timestamp: 2023-08-16T09:56:04Z
Deletion Grace Period Seconds: 0
Deletion Timestamp: 2023-08-16T10:01:11Z
Finalizers:
snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection
Generation: 2
Managed Fields:
API Version: snapshot.storage.k8s.io/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:snapshot.storage.kubernetes.io/volumesnapshot-being-deleted:
f:finalizers:
.:
v:"snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection":
f:spec:
.:
f:deletionPolicy:
f:driver:
f:source:
.:
f:volumeHandle:
f:volumeSnapshotClassName:
f:volumeSnapshotRef:
.:
f:apiVersion:
f:kind:
f:name:
f:namespace:
f:resourceVersion:
f:uid:
Manager: Go-http-client
Operation: Update
Time: 2023-08-16T10:01:11Z
Resource Version: 7285
UID: 5b8edf5e-9be9-4281-a71c-c0105105a7c3
Spec:
Deletion Policy: Retain
Driver: vpc.block.csi.ibm.io
Source:
Volume Handle: r010-2ed51af8-3f00-417f-8af4-4a327d11b83f
Volume Snapshot Class Name: stork-csi-snapshot-class-vpc.block.csi.ibm.io
Volume Snapshot Ref:
API Version: snapshot.storage.k8s.io/v1
Kind: VolumeSnapshot
Name: csi-block-pvc-custom-verify-20f50b7d
Namespace: default
Resource Version: 6563
UID: 0fe1f275-323c-4c31-bd95-1fbc6ecb83f1
Events: <none>
ambikanair@Ambikas-MBP demo % kubectl get vsc
NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT VOLUMESNAPSHOTNAMESPACE AGE
snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io csi-block-pvc-custom-verify-20f50b7d default 12m
snapcontent-1f8f825c-ba60-4d9f-aab0-5803d09ce133 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io backup-6b74de04f03e-ede07b2849bb default 6m1s
ambikanair@Ambikas-MBP demo % kubectl describe vsc snapcontent-1f8f825c-ba60-4d9f-aab0-5803d09ce133
Name: snapcontent-1f8f825c-ba60-4d9f-aab0-5803d09ce133
Namespace:
Labels: <none>
Annotations: snapshot.storage.kubernetes.io/volumesnapshot-being-deleted: yes
API Version: snapshot.storage.k8s.io/v1
Kind: VolumeSnapshotContent
Metadata:
Creation Timestamp: 2023-08-16T10:03:02Z
Deletion Grace Period Seconds: 0
Deletion Timestamp: 2023-08-16T10:08:02Z
Finalizers:
snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection
Generation: 2
Managed Fields:
API Version: snapshot.storage.k8s.io/v1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:snapshot.storage.kubernetes.io/volumesnapshot-being-deleted:
f:finalizers:
.:
v:"snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection":
f:spec:
.:
f:deletionPolicy:
f:driver:
f:source:
.:
f:volumeHandle:
f:volumeSnapshotClassName:
f:volumeSnapshotRef:
.:
f:apiVersion:
f:kind:
f:name:
f:namespace:
f:resourceVersion:
f:uid:
Manager: Go-http-client
Operation: Update
Time: 2023-08-16T10:08:02Z
Resource Version: 8407
UID: f50cdfe7-2ff7-432e-bae1-461383358aba
Spec:
Deletion Policy: Retain
Driver: vpc.block.csi.ibm.io
Source:
Volume Handle: r010-2ed51af8-3f00-417f-8af4-4a327d11b83f
Volume Snapshot Class Name: stork-csi-snapshot-class-vpc.block.csi.ibm.io
Volume Snapshot Ref:
API Version: snapshot.storage.k8s.io/v1
Kind: VolumeSnapshot
Name: backup-6b74de04f03e-ede07b2849bb
Namespace: default
Resource Version: 7549
UID: 1f8f825c-ba60-4d9f-aab0-5803d09ce133
Events: <none>
ambikanair@Ambikas-MBP demo %
ambikanair@Ambikas-MBP demo % kubectl get vsc
NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT VOLUMESNAPSHOTNAMESPACE AGE
snapcontent-0fe1f275-323c-4c31-bd95-1fbc6ecb83f1 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io csi-block-pvc-custom-verify-20f50b7d default 15m
snapcontent-1f8f825c-ba60-4d9f-aab0-5803d09ce133 Retain vpc.block.csi.ibm.io stork-csi-snapshot-class-vpc.block.csi.ibm.io backup-6b74de04f03e-ede07b2849bb default 8m59s
ambikanair@Ambikas-MBP demo % kubectl get vsc
No resources found
ambikanair@Ambikas-MBP demo % kubectl get pods -n kube-system | grep block
kuibm-vpc-block-csi-controller-0 7/7 Running 0 90s
ibm-vpc-block-csi-node-4ct9v 4/4 Running 0 90s
ibm-vpc-block-csi-node-g275w 4/4 Running 0 90s
ibm-vpc-block-csi-node-l72tf 4/4 Running 0 90s
ambikanair@Ambikas-MBP demo % kubectl describe pod -n kube-system ibm-vpc-block-csi-controller-0 | grep image
Normal Pulling 103s kubelet Pulling image "icr.io/ext/sig-storage/csi-snapshotter:v6.2.1"
Normal Pulled 102s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-snapshotter:v6.2.1" in 739.299429ms (739.323235ms including waiting)
Normal Pulling 102s kubelet Pulling image "icr.io/ext/sig-storage/csi-provisioner:v3.4.1"
Normal Pulled 101s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-provisioner:v3.4.1" in 707.508576ms (707.517646ms including waiting)
Normal Pulling 101s kubelet Pulling image "icr.io/ext/sig-storage/csi-attacher:v4.2.0"
Normal Pulled 100s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-attacher:v4.2.0" in 675.315077ms (675.32486ms including waiting)
Normal Pulling 100s kubelet Pulling image "icr.io/ext/sig-storage/csi-resizer:v1.7.0"
Normal Pulled 99s kubelet Successfully pulled image "icr.io/ext/sig-storage/csi-resizer:v1.7.0" in 740.481742ms (740.507067ms including waiting)
Normal Pulled 99s kubelet Container image "icr.io/ext/sig-storage/livenessprobe:v2.9.0" already present on machine
Normal Pulling 99s kubelet Pulling image "de.icr.io/armada-master/ibm-vpc-block-csi-driver:v5.1.12"
Normal Pulled 99s kubelet Successfully pulled image "de.icr.io/armada-master/ibm-vpc-block-csi-driver:v5.1.12" in 97.42673ms (97.435691ms including waiting)
Normal Pulling 99s kubelet Pulling image "icr.io/obs/armada-storage-secret:v1.2.25"
Normal Pulled 98s kubelet Successfully pulled image "icr.io/obs/armada-storage-secret:v1.2.25" in 718.22364ms (718.247554ms including waiting)
ambikanair@Ambikas-MBP demo %
Case ID: #CS0388632