search

kubernetes / cloud-provider-openstack

Apache License 2.0
616 stars 601 forks source link

[cinder-csi-plugin] Filesystem not resized #2059

Closed cgallay closed 6 months ago

cgallay commented 1 year ago

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug

What happened: Expanding a PVC did not increase the filesystem size. All logs seem green but actual size within the pod stays unchanged.

What you expected to happen: Increase filesystem from 1Gi to 2Gi for the test pod below.

How to reproduce it: Helm chart 2.3.0

csi:
  attacher:
    resources:
      requests:
        cpu: 20m
        memory: 32M
  provisioner:
    resources:
      requests:
        cpu: 20m
        memory: 32M
  snapshotter:
    resources:
      requests:
        cpu: 20m
        memory: 32M
  resizer:
    resources:
      requests:
        cpu: 20m
        memory: 32M
  livenessprobe:
    resources:
      requests:
        cpu: 20m
        memory: 32M
  plugin:
    nodePlugin:
      tolerations: []
    controllerPlugin:
      replicas: 2
      nodeSelector:
        node-role.kubernetes.io/master: "true"
      tolerations:
        - effect: NoExecute
          key: CriticalAddonsOnly
          operator: "Exists"
secret:
  enabled: true
  create: true
  name: cinder-csi-cloud-config
  data:
    cloud.conf: |-
      [Global]
      auth-url = ${auth_url}
      application-credential-name = ${app_name}
      application-credential-id = ${app_id}
      application-credential-secret = ${app_secret}
      region = ${region}
      tenant-id = ${project_id}
      [BlockStorage]
      ignore-volume-az = true
      rescan-on-resize = true # tried without, same result
storageClass:
  enabled: false

Storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: csi-cinder-delete
provisioner: cinder.csi.openstack.org
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
reclaimPolicy: Delete
parameters:
  availability: nova

Following the example:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-pvc-cinderplugin
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: csi-cinder-delete
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
    - image: nginx
      imagePullPolicy: IfNotPresent
      name: nginx
      ports:
        - containerPort: 80
          protocol: TCP
      volumeMounts:
        - mountPath: /var/lib/www/html
          name: csi-data-cinderplugin
  volumes:
    - name: csi-data-cinderplugin
      persistentVolumeClaim:
        claimName: csi-pvc-cinderplugin
        readOnly: false

Resize: 1Gi to 2Gi

Same result before and after:

kubectl exec nginx -- df -h /var/lib/www/html
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdl        974M   24K  958M   1% /var/lib/www/html

Controller plugin:

I1219 13:06:10.782927       1 event.go:285] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"testns", Name:"csi-pvc-cinderplugin", UID:"4c8898b8-df67-48d8-835c-364624c866d2", APIVersion:"v1", ResourceVersion:"3401456", FieldPath:""}): type: 'Normal' reason: 'Resizing' External resizer is resizing volume pvc-4c8898b8-df67-48d8-835c-364624c866d2
I1219 13:06:12.321015       1 event.go:285] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"testns", Name:"csi-pvc-cinderplugin", UID:"4c8898b8-df67-48d8-835c-364624c866d2", APIVersion:"v1", ResourceVersion:"3401456", FieldPath:""}): type: 'Normal' reason: 'FileSystemResizeRequired' Require file system resize of volume on nod

Node plugin:

I1219 13:05:40.216658       1 mount_linux.go:525] Disk "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_3e3af681-a134-4ab5-bb6e-15ce8acada29" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_3e3af681-a134-4ab5-bb6e-15ce8acada29]
I1219 13:05:40.558868       1 mount_linux.go:535] Disk successfully formatted (mkfs): ext4 - /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_3e3af681-a134-4ab5-bb6e-15ce8acada29 /var/lib/kubelet/plugins/kubernetes.io/csi/cinder.csi.openstack.org/d40a20a10b17cf421e36d78fed6065c218163f498b54e2515f0e3f38b693aca5/globalmount
I1219 13:06:33.046015       1 resizefs_linux.go:71] Device /dev/sdl resized successfully

Events:

testns   17m         Normal    WaitForFirstConsumer         persistentvolumeclaim/csi-pvc-cinderplugin                                                                                     waiting for first consumer to be created before binding
testns   17m         Normal    ExternalProvisioning         persistentvolumeclaim/csi-pvc-cinderplugin                                                                                     waiting for a volume to be created, either by external provisioner "cinder.csi.openstack.org" or manually created by system administrator
testns   17m         Normal    Provisioning                 persistentvolumeclaim/csi-pvc-cinderplugin                                                                                     External provisioner is provisioning volume for claim "testns/csi-pvc-cinderplugin"
testns   17m         Normal    ProvisioningSucceeded        persistentvolumeclaim/csi-pvc-cinderplugin                                                                                     Successfully provisioned volume pvc-4c8898b8-df67-48d8-835c-364624c866d2
testns   17m         Normal    Scheduled                    pod/nginx                                                                                                                      Successfully assigned testns/nginx to expo-prd-a-1
testns   17m         Normal    SuccessfulAttachVolume       pod/nginx                                                                                                                      AttachVolume.Attach succeeded for volume "pvc-4c8898b8-df67-48d8-835c-364624c866d2"
testns   17m         Normal    Pulling                      pod/nginx                                                                                                                      Pulling image "nginx"
testns   17m         Normal    Pulled                       pod/nginx                                                                                                                      Successfully pulled image "nginx" in 4.401861383s
testns   17m         Normal    Started                      pod/nginx                                                                                                                      Started container nginx
testns   17m         Normal    Created                      pod/nginx                                                                                                                      Created container nginx
testns   17m         Normal    Resizing                     persistentvolumeclaim/csi-pvc-cinderplugin                                                                                     External resizer is resizing volume pvc-4c8898b8-df67-48d8-835c-364624c866d2
testns   17m         Warning   ExternalExpanding            persistentvolumeclaim/csi-pvc-cinderplugin                                                                                     Ignoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC.
testns   17m         Normal    FileSystemResizeRequired     persistentvolumeclaim/csi-pvc-cinderplugin                                                                                     Require file system resize of volume on node
testns   16m         Normal    FileSystemResizeSuccessful   persistentvolumeclaim/csi-pvc-cinderplugin                                                                                     MountVolume.NodeExpandVolume succeeded for volume "pvc-4c8898b8-df67-48d8-835c-364624c866d2" expo-prd-a-1
testns   16m         Normal    FileSystemResizeSuccessful   pod/nginx                                                                                                                      MountVolume.NodeExpandVolume succeeded for volume "pvc-4c8898b8-df67-48d8-835c-364624c866d2" expo-prd-a-1

kubectl:

kubectl get pvc
NAME                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS        AGE
csi-pvc-cinderplugin               Bound    pvc-4c8898b8-df67-48d8-835c-364624c866d2   2Gi        RWO            csi-cinder-delete   5m

OpenStack:

+------------------------------+-----------------------------------------------+
| Field                        | Value                                         |
+------------------------------+-----------------------------------------------+
| attachments                  | [{'id':                                       |
|                              | '3e3af681-a134-4ab5-bb6e-15ce8acada29',       |
|                              | 'attachment_id':                              |
|                              | '4213d30e-0311-4d6e-b087-aad9dbb68155',       |
|                              | 'volume_id':                                  |
|                              | '3e3af681-a134-4ab5-bb6e-15ce8acada29',       |
|                              | 'server_id':                                  |
|                              | '66c31dad-e801-458e-8b90-314f943219d5',       |
|                              | 'host_name': None, 'device': '/dev/sdl',      |
|                              | 'attached_at': '2022-12-19T13:05:38.000000'}] |
| availability_zone            | nova                                          |
| bootable                     | false                                         |
| consistencygroup_id          | None                                          |
| created_at                   | 2022-12-19T13:05:36.000000                    |
| description                  | Created by OpenStack Cinder CSI driver        |
| encrypted                    | False                                         |
| id                           | 3e3af681-a134-4ab5-bb6e-15ce8acada29          |
| multiattach                  | False                                         |
| name                         | pvc-4c8898b8-df67-48d8-835c-364624c866d2      |
| os-vol-tenant-attr:tenant_id | e388838752254d669887b10c871eae5b              |
| properties                   | cinder.csi.openstack.org/cluster='kubernetes' |
|                              | , csi.storage.k8s.io/pv/name='pvc-4c8898b8-   |
|                              | df67-48d8-835c-364624c866d2',                 |
|                              | csi.storage.k8s.io/pvc/name='csi-pvc-         |
|                              | cinderplugin',                                |
|                              | csi.storage.k8s.io/pvc/namespace='testns'     |
| replication_status           | None                                          |
| size                         | 2                                             |
| snapshot_id                  | None                                          |
| source_volid                 | None                                          |
| status                       | in-use                                        |
| type                         | CEPH_1_perf1                                  |
| updated_at                   | 2022-12-19T13:06:11.000000                    |
| user_id                      | REDACTED                                      |
+------------------------------+-----------------------------------------------+

Anything else we need to know?: I was unable to find a workaround, that would be critical to find. What other logs/debug steps can I try?

Environment:

jichenjc commented 1 year ago

/dev/sdl 974M 24K 958M 1% /var/lib/www/html

I encountered this before and I switched to another openstack it works fine so I think it might be related to the cinder storage backend , from CPO CSI perspective it has done the job ..

zifeo commented 1 year ago

@jichenjc what else could be done here to debug?

jichenjc commented 1 year ago

https://github.com/kubernetes/cloud-provider-openstack/issues/1437 this is the history, I think maybe we can check cinder logs

I remember I have a company openstack with it the resize failed, then I created a devstack and it works fine maybe need check cinder logs to see anything related to resize volume functions

zifeo commented 1 year ago

@jichenjc setting log info to 5 gave this with rescan:

openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.456381       1 utils.go:91] [ID:333] GRPC call: /csi.v1.Node/NodeGetCapabilities
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.456396       1 utils.go:92] [ID:333] GRPC request: {}
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.456431       1 nodeserver.go:481] NodeGetCapabilities called with req: &csi.NodeGetCapabilitiesRequest{XXX_NoUnkeyedLiteral:struct {}{}, XXX_unrecognized:[]uint8(nil), XXX_sizecache:0}
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.456440       1 utils.go:97] [ID:333] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":2}}}]}
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.456996       1 utils.go:91] [ID:334] GRPC call: /csi.v1.Node/NodeGetCapabilities
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.457006       1 utils.go:92] [ID:334] GRPC request: {}
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.457039       1 nodeserver.go:481] NodeGetCapabilities called with req: &csi.NodeGetCapabilitiesRequest{XXX_NoUnkeyedLiteral:struct {}{}, XXX_unrecognized:[]uint8(nil), XXX_sizecache:0}
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.457049       1 utils.go:97] [ID:334] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":2}}}]}
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.458077       1 utils.go:91] [ID:335] GRPC call: /csi.v1.Node/NodeExpandVolume
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.458085       1 utils.go:92] [ID:335] GRPC request: {"capacity_range":{"required_bytes":2147483648},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/cinder.csi.openstack.org/fc6548b4538f9ebf615ca2947daf73ded3977fa9575832e0d80fb48f772e9eef/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_id":"d4dbf429-c3a5-4c03-b1aa-71741b6399e1","volume_path":"/var/lib/kubelet/pods/16a9e8ec-c374-4415-bf2a-0d10fb6850d4/volumes/kubernetes.io~csi/pvc-a68e8ee6-68ac-43b5-9a44-e98f9b38ba01/mount"}
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.458147       1 nodeserver.go:533] NodeExpandVolume: called with args {"capacity_range":{"required_bytes":2147483648},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/cinder.csi.openstack.org/fc6548b4538f9ebf615ca2947daf73ded3977fa9575832e0d80fb48f772e9eef/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_id":"d4dbf429-c3a5-4c03-b1aa-71741b6399e1","volume_path":"/var/lib/kubelet/pods/16a9e8ec-c374-4415-bf2a-0d10fb6850d4/volumes/kubernetes.io~csi/pvc-a68e8ee6-68ac-43b5-9a44-e98f9b38ba01/mount"}
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.528023       1 blockdevice_linux.go:73] Detecting "/var/lib/kubelet/pods/16a9e8ec-c374-4415-bf2a-0d10fb6850d4/volumes/kubernetes.io~csi/pvc-a68e8ee6-68ac-43b5-9a44-e98f9b38ba01/mount" volume size
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.528059       1 blockdevice_linux.go:79] Detected "/var/lib/kubelet/pods/16a9e8ec-c374-4415-bf2a-0d10fb6850d4/volumes/kubernetes.io~csi/pvc-a68e8ee6-68ac-43b5-9a44-e98f9b38ba01/mount" volume size: 1073741824
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.528143       1 blockdevice_linux.go:108] Resolved block device path from "/dev/sde" to "/sys/devices/pci0000:00/0000:00:04.0/virtio1/host2/target2:0:0/2:0:0:4/rescan"
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.528152       1 blockdevice_linux.go:109] Rescanning "/dev/sde" block device geometry
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.547180       1 blockdevice_linux.go:73] Detecting "/var/lib/kubelet/pods/16a9e8ec-c374-4415-bf2a-0d10fb6850d4/volumes/kubernetes.io~csi/pvc-a68e8ee6-68ac-43b5-9a44-e98f9b38ba01/mount" volume size
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin I1221 02:28:11.547283       1 blockdevice_linux.go:79] Detected "/var/lib/kubelet/pods/16a9e8ec-c374-4415-bf2a-0d10fb6850d4/volumes/kubernetes.io~csi/pvc-a68e8ee6-68ac-43b5-9a44-e98f9b38ba01/mount" volume size: 1073741824
openstack-cinder-csi-nodeplugin-lxl8f cinder-csi-plugin E1221 02:28:11.547297       1 utils.go:95] [ID:335] GRPC error: rpc error: code = Internal desc = Could not verify "d4dbf429-c3a5-4c03-b1aa-71741b6399e1" volume size: current volume size is less than expected one: 1073741824 < 2147483648

And without rescan:

openstack-cinder-csi-nodeplugin-g7tj4 cinder-csi-plugin I1221 02:42:43.407532       1 mount_linux.go:563] Attempting to determine if disk "/dev/sdc" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/sdc])
openstack-cinder-csi-nodeplugin-g7tj4 cinder-csi-plugin I1221 02:42:43.408921       1 mount_linux.go:566] Output: "DEVNAME=/dev/sdc\nTYPE=ext4\n"
openstack-cinder-csi-nodeplugin-g7tj4 cinder-csi-plugin I1221 02:42:43.408943       1 resizefs_linux.go:56] ResizeFS.Resize - Expanding mounted volume /dev/sdc
openstack-cinder-csi-nodeplugin-g7tj4 cinder-csi-plugin I1221 02:42:43.427309       1 resizefs_linux.go:71] Device /dev/sdc resized successfully

Is there other logs somewhere? Maybe in the OS? Is there a know workaround?

jichenjc commented 1 year ago

seems the volume does NOT extend due to some reason I would first check whether I can extend my volume at openstack level (not using CAPO at all)

e.g try this command on your volume directly and see whether it's report like error_extending and check the volume is really resized in file system ,as CPO rely on cinder if above doesn't work, check cinder log like cinder-volume.log in openstack and it should give us more info

# cinder extend
usage: cinder extend <volume> <new_size>
error: the following arguments are required: <volume>, <new_size>
Try 'cinder help extend' for more information.
jichenjc commented 1 year ago

I tried devstack and it works fine , it takes around 2-3 min to become 2G in disk 

root@jitest43:~# cinder list
+--------------------------------------+--------+------------------------------------------+------+-------------+----------+--------------------------------------+
| ID                                   | Status | Name                                     | Size | Volume Type | Bootable | Attached to                          |
+--------------------------------------+--------+------------------------------------------+------+-------------+----------+--------------------------------------+
| 93cc2adb-f5f8-415b-b561-e2ba8dc44659 | in-use | pvc-abd42a61-c8f8-42aa-84ed-53c5cef39660 | 2    | lvmdriver-1 | false    | 07b0a955-a979-47cd-8aea-c19fe491e319 |
+--------------------------------------+--------+------------------------------------------+------+-------------+----------+--------------------------------------+
root@jitest43:~# kubectl exec nginx -- df -h /var/lib/www/html
Filesystem      Size  Used Avail Use% Mounted on
/dev/vdb        974M   24K  958M   1% /var/lib/www/html
root@jitest43:~# kubectl exec nginx -- df -h /var/lib/www/html
Filesystem      Size  Used Avail Use% Mounted on
/dev/vdb        2.0G   24K  2.0G   1% /var/lib/www/html
zifeo commented 1 year ago

@jichenjc thanks for checking and sorry for the delayed answer. I was verifying with our OpenStack provider and it took some time to solve the issue over the Christmas break. They changed some settings (not aware of which) and the issue was remediated today. For future reference, users check under the volume "messages" tab where the OpenStack show if there is any potential internal error:

PastedGraphic-1
jichenjc commented 1 year ago

ok, I use CLI mostly ,in such case, guess cinder volume show should help us in getting same info to the UI anyway, I Think we are good now so close this issue, feel free to reopen

kpauljoseph commented 1 year ago

@zifeo Hi, I'm facing the exact same issue in my setup as well. But couldn't find any messages for the specific volume that's causing an issue. Would you be able to tell me what sort of settings need to be changed in OpenStack to rectify this?

zifeo commented 1 year ago

@kpauljoseph I am sadly only an OpenStack user, not an admin and thus have no idea how the provider solved that issue.

kpauljoseph commented 1 year ago

Hi @zifeo, I was trying to track the openstack configs required for this and correlated the outputs of $ openstack volume show that I could gather from all the similar issues for this resize error and found a set of enabled configs: | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None |

Could you please run the above command on any of your pvc linked volumes on the setup that the openstack admins corrected to resolve the issue and let me know if you're able to see any of the lines I mentioned above?

I had a look at your output from earlier on in this thread and noticed these fields weren't present and I assume that output was from before it got resolved.

Would be of great help. Thanks.

zifeo commented 1 year ago

@kpauljoseph Sure, however running the command on our volumes does not output those attributes, we only have os-vol-tenant-attr:tenant_id set, no mig-status at all.

jichenjc commented 1 year ago

reopen this for further discussion

seanschneeweiss commented 1 year ago

This is probably related to Cinder Bug https://bugs.launchpad.net/charm-cinder/+bug/1939389/comments/35

This should be limited to live/online resizing a disk.

From comment #35.

This is because if the [nova] section in the cinder.conf is missing, the api request to the nova-api from the cinder is executed as the user who initiated the cinder cli call who does not have privileges to execute 'volume-extend'.

Sean Schneeweiss sean.schneeweiss@mercedes-benz.com, Mercedes-Benz Tech Innovation GmbH, Provider Information

jichenjc commented 1 year ago

thanks for sharing, that's charm which should be some openstack package method so not all openstack distro has this problem but worthy check of everyone report such problem due to code in cinder like this :

it expects NOVA_GROUP provide some input so it's eligible to send API request to nova side to avoid the 403 issue 

    if privileged_user and CONF[NOVA_GROUP].auth_type:
        LOG.debug('Creating Keystone auth plugin from conf')
        n_auth = ks_loading.load_auth_from_conf_options(CONF, NOVA_GROUP)  
    else:
        if CONF[NOVA_GROUP].token_auth_url:
            url = CONF[NOVA_GROUP].token_auth_url
        else:
            url = _get_identity_endpoint_from_sc(context)
        LOG.debug('Creating Keystone token plugin using URL: %s', url)
        n_auth = identity.Token(auth_url=url,
                                token=context.auth_token,
                                project_name=context.project_name,
                                project_domain_id=context.project_domain_id)

    if CONF.auth_strategy == 'keystone':
        n_auth = service_auth.get_auth_plugin(context, auth=n_auth)

    keystone_session = ks_loading.load_session_from_conf_options(
        CONF,
        NOVA_GROUP,
        auth=n_auth)

    c = nova_client.Client(
        api_versions.APIVersion(api_version or NOVA_API_VERSION),
        session=keystone_session,
        insecure=CONF[NOVA_GROUP].insecure,
        timeout=timeout,
        region_name=CONF[NOVA_GROUP].region_name,
        endpoint_type=CONF[NOVA_GROUP].interface,
        cacert=CONF[NOVA_GROUP].cafile,
        global_request_id=context.global_id,
        extensions=nova_extensions)
k8s-triage-robot commented 8 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 7 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 6 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 6 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes/cloud-provider-openstack/issues/2059#issuecomment-2005647792): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.