Linodes currently can only mount 7 volumes PVCs, not 8

rsienko commented 7 months ago

General:

[x] Have you removed all sensitive information, including but not limited to access keys and passwords?
[x] Have you checked to ensure there aren't other open or closed Pull Requests for the same bug/feature/question?

Feature Requests:

[x] Have you explained your rationale for why this feature is needed?
[x] Have you offered a proposed implementation/solution?

Bug Reporting

Linodes need a boot disk, hence maximum PVCs is currently 7.

Expected Behavior

Kubernetes scheduler should not try to assign more than 7 volumes to a single Linode.

Actual Behavior

CSI node advertises that it can bind 8 volumes. This eventually results in pod failing to run when it tries to bind 8th volume on the node.

Steps to Reproduce the Problem

It's not deterministic, but the following is likely to reproduce:

Prepare 3 nodes with no bound PVCs.
Create a StatefulSet, single replica, 7 volumes. This pod should run.
Create another StatefulSet, many replicas, each single volume. Try to bind 21 to 24 volumes in total.
Eventually, a pod will be scheduled on a node trying to bind 8th volume and it will fail to run.

Environment Specifications

Screenshots, Code Blocks, and Logs

Additional Notes

For general help or discussion, join the Kubernetes Slack team channel #linode. To sign up, use the Kubernetes Slack inviter.

The Linode Community is a great place to get additional support.

codestation commented 2 months ago

Can this be reopened? This change was lost when v0.7.0 was merged and i am getting AttachVolume.Attach failed for volume "pvc-xxxxxxxxxxx" : rpc error: code = ResourceExhausted desc = max number of volumes (8) already attached to instance constantly, so my deployments always gets stuck. The nodes already has 7 volumes attached (plus a boot volume).

unn commented 2 months ago

@nesv for vis

nesv commented 2 months ago

@codestation Could you please open a new issue? That way we can track it appropriately against any potential bug fixes.

codestation commented 2 months ago

Opened #182 as requested.

nesv commented 2 months ago

Opened #182 as requested.

Thank you very much!

linode / linode-blockstorage-csi-driver