Count volumes attached to a node

[thommeo]

What would you like to be added:

1. A gauge metric reflecting number of the persistent volumes attached to a node.

Why is this needed:

The number of volumes that can be attached to a Node is limited.

• Kubernetes general limitations
• GCP - 127 persistent disks per node
• DigitalOcean - 15 volumes per node
• Linode - 8 (apparently that is even less because some system disks are also counted towards that limit)

Event with higher limits e.g. with Google we can still reach them. So monitoring the number of volumes attached per node and number of volumes that can be attached per node is essential to scale out when it's required.

Describe the solution you'd like

Here is how I tried to tackle this, but apparently it is not possible and will not be possible to create custom resource states on the core objects. So this is NOT working:

        kind: CustomResourceStateMetrics
        spec:
          resources:
            - groupVersionKind:
                group: ""
                kind: Node
                version: v1
              labelsFromPath:
                node: [metadata, name]
              metrics:
                - name: node_volumes_attached_count
                  help: Number of volumes attached to the node
                  type: Gauge
                  gauge:
                    path: [status, volumesAttached]
                    valueFrom:
                      count: {}
                - name: node_volumes_in_use_count
                  help: Number of volumes in use on the node
                  type: Gauge
                  gauge:
                    path: [status, volumesInUse]
                    valueFrom:
                      count: {}

I am not sure if this max volume limit is exposed to Kubernetes by the vendors in any way, but we definitely can count the attached volumes. Knowing the given provider limitations, we can come up with the alerting strategy based on node labels and the count of the volumes that are currently attached.

[dgrisonnet]

/triage accepted /assign