search

Azure / kubernetes-volume-drivers

Kubernetes volume drivers for Azure
MIT License
127 stars 54 forks source link

Disk IO metrics from disks created using the local persistent volume on L series VM #141

Open jwendl opened 4 months ago

jwendl commented 4 months ago

Is your feature request related to a problem? Please describe. Getting /dev/nvme0n1 /dev/nvme1n1 metrics from a pvc created using the local volume mount https://github.com/Azure/kubernetes-volume-drivers/tree/master/local

Describe the solution you'd like in detail Having disk io metrics show up using application insights / azure monitor

Describe alternatives you've considered Creating our own addon or doing disk metrics from our application

Additional context Helps us figure out our application issues on a single node since our application uses a lot of disk to cache large binary files

andyzhangx commented 4 months ago

@jwendl

unfortunately disk pv iops metrics is not collected in k8s now, details: https://github.com/kubernetes/kubernetes/issues/92287, it requires containerd support at least. instead, I think you could use tool like iostat to collect io metrics inside your applications, .e.g.

how to collect disk io metrics on a disk using linux command?

CloudCopilot You can use the iostat command to collect disk I/O metrics on a Linux system. The iostat command is part of the sysstat package, which is typically installed by default on most Linux distributions.

Here's an example of how to use the iostat command to collect disk I/O metrics:

Open a terminal window on your Linux system.

Run the following command to install the sysstat package if it is not already installed:

sudo apt-get install sysstat This command is for Ubuntu and Debian-based systems. If you are using a different Linux distribution, the package manager and command may be different.

Run the following command to collect disk I/O metrics for a specific disk:

iostat -d -k /dev/sda This command collects disk I/O metrics for the /dev/sda disk in kilobytes per second. You can replace /dev/sda with the path to the disk that you want to monitor.

The iostat command will display the following metrics:

tps: the number of I/O transactions per second. kB_read/s: the number of kilobytes read from the disk per second. kB_wrtn/s: the number of kilobytes written to the disk per second. kB_dscd/s: the number of kilobytes discarded from the disk per second. kB_read: the total number of kilobytes read from the disk. kB_wrtn: the total number of kilobytes written to the disk. The metrics are updated every second by default. You can use the -t option to specify a different interval in seconds.

I hope this helps! Let me know if you have any other questions.