kubernetes-sigs / azuredisk-csi-driver

Azure Disk CSI Driver
Apache License 2.0
147 stars 192 forks source link

feat: support NVME disk on Linux node #2429

Closed andyzhangx closed 4 months ago

andyzhangx commented 4 months ago

What type of PR is this? /kind feature

What this PR does / why we need it: feat: support NVME disk

Which issue(s) this PR fixes:

Fixes #

Requirements:

Special notes for your reviewer:

``` I0726 12:16:23.959590 1 utils.go:106] GRPC request: {"publish_context":{"LUN":"2"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/feff39288781b8a9437126cfecfe10952213f446af81ade220093cdd382ae7aa/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1721990465719-962-disk.csi.azure.com"},"volume_id":"/subscriptions/xxx/resourceGroups/shoot--hc-dev--d029266amd-haas/providers/Microsoft.Compute/disks/pv-shoot--hc-dev--d029266amd-haas-117eba57-0191-401e-87be-cb43ad9acb78"} I0726 12:16:25.424872 1 azure_common_linux.go:182] model doesn't match VHD, got Virtual DVD-ROM W0726 12:16:25.424891 1 azure_common_linux.go:110] failed to find disk by lun 2, err , fall back to search in following device path: [/dev/disk/azure/scsi1/lun2 /dev/disk/azure/data/by-lun/2] I0726 12:16:25.424914 1 azure_common_linux.go:114] found device path /dev/disk/azure/data/by-lun/2 linked to ../../../../nvme0n4 by lun 2 I0726 12:16:25.425219 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/data/by-lun/2 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/feff39288781b8a9437126cfecfe10952213f446af81ade220093cdd382ae7aa/globalmount with mount options([]) I0726 12:16:25.425230 1 mount_linux.go:577] Attempting to determine if disk "/dev/disk/azure/data/by-lun/2" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/data/by-lun/2]) I0726 12:16:25.428812 1 mount_linux.go:580] Output: "" I0726 12:16:25.428833 1 mount_linux.go:515] Disk "/dev/disk/azure/data/by-lun/2" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/azure/data/by-lun/2] I0726 12:16:25.991547 1 mount_linux.go:526] Disk successfully formatted (mkfs): ext4 - /dev/disk/azure/data/by-lun/2 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/feff39288781b8a9437126cfecfe10952213f446af81ade220093cdd382ae7aa/globalmount I0726 12:16:25.991570 1 mount_linux.go:544] Attempting to mount disk /dev/disk/azure/data/by-lun/2 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/feff39288781b8a9437126cfecfe10952213f446af81ade220093cdd382ae7aa/globalmount I0726 12:16:25.991584 1 mount_linux.go:218] Mounting cmd (mount) with arguments (-t ext4 -o defaults /dev/disk/azure/data/by-lun/2 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/feff39288781b8a9437126cfecfe10952213f446af81ade220093cdd382ae7aa/globalmount) I0726 12:16:25.998967 1 nodeserver.go:161] NodeStageVolume: format /dev/disk/azure/data/by-lun/2 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/feff39288781b8a9437126cfecfe10952213f446af81ade220093cdd382ae7aa/globalmount successfully. I0726 12:16:25.999763 1 mount_linux.go:577] Attempting to determine if disk "/dev/disk/azure/data/by-lun/2" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/data/by-lun/2]) I0726 12:16:26.002523 1 mount_linux.go:580] Output: "DEVNAME=/dev/disk/azure/data/by-lun/2\nTYPE=ext4\n" I0726 12:16:26.002534 1 resizefs_linux.go:137] ResizeFs.needResize - checking mounted volume /dev/disk/azure/data/by-lun/2 I0726 12:16:26.003661 1 resizefs_linux.go:141] Ext size: filesystem size=10737418240, block size=4096 I0726 12:16:26.003671 1 resizefs_linux.go:156] Volume /dev/disk/azure/data/by-lun/2: device size=10737418240, filesystem size=10737418240, block size=4096 # df -h | grep nvme /dev/nvme0n1p4 149G 14G 136G 10% / /dev/nvme0n1p3 1014M 132M 883M 13% /boot /dev/nvme0n1p2 512M 3.0M 509M 1% /boot/efi /dev/nvme0n3 9.8G 2.1M 9.8G 1% /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/6f066b4e90fa683f5b1003515dbf1c1c9e3b96b2b5d0486aba23c3d260aec707/globalmount /dev/nvme0n2 280G 3.0G 277G 2% /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/7b2be2d25460f661290486981f7ec2c58371afd98dbbeb50e5f8bc31fe983ea9/globalmount /dev/nvme0n4 9.8G 2.1M 9.8G 1% /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/feff39288781b8a9437126cfecfe10952213f446af81ade220093cdd382ae7aa/globalmount ``` ``` - nvme disk # ls -lt /dev/disk/azure/* lrwxrwxrwx 1 root root 15 Jul 25 07:20 /dev/disk/azure/root-part1 -> ../../nvme0n1p1 lrwxrwxrwx 1 root root 15 Jul 25 07:20 /dev/disk/azure/root-part2 -> ../../nvme0n1p2 lrwxrwxrwx 1 root root 15 Jul 25 07:20 /dev/disk/azure/root-part3 -> ../../nvme0n1p3 lrwxrwxrwx 1 root root 15 Jul 25 07:20 /dev/disk/azure/root-part4 -> ../../nvme0n1p4 lrwxrwxrwx 1 root root 13 Jul 25 07:20 /dev/disk/azure/root -> ../../nvme0n1 /dev/disk/azure/data: total 0 drwxr-xr-x 2 root root 80 Jul 25 07:20 by-lun # ls -lt /dev/disk/azure/data/* total 0 lrwxrwxrwx 1 root root 19 Jul 25 07:20 0 -> ../../../../nvme0n2 lrwxrwxrwx 1 root root 19 Jul 25 07:20 1 -> ../../../../nvme0n3 - scsi disk # ls -lt /dev/disk/azure/scsi1 total 0 lrwxrwxrwx 1 root root 12 Jul 25 23:37 lun1 -> ../../../sdd lrwxrwxrwx 1 root root 12 Jul 24 15:51 lun5 -> ../../../sdh lrwxrwxrwx 1 root root 12 Jul 24 14:38 lun2 -> ../../../sdf lrwxrwxrwx 1 root root 12 Jul 24 14:38 lun0 -> ../../../sdc ```

Release note:

feat: support NVME disk
k8s-ci-robot commented 4 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/azuredisk-csi-driver/blob/master/OWNERS)~~ [andyzhangx] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
andyzhangx commented 4 months ago

/retest

andyzhangx commented 4 months ago

/retest

andyzhangx commented 4 months ago

/retest

andyzhangx commented 4 months ago

/cherrypick release-1.30

andyzhangx commented 4 months ago

/cherrypick release-1.29

k8s-infra-cherrypick-robot commented 4 months ago

@andyzhangx: new pull request created: #2434

In response to [this](https://github.com/kubernetes-sigs/azuredisk-csi-driver/pull/2429#issuecomment-2253687720): >/cherrypick release-1.30 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
k8s-infra-cherrypick-robot commented 4 months ago

@andyzhangx: new pull request created: #2435

In response to [this](https://github.com/kubernetes-sigs/azuredisk-csi-driver/pull/2429#issuecomment-2253687742): >/cherrypick release-1.29 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.