search

kubernetes-csi / csi-driver-smb

This driver allows Kubernetes to access SMB Server on both Linux and Windows nodes.
Apache License 2.0
496 stars 136 forks source link

fix: delete volume error in archive deletion mode #831

Closed andyzhangx closed 2 months ago

andyzhangx commented 2 months ago

What type of PR is this? /kind bug

What this PR does / why we need it: fix: delete volume error in archive deletion mode

when onDelete: archive, there could be delete volume error with error msg rename xxx: no such file or directory since there is a bug in csi-provisioner that DeleteVolume may be performed twice, this PR adds a 1 min cache and lock to workaround this issue.

[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334699       1 utils.go:76] GRPC call: /csi.v1.Controller/DeleteVolume
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334752       1 utils.go:77] GRPC request: {"secrets":"***stripped***","volume_id":"135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive"}
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334843       1 controllerserver.go:154] DeleteVolume: found mountOptions(dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks) for volume(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334864       1 controllerserver.go:169] begin to delete or archive subdirectory since secret is provided
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334876       1 controllerserver.go:292] internally mounting //135.236.234.180/share at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.335104       1 nodeserver.go:209] NodeStageVolume: targetPath(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2) volumeID(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive) context(map[source://135.236.234.180/share]) mountflags([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks]) mountOptions([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks])
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.335322       1 mount_linux.go:218] Mounting cmd (mount) with arguments (-t cifs -o dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks,<masked> //135.236.234.180/share /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.373268       1 nodeserver.go:241] volume(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive) mount "//135.236.234.180/share" on "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2" succeeded
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.374874       1 controllerserver.go:186] DeleteVolume: subdirectory(smb-4792/pvc-ckk2v) contains '/', make sure the parent directory(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792) exists
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.381550       1 controllerserver.go:193] archiving subdirectory /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/smb-4792/pvc-ckk2v --> /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.381577       1 controllerserver.go:195] removing archived subdirectory at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.383991       1 controllerserver.go:199] removed archived subdirectory at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.391986       1 controllerserver.go:310] internally unmounting /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.392012       1 nodeserver.go:264] NodeUnstageVolume: CleanupMountPoint on /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2 with volume 135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.393671       1 mount_helper_common.go:93] unmounting "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2" (corruptedMount: false, mounterCanSkipMountPointChecks: true)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.393698       1 mount_linux.go:360] Unmounting /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.399036       1 mount_helper_common.go:150] Deleting path "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2"
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.399168       1 nodeserver.go:273] NodeUnstageVolume: unmount volume 135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive on /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2 successfully
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.399192       1 utils.go:83] GRPC response: {}
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427703       1 utils.go:76] GRPC call: /csi.v1.Controller/DeleteVolume
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427736       1 utils.go:77] GRPC request: {"secrets":"***stripped***","volume_id":"135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive"}
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427840       1 controllerserver.go:154] DeleteVolume: found mountOptions(dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks) for volume(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427855       1 controllerserver.go:169] begin to delete or archive subdirectory since secret is provided
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427871       1 controllerserver.go:292] internally mounting //135.236.234.180/share at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.428098       1 nodeserver.go:209] NodeStageVolume: targetPath(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2) volumeID(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive) context(map[source://135.236.234.180/share]) mountflags([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks]) mountOptions([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks])
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.428345       1 mount_linux.go:218] Mounting cmd (mount) with arguments (-t cifs -o dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks,<masked> //135.236.234.180/share /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.454514       1 nodeserver.go:241] volume(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive) mount "//135.236.234.180/share" on "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2" succeeded
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.454559       1 controllerserver.go:186] DeleteVolume: subdirectory(smb-4792/pvc-ckk2v) contains '/', make sure the parent directory(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792) exists
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.456730       1 controllerserver.go:193] archiving subdirectory /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/smb-4792/pvc-ckk2v --> /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.456753       1 controllerserver.go:195] removing archived subdirectory at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.470770       1 controllerserver.go:199] removed archived subdirectory at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.476121       1 controllerserver.go:310] internally unmounting /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.476146       1 nodeserver.go:264] NodeUnstageVolume: CleanupMountPoint on /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2 with volume 135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.476172       1 mount_helper_common.go:93] unmounting "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2" (corruptedMount: false, mounterCanSkipMountPointChecks: true)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.476185       1 mount_linux.go:360] Unmounting /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.484264       1 mount_helper_common.go:150] Deleting path "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2"
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.484631       1 nodeserver.go:273] NodeUnstageVolume: unmount volume 135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive on /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2 successfully
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] E0829 08:02:15.484657       1 utils.go:81] GRPC error: rpc error: code = Internal desc = archive subdirectory(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/smb-4792/pvc-ckk2v, /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v) failed with rename /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/smb-4792/pvc-ckk2v /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v: no such file or directory

Which issue(s) this PR fixes:

Fixes #

Requirements:

Special notes for your reviewer:

Release note:

fix: delete volume error in archive deletion mode
k8s-ci-robot commented 2 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-csi/csi-driver-smb/blob/master/OWNERS)~~ [andyzhangx] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
coveralls commented 2 months ago

Pull Request Test Coverage Report for Build 10634580152

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

Changes Missing Coverage Covered Lines Changed/Added Lines %
pkg/smb/smb.go 1 3 33.33%
pkg/smb/controllerserver.go 11 20 55.0%
<!-- Total: 12 23 52.17% -->
Totals Coverage Status
Change from base Build 10609986454: -0.5%
Covered Lines: 954
Relevant Lines: 1217
💛 - Coveralls
andyzhangx commented 2 months ago

/retest

andyzhangx commented 2 months ago

/retest