search

kubernetes-sigs / aws-fsx-csi-driver

CSI Driver of Amazon FSx for Lustre https://aws.amazon.com/fsx/lustre/
Apache License 2.0
121 stars 77 forks source link

Multiple GRPC errors while accessing the fsx pvc mount #380

Open paramjeet01 opened 2 months ago

paramjeet01 commented 2 months ago

/kind bug

What happened? We are receiving the below error but no issues are seen in accessing the PVC.

"GRPC error" err="rpc error: code = Aborted desc = An operation with the given volume=\"fs-0e******\" is already in progress"@timestamp:2024-04-30T11:18:04.927Z k8s-cluster-name: mycluster*** level:E name:driver.go:104 pid:1 source:fsx-csi-node-d2dft time:0430 11:18:04.927495 container-name:fsx-plugin

What you expected to happen? The error log should be removed or the reason should be found

How to reproduce it (as minimally and precisely as possible)?

Anything else we need to know?:

Environment

jacobwolfaws commented 2 months ago

Hi @paramjeet01, this error log is part of our idempotency check for in-flight requests (one example here: https://github.com/kubernetes-sigs/aws-fsx-csi-driver/blob/master/pkg/driver/node.go#L132). We can examine increasing the logging level, but this log has been very helpful in identifying past bugs with the driver and we have no plans to remove it at this time.

paramjeet01 commented 2 months ago

@jacobwolfaws , Thanks for reverting back early. Is it possible mark this log as DEBUG than ERROR log.