accessModes: ReadOnlyMany not working

skassovic commented 1 year ago

/kind bug

What happened? Wanted to use ROX accessMode for volume provisioned by efs csi driver and ended up with "Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY"

similar to issue: #502

What you expected to happen? Volume with ROX access mode mounted to pod.

How to reproduce it (as minimally and precisely as possible)? PV:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: rox-efs-pv
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadOnlyMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs-sc
  csi:
    driver: efs.csi.aws.com
    volumeHandle: <efs_id>:/<sub_path>

PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: rox-efs-claim
spec:
  accessModes:
    - ReadOnlyMany
  storageClassName: efs-sc
  volumeName: rox-efs-pv
  resources:
    requests:
      storage: 5Gi

POD:

apiVersion: v1
kind: Pod
metadata:
  name: rox-efs-pod
spec:
  volumes:
    - name: efs-storage
      persistentVolumeClaim:
        claimName: rox-efs-claim
  containers:
    - name: task-pv-container
      image: <image>
      command: ["/bin/sh"]
      args: ["-c", "tail -f /dev/null"]
      volumeMounts:
        - mountPath: "/mnt"
          name: efs-storage

Anything else we need to know?:

Environment

Kubernetes version (use kubectl version): 1.20
Driver version: 1.4.9

Please also attach debug logs to help us better diagnose Pod events:

Events:
  Type     Reason             Age                 From                Message
  ----     ------             ----                ----                -------
  Normal   NotTriggerScaleUp  17m                 cluster-autoscaler  pod didn't trigger scale-up: 7 pod has unbound immediate PersistentVolumeClaims, 6 max node group size reached
  Warning  FailedScheduling   17m (x2 over 17m)   default-scheduler   0/32 nodes are available: 32 pod has unbound immediate PersistentVolumeClaims.
  Normal   Scheduled          17m                 default-scheduler   Successfully assigned default/rox-efs-pod to <node>
  Warning  FailedMount        15m                 kubelet             Unable to attach or mount volumes: unmounted volumes=[efs-storage], unattached volumes=[default-token-msc2p efs-storage]: timed out waiting for the condition
  Warning  FailedMount        118s (x6 over 13m)  kubelet             Unable to attach or mount volumes: unmounted volumes=[efs-storage], unattached volumes=[efs-storage default-token-msc2p]: timed out waiting for the condition
  Warning  FailedMount        76s (x16 over 17m)  kubelet             MountVolume.SetUp failed for volume "rox-efs-pv" : rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY

efs-plugin logs:

$ kubectl logs efs-csi-node-jq9bv -c efs-plugin -n kube-system
I0316 10:28:42.028259       1 config_dir.go:88] Creating symlink from '/etc/amazon/efs' to '/var/amazon/efs'
I0316 10:28:42.029513       1 metadata.go:63] getting MetadataService...
I0316 10:28:42.031494       1 metadata.go:68] retrieving metadata from EC2 metadata service
I0316 10:28:42.033758       1 driver.go:140] Did not find any input tags.
I0316 10:28:42.046362       1 driver.go:113] Registering Node Server
I0316 10:28:42.046377       1 driver.go:115] Registering Controller Server
I0316 10:28:42.046392       1 driver.go:118] Starting efs-utils watchdog
I0316 10:28:42.046494       1 efs_watch_dog.go:216] Copying /etc/amazon/efs/efs-utils.conf since it doesn't exist
I0316 10:28:42.046576       1 efs_watch_dog.go:216] Copying /etc/amazon/efs/efs-utils.crt since it doesn't exist
I0316 10:28:42.047670       1 driver.go:124] Starting reaper
I0316 10:28:42.047690       1 driver.go:127] Listening for connections on address: &net.UnixAddr{Name:"/csi/csi.sock", Net:"unix"}
E0321 09:05:04.662403       1 driver.go:103] GRPC error: rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY
E0321 09:05:05.260037       1 driver.go:103] GRPC error: rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY
E0321 09:05:06.264597       1 driver.go:103] GRPC error: rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY
E0321 09:05:08.272069       1 driver.go:103] GRPC error: rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY
E0321 09:05:12.288087       1 driver.go:103] GRPC error: rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY
E0321 09:05:20.316451       1 driver.go:103] GRPC error: rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY
E0321 09:05:36.380816       1 driver.go:103] GRPC error: rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY
E0321 09:06:08.403327       1 driver.go:103] GRPC error: rpc error: code = InvalidArgument desc = Volume capability not supported: invalid access mode: MULTI_NODE_READER_ONLY

Ashley-wenyizha commented 1 year ago

hi @skassovic

thanks for reaching out.

efs-csi-driver does not support ReadOnlyMany, the 2 modes we support now are

https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/pkg/driver/node.go#L36-L37

skassovic commented 1 year ago

hi @Ashley-wenyizha

thanks for confirmation, any plan to support ROX in near future?

Ben-Mark commented 1 year ago

Every kubernetes version comes with a backwards compatibility break, every single time. I don't understand how come from 2015, Kubernetes isn't supporting backwards compatibility.

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

elgalu commented 11 months ago

/remove-lifecycle stale

StevenPG commented 7 months ago

Any update on this? Whether there's intent to support ROX in the future?

bhurlow commented 5 months ago

ReadOnlyMany would be really useful for caching commonly used items, and seems less complex than considering multiple writers

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle rotten
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 3 days ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 3 days ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/aws-efs-csi-driver/issues/972#issuecomment-2251431043): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

kubernetes-sigs / aws-efs-csi-driver

accessModes: ReadOnlyMany not working #972