rancher / rio

Application Deployment Engine for Kubernetes
https://rio.io
Apache License 2.0
2.27k stars 228 forks source link

Riofile, persistent volumes and access mode RWX #1034

Closed fff0x closed 4 years ago

fff0x commented 4 years ago

Describe the bug I tried to add persistent volumes to my services using the Riofile, which are backed up by a ceph cluster, so accessmode ReadWriteMany should be supported. Unfortunately the documentation isn't the best at this point, so I have no clue if it's correct what I'm doing here.

To Reproduce Assume the following Riofile content:

services:
 bb-1:
   version: v0
   scale: 1
   image: busybox
   imagePullPolicy: IfNotPresent
   command:
   - sleep
   args:
   - "3600"
   volumes:
   - test:/data,persistent=true

 bb-2:
   version: v0
   scale: 1
   image: busybox
   imagePullPolicy: IfNotPresent
   command:
   - sleep
   args:
   - "3600"
   volumes:
   - test:/data,persistent=true

kubernetes:
  manifest: |-
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: test
      namespace: default
    spec:
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 2Gi
      storageClassName: rook-cephfs
  1. Apply the stack with rio up
  2. Get logoutput from rio system logs (see below)

Expected behavior

All pods come up, builds are starting, the cephfs is mounted into every container.

Kubernetes version: I'm running k3s version: v1.17.4+k3s1, Rio is installed without rdns support. If I did not use the same persistent volume name for multiple services, then everything is working as expected.

Rio version: rio info

Rio Version: v0.7.0 (4afd4901)
Rio CLI Version: v0.7.0 (4afd4901)
Cluster Domain: xxx.local
Cluster Domain IPs: 192.168.2.160
System Namespace: rio-system
Wildcard certificates: xxx.local(true)

Additional context rio system logs output:

E0502 17:37:18.906657       1 controller.go:135] error syncing 'default/bb-1': handler service: failed to update default/test /v1, Kind=PersistentVolumeClaim for service default/bb-1: PersistentVolumeClaim "test" is invalid: spec: Forbidden: is immutable after creation except resources.requests for bound claims, handler template: skip processing, requeuing
rio-controller | E0502 17:37:23.521861       1 controller.go:135] error syncing 'default/bb-2': handler service: failed to update default/test /v1, Kind=PersistentVolumeClaim for service default/bb-2: PersistentVolumeClaim "test" is invalid: spec: Forbidden: is immutable after creation except resources.requests for bound claims, handler template: skip processing, requeuing
StrongMonkey commented 4 years ago

@mbuelte Thanks for reporting it. Yeah we haven't done too much testing around persistent volume, I will take a look at this issue.

fff0x commented 4 years ago

Thank you! It seems they are conflicting each other. Meanwhile I'll try to patch in the VolumeMount later to the deployment as an workaround.

StrongMonkey commented 4 years ago

Should be fixed in v0.7.1-rc2, if you can help validate that @mbuelte

fff0x commented 4 years ago

Works perfect. Many thanks!