FrankelJb
commented
1 year ago Adding my experience:
Just moved my directory from a hostpath to nfs then started encountering permission issues. I chown -R 33:33 the whole directory and set the security context. This is my error now:
4 Configuring Redis as session handler
3 /entrypoint.sh: 78: cannot create /usr/local/etc/php/conf.d/redis-session.ini: Permission denied
2 Initializing nextcloud 25.0.3.2 ...
1 touch: cannot touch '/var/www/html/nextcloud-init-sync.lock': Permission denied
jessebot
Jeroen0494
tomasodehnal
oliverhu
MrFishFinger
QuinnBast
steled
Description
I've edited this for full context of how we got here, as this issue is getting kind of long, because it needed to be tested in a lot of different ways which lead me in several directions.
This issue is a continuation of the conversation started after #269 was merged. I was originally trying to changed the
podSecurityContext.runAsUserandpodSecurityContext.runAsGroupto33because I was trying to diagnose why the/var/www/html/configdirectory was always owned by root. I am deploying the nextcloud helm chart using persistent volumes on k3s with the default local path provisioner.I learned that the
podSecurityContext.fsGroupwas always being set to82anytime you usednginx.enabledand didn't setpodSecurityContext.fsGroupexplicitly, so I submitted a draft PR here to fix it to so that it checksimage.flavorforalpine: https://github.com/nextcloud/helm/pull/379Through the comments here you can see other things I'm currently testing, because I'm still not sure is it's just the local path provisioner on k3s or k3s itself or what, but the best I can get is 🤷 I'll update this issue description with more clarity as it comes.
Original Issue that was opened on Jan 23
The nginx container in the nextcloud pod won't start and complains of a readonly file system, even if I try to only set the
nextcloud.securityContext.I have created a new cluster and deployed nextcloud with the securityContext parameters from the values.yaml of this repo, including the nginx security context. My entire
values.yamlis here, but the parts that matter are:securityContext parameters in my old `values.yaml`
```yaml nextcloud: # securityContext parameters. For example you may need to define runAsNonRoot directive securityContext: runAsUser: 33 runAsGroup: 33 runAsNonRoot: true readOnlyRootFilesystem: false # securityContext parameters. For example you may need to define runAsNonRoot directive podSecurityContext: runAsUser: 33 runAsGroup: 33 runAsNonRoot: true readOnlyRootFilesystem: false ... nginx: ## You need to set an fpm version of the image for nextcloud if you want to use nginx! enabled: true image: repository: nginx tag: alpine pullPolicy: Always # this is copied almost directly from the values.yaml, but I changed readOnlyRootFilesystem to false while testing securityContext: runAsUser: 82 runAsGroup: 33 runAsNonRoot: true readOnlyRootFilesystem: false ```The nextcloud pod is in a crashloopbackoff state with the offending container being nginx, and this being the logs:
This is the resulting deployment.yaml when I do a
kubectl get deployment -n nextcloud nextcloud-web-app > deployment.yaml:Click me for the nextcloud
```yaml apiVersion: apps/v1 kind: Deployment metadata: annotations: deployment.kubernetes.io/revision: "1" creationTimestamp: "2023-01-23T14:43:58Z" generation: 52 labels: app.kubernetes.io/component: app app.kubernetes.io/instance: nextcloud-web-app app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: nextcloud argocd.argoproj.io/instance: nextcloud-web-app helm.sh/chart: nextcloud-3.4.1 name: nextcloud-web-app namespace: nextcloud resourceVersion: "3340" uid: cde1dd07-103a-4c04-931d-071ab3c5b448 spec: progressDeadlineSeconds: 600 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app.kubernetes.io/component: app app.kubernetes.io/instance: nextcloud-web-app app.kubernetes.io/name: nextcloud strategy: type: Recreate template: metadata: annotations: nextcloud-config-hash: d1d9ac6f86f643b460f8e4e8e886b65382ad49aede8762f8ea74ccd86b7e3f28 nginx-config-hash: 16c61772d9e74de7322870fd3a045598ea01f6e16be155d116423e6a246dcddc php-config-hash: 44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a creationTimestamp: null labels: app.kubernetes.io/component: app app.kubernetes.io/instance: nextcloud-web-app app.kubernetes.io/name: nextcloud spec: containers: - env: - name: POSTGRES_HOST value: nextcloud-web-app-postgresql - name: POSTGRES_DB value: nextcloud - name: POSTGRES_USER valueFrom: secretKeyRef: key: username name: nextcloud-pgsql-credentials - name: POSTGRES_PASSWORD valueFrom: secretKeyRef: key: nextcloudPassword name: nextcloud-pgsql-credentials - name: NEXTCLOUD_ADMIN_USER valueFrom: secretKeyRef: key: username name: nextcloud-admin-credentials - name: NEXTCLOUD_ADMIN_PASSWORD valueFrom: secretKeyRef: key: password name: nextcloud-admin-credentials - name: NEXTCLOUD_TRUSTED_DOMAINS value: nextcloud.vleermuis.tech - name: NEXTCLOUD_DATA_DIR value: /var/www/html/data image: nextcloud:25.0.3-fpm imagePullPolicy: Always name: nextcloud resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/www/ name: nextcloud-main subPath: root - mountPath: /var/www/html name: nextcloud-main subPath: html - mountPath: /var/www/html/data name: nextcloud-main subPath: data - mountPath: /var/www/html/config name: nextcloud-main subPath: config - mountPath: /var/www/html/custom_apps name: nextcloud-main subPath: custom_apps - mountPath: /var/www/tmp name: nextcloud-main subPath: tmp - mountPath: /var/www/html/themes name: nextcloud-main subPath: themes - mountPath: /var/www/html/config/logging.config.php name: nextcloud-config subPath: logging.config.php - mountPath: /var/www/html/config/proxy.config.php name: nextcloud-config subPath: proxy.config.php - mountPath: /var/www/html/config/.htaccess name: nextcloud-config subPath: .htaccess - mountPath: /var/www/html/config/apache-pretty-urls.config.php name: nextcloud-config subPath: apache-pretty-urls.config.php - mountPath: /var/www/html/config/apcu.config.php name: nextcloud-config subPath: apcu.config.php - mountPath: /var/www/html/config/apps.config.php name: nextcloud-config subPath: apps.config.php - mountPath: /var/www/html/config/autoconfig.php name: nextcloud-config subPath: autoconfig.php - mountPath: /var/www/html/config/redis.config.php name: nextcloud-config subPath: redis.config.php - mountPath: /var/www/html/config/smtp.config.php name: nextcloud-config subPath: smtp.config.php - image: nginx:alpine imagePullPolicy: Always livenessProbe: failureThreshold: 3 httpGet: httpHeaders: - name: Host value: nextcloud.vleermuis.tech path: /status.php port: http scheme: HTTP initialDelaySeconds: 45 periodSeconds: 15 successThreshold: 1 timeoutSeconds: 5 name: nextcloud-nginx ports: - containerPort: 80 name: http protocol: TCP readinessProbe: failureThreshold: 3 httpGet: httpHeaders: - name: Host value: nextcloud.vleermuis.tech path: /status.php port: http scheme: HTTP initialDelaySeconds: 45 periodSeconds: 15 successThreshold: 1 timeoutSeconds: 5 resources: {} securityContext: readOnlyRootFilesystem: false runAsGroup: 33 runAsNonRoot: true runAsUser: 82 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /var/www/ name: nextcloud-main subPath: root - mountPath: /var/www/html name: nextcloud-main subPath: html - mountPath: /var/www/html/data name: nextcloud-main subPath: data - mountPath: /var/www/html/config name: nextcloud-main subPath: config - mountPath: /var/www/html/custom_apps name: nextcloud-main subPath: custom_apps - mountPath: /var/www/tmp name: nextcloud-main subPath: tmp - mountPath: /var/www/html/themes name: nextcloud-main subPath: themes - mountPath: /etc/nginx/nginx.conf name: nextcloud-nginx-config subPath: nginx.conf dnsPolicy: ClusterFirst initContainers: - command: - sh - -c - until pg_isready -h nextcloud-web-app-postgresql -U ${POSTGRES_USER} ; do sleep 2 ; done env: - name: POSTGRES_USER valueFrom: secretKeyRef: key: username name: nextcloud-pgsql-credentials image: bitnami/postgresql:14.4.0-debian-11-r23 imagePullPolicy: IfNotPresent name: postgresql-isready resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File restartPolicy: Always schedulerName: default-scheduler securityContext: fsGroup: 82 runAsGroup: 33 runAsNonRoot: true runAsUser: 33 serviceAccount: nextcloud-serviceaccount serviceAccountName: nextcloud-serviceaccount terminationGracePeriodSeconds: 30 volumes: - name: nextcloud-main persistentVolumeClaim: claimName: nextcloud-files - configMap: defaultMode: 420 name: nextcloud-web-app-config name: nextcloud-config - configMap: defaultMode: 420 name: nextcloud-web-app-nginxconfig name: nextcloud-nginx-config status: conditions: - lastTransitionTime: "2023-01-23T14:43:58Z" lastUpdateTime: "2023-01-23T14:43:58Z" message: Deployment does not have minimum availability. reason: MinimumReplicasUnavailable status: "False" type: Available - lastTransitionTime: "2023-01-23T14:43:58Z" lastUpdateTime: "2023-01-23T14:43:58Z" message: ReplicaSet "nextcloud-web-app-66fc5dfcb7" is progressing. reason: ReplicaSetUpdated status: "True" type: Progressing observedGeneration: 52 replicas: 1 unavailableReplicas: 1 updatedReplicas: 1 ```deployment.yamlWhere does the UID 82 come from? (Edit: it comes from the alpine nextcloud and nginx images - that's www-data)
I set that to 33 (nextcloud's www-data user) to test, but it didn't seem to make a difference. Just so it's clear, without editing any of the security contexts, everything works, but I would like the security context to work, because otherwise it causes my restores from backups to fail, because the
/var/www/html/configdirectory is always created with root ownership, which means if the restores run as www-data, they can't restore that particular directory, which is important. I'm hoping the security context fixes that, so that nothing has to run as root in this stack.I'm deploying the
3.4.1nextcloud helm chart via Argo CD onto k3s on Ubuntu 22.04. Update: problem still present in3.5.7helm chart.