search

nextcloud / helm

A community maintained helm chart for deploying Nextcloud on Kubernetes.
GNU Affero General Public License v3.0
296 stars 258 forks source link

Nextcloud pods failed with EFS as persistent volume #433

Open SerhiiK opened 10 months ago

SerhiiK commented 10 months ago

Describe your Issue

Nextcldoud can't work properly with EFS as persistent volume through user and group issues.

Logs and Errors

If use default storageClass faced permission issue

Logs Initializing nextcloud 27.0.2.1 ... rsync: [generator] chown "/var/www/html/." failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/.github" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/.github/workflows" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php/ext" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php/src" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php/src/AWS" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php/src/AWS/CRT" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php/src/AWS/CRT/Auth" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php/src/AWS/CRT/HTTP" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php/src/AWS/CRT/IO" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-crt-php/src/AWS/CRT/Internal" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ACMPCA" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/EMRServerless" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/EMRServerless/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Ec2" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Ec2/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Ecr" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Ecr/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Ecs" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Ecs/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Efs" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Efs/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElastiCache" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElastiCache/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticBeanstalk" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticBeanstalk/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticInference" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticInference/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticLoadBalancing" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticLoadBalancing/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticLoadBalancingV2" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticLoadBalancingV2/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticTranscoder" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticTranscoder/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticsearchService" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/ElasticsearchService/Exception" failed: Operation not permitted (1) .... rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Emr" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Emr/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Endpoint" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Endpoint/UseDualstackEndpoint" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Endpoint/UseDualstackEndpoint/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Endpoint/UseFipsEndpoint" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/Endpoint/UseFipsEndpoint/Exception" failed: Operation not permitted (1) rsync: [generator] chown "/var/www/html/3rdparty/aws/aws-sdk-php/src/EndpointDiscovery" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.htaccess.nDcRHg" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.user.ini.7Bw0B5" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.AUTHORS.N71yXb" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.COPYING.v9zX8I" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.console.php.WnbczZ" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.cron.php.jsVxby" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.index.html.PS802l" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.index.php.NDpOn3" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.occ.R7KEHH" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.public.php.3RP2WP" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.remote.php.FKkukp" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.robots.txt.d62oBz" failed: Operation not permitted (1) rsync: [receiver] chown "/var/www/html/.status.php.6Xqguy" failed: Operation not permitted (1)

Efs driver by default can use random POSIX user id and group id. In this example is 1004:1004. Try to fix it by creating special storageClass with parameters uid and gid

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-nextcloud
mountOptions:
  - tls
parameters:
  basePath: /dynamic_provisioning
  directoryPerms: '700'
  fileSystemId: fs-[ID]
  gid: '33'
  provisioningMode: efs-ap
  uid: '33'
provisioner: efs.csi.aws.com
reclaimPolicy: Retain
volumeBindingMode: Immediate

After it all files has owner www-data:www-data but nextcloud shows next logs:

Logs ``` crond: crond (busybox 1.30.1) started, log level 0 crond: user:www-data entry:(null) 100001000010000100001000010000100001000010000100001000010000 111111111111111111111111 11111111111111111111111111111111 111111111111 1111111 crond: user:www-data entry:(null) 100001000010000100001000010000100001000010000100001000010000 111111111111111111111111 11111111111111111111111111111111 111111111111 1111111 crond: wakeup dt=48 crond: file www-data: crond: line php -f /var/www/html/cron.php crond: job: 0 php -f /var/www/html/cron.php crond: child running /bin/bash crond: USER www-data pid 7 cmd php -f /var/www/html/cron.php Warning: require_once(/var/www/html/lib/versioncheck.php): Failed to open stream: No such file or directory in /var/www/html/cron.php on line 40 Fatal error: Uncaught Error: Failed opening required '/var/www/html/lib/versioncheck.php' (include_path='.:/usr/local/lib/php') in /var/www/html/cron.php:40 Stack trace: #0 {main} thrown in /var/www/html/cron.php on line 40 crond: wakeup dt=10 crond: wakeup dt=50 crond: file www-data: crond: line php -f /var/www/html/cron.php ``` Nextcloud container in pod can't pass probes and failed.

Describe your Environment

        internalDatabase:
          enabled: false

        externalDatabase:
          enabled: true
          host: [URL]
          database: next
          user: admin
          password: pass

        persistence:
          enabled: true
          storageClass: "efs-nextcloud"
          accessMode: ReadWriteMany

        cronjob:
          enabled: true

        nextcloud:
          host: [HOST]
          podSecurityContext:
            fsGroup: 33
          defaultConfigs:
            autoconfig.php: true
          configs:
            test.config.php: |-
              <?php
              $CONFIG = array (
                'instanceid' => getenv('CONFIG_INSTANCEID'),
                'overwrite.cli.url' => '[HOST]',
                'passwordsalt' => getenv('CONFIG_PASSWORDSALT'),
                'secret' => getenv('CONFIG_SECRET'),
                'trusted_domains' => 
                  array (
                    0 => '[HOST]',
                  ),
                'overwrite.cli.url' => '[HOST]',
                'overwriteprotocol' => 'https',  
                'objectstore' => 
                  array (
                    'class' => '\\OC\\Files\\ObjectStore\\S3',
                    'arguments' => 
                      array (
                        'bucket' => getenv('CONFIG_BUCKET'),
                        'autocreate' => true,
                        'key' => getenv('CONFIG_S3_ACCESS_KEY'),
                        'secret' => getenv('CONFIG_S3_SECRET_KEY'),
                      ),
                  ),
              );

          extraEnv:
          ...

        ingress:     
          enabled: true
          ...
jessebot commented 10 months ago

Thanks for submitting an issue!

In the future, please post all logs as text instead of an image, so that other users can search your errors in the future, and to make this more accessible to users with vision disabilities as since there's no alt text, a screen reader app used by someone who is blind or has no vision, will just show that as "image" with no further context.

If this is not your complete values.yaml, could you please post your whole values.yaml? This helps us evaluate all possible issues better. For instance, are you using nginx? which nextcloud image flavor/tag are you using? Depending on these answers, the security context user changes. Some directories are always owned by root as well, so that's something to keep in mind.

Can you also try this with the latest version of the nextcloud helm chart (4.1.0) and let us know if the issue persists? There's been some minor changes as well as the nextcloud image itself that could be at play and we just want to make sure.

jessebot commented 10 months ago

I think #318 may also be the same issue.