search

nextcloud / helm

A community maintained helm chart for deploying Nextcloud on Kubernetes.
GNU Affero General Public License v3.0
315 stars 260 forks source link

changing containerPort doesn't affect apache #363

Open hobyte opened 1 year ago

hobyte commented 1 year ago

Describe your Issue

setting nextcloud.containerPort value doesn't change the Port apache httpd is listening to. it only changes the container Port in the deployment yaml, while the apache config still uses Port 80

Logs and Errors

nextcloud pod logs:

AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 10.244.0.103. Set the 'ServerName' directive globally to suppress this message
(13)Permission denied: AH00072: make_sock: could not bind to address [::]:80
(13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down

Describe your Environment

nextcloud:
  host: cloud.example.com
  configs:
    custom.config.php: |-
      <?php
      $CONFIG = array (
        'overwriteprotocol' => 'https',
        'overwrite.cli.url' => 'https://cloud.example.com',
        'filelocking.enabled' => 'true',
        'loglevel' => '0',
        'enable_previews' => false
      );
  securityContext:
    runAsUser: 1001
    runAsGroup: 1001
  containerPort: 8080
replicaCount: 1
internalDatabase:
  enabled: false
mariadb:
  enabled: true
  primary:
    persistence:
      enabled: true
persistence:
  enabled: true
readinessProbe:
  enabled: false
livenessProbe:
  enabled: false

Additional context, if any

I use a hetzner storage box for my kubernetes storage. It's mounted with a smb storage class. This is my storageClass definition:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: storagebox
  namespace: storage
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: smb.csi.k8s.io
parameters:
  source: "//xyz.your-storagebox.de/xyz"
  csi.storage.k8s.io/provisioner-secret-name: "smbcreds"
  csi.storage.k8s.io/provisioner-secret-namespace: "storage"
  csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
  csi.storage.k8s.io/node-stage-secret-namespace: "storage"
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1001
  - gid=1001
  - cache=loose
  - actimeo=30
  - noserverino

Due to permission issues, User and UserGroup are set to 1001 in nextcloud. I don't think the storage class itself is the issue, but the container port, that isn't configured completely

Mx7ca commented 1 year ago

I'm currently also trying to change the user to 33 (www-data) and I encounter the same problem.

If I understand it correctly, the problem is that the docker-image per default uses Port 80 for Apache2 which seems not to be configurable via Kubernetes. According to https://github.com/nextcloud/helm/pull/98#issuecomment-844551455, it should be possible to use an FPM image in combination with nginx, but even if one could get the webserver listen to something different than port 80, I think the static targetPort in the service template would cause problems.

Update: I've got nginx to run on port 8080 and the targetPort didn't cause a problem. However, now I've got the nginx container that fails if not run as root, so that's not really an improvement. Also the data directory has mode 775 and is owned by root which prevents Nextcloud to start. For the nginx-config, I basically just copied the default value from the nginx-template and changed the listen parameter:

nginx config nginx: enabled: true # securityContext: # runAsUser: 33 config: default: false custom: |- worker_processes auto; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; upstream php-handler { server 127.0.0.1:9000; } server { listen 8080; # HSTS settings # WARNING: Only add the preload option once you read about # the consequences in https://hstspreload.org/. This option # will add the domain to a hardcoded list that is shipped # in all major browsers and getting removed from this list # could take several months. #add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;" always; # set max upload size client_max_body_size 10G; fastcgi_buffers 64 4K; # Enable gzip but do not remove ETag headers gzip on; gzip_vary on; gzip_comp_level 4; gzip_min_length 256; gzip_proxied expired no-cache no-store private no_last_modified no_etag auth; gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy; # Pagespeed is not supported by Nextcloud, so if your server is built # with the `ngx_pagespeed` module, uncomment this line to disable it. #pagespeed off; # HTTP response headers borrowed from Nextcloud `.htaccess` add_header Referrer-Policy "no-referrer" always; add_header X-Content-Type-Options "nosniff" always; add_header X-Download-Options "noopen" always; add_header X-Frame-Options "SAMEORIGIN" always; add_header X-Permitted-Cross-Domain-Policies "none" always; add_header X-Robots-Tag "none" always; add_header X-XSS-Protection "1; mode=block" always; # Remove X-Powered-By, which is an information leak fastcgi_hide_header X-Powered-By; # Path to the root of your installation root /var/www/html; # Specify how to handle directories -- specifying `/index.php$request_uri` # here as the fallback means that Nginx always exhibits the desired behaviour # when a client requests a path that corresponds to a directory that exists # on the server. In particular, if that directory contains an index.php file, # that file is correctly served; if it doesn't, then the request is passed to # the front-end controller. This consistent behaviour means that we don't need # to specify custom rules for certain paths (e.g. images and other assets, # `/updater`, `/ocm-provider`, `/ocs-provider`), and thus # `try_files $uri $uri/ /index.php$request_uri` # always provides the desired behaviour. index index.php index.html /index.php$request_uri; # Rule borrowed from `.htaccess` to handle Microsoft DAV clients location = / { if ( $http_user_agent ~ ^DavClnt ) { return 302 /remote.php/webdav/$is_args$args; } } location = /robots.txt { allow all; log_not_found off; access_log off; } # Make a regex exception for `/.well-known` so that clients can still # access it despite the existence of the regex rule # `location ~ /(\.|autotest|...)` which would otherwise handle requests # for `/.well-known`. location ^~ /.well-known { # The following 6 rules are borrowed from `.htaccess` location = /.well-known/carddav { return 301 /remote.php/dav/; } location = /.well-known/caldav { return 301 /remote.php/dav/; } # Anything else is dynamically handled by Nextcloud location ^~ /.well-known { return 301 /index.php$uri; } try_files $uri $uri/ =404; } # Rules borrowed from `.htaccess` to hide certain paths from clients location ~ ^/(?:build|tests|config|lib|3rdparty|templates|data)(?:$|/) { return 404; } location ~ ^/(?:\.|autotest|occ|issue|indie|db_|console) { return 404; } # Ensure this block, which passes PHP files to the PHP process, is above the blocks # which handle static assets (as seen below). If this block is not declared first, # then Nginx will encounter an infinite rewriting loop when it prepends `/index.php` # to the URI, resulting in a HTTP 500 error response. location ~ \.php(?:$|/) { fastcgi_split_path_info ^(.+?\.php)(/.*)$; set $path_info $fastcgi_path_info; try_files $fastcgi_script_name =404; include fastcgi_params; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param PATH_INFO $path_info; #fastcgi_param HTTPS on; fastcgi_param modHeadersAvailable true; # Avoid sending the security headers twice fastcgi_param front_controller_active true; # Enable pretty urls fastcgi_pass php-handler; fastcgi_intercept_errors on; fastcgi_request_buffering off; } location ~ \.(?:css|js|svg|gif)$ { try_files $uri /index.php$request_uri; expires 6M; # Cache-Control policy borrowed from `.htaccess` access_log off; # Optional: Don't log access to assets } location ~ \.woff2?$ { try_files $uri /index.php$request_uri; expires 7d; # Cache-Control policy borrowed from `.htaccess` access_log off; # Optional: Don't log access to assets } location / { try_files $uri $uri/ /index.php$request_uri; } } }

Maybe some of my test results will help somebody to find a proper solution. But I assume the main objective of this issue (change port used by apache2) needs to be done in the docker-container.

jessebot commented 1 year ago

I've just merged this PR, which might help: https://github.com/nextcloud/helm/commit/a215de8e0cafd940818b888e94806a387500abc4

jessebot commented 1 year ago

Could you try again and see if the issue persists with helm chart version 3.5.12?

jacksgt commented 1 year ago

Hi @jessebot , thanks for your work, unfortunately this does not fix the issue. The problem is that Apache is configured to listen on port 80 (which cannot be changed):

$ kubectl exec -it deploy/nextcloud -- cat /etc/apache2/sites-enabled/000-default.conf 
<VirtualHost *:80>
...
georglauterbach commented 1 year ago

Same issue here. Running a rootless container for obvious security benefits but I need to mount a file like manual.conf to /etc/apache2/ that contains

Listen 8080
ServerName <FQDN>

This is annoying and inconsistent with the web server configuration. ping @jessebot :)

jessebot commented 1 year ago

Sorry, been busy at work! I don't actually use the apache flavor of the docker tag, but let me poke around https://github.com/nextcloud/docker and see if anything jumps out at me :)

jessebot commented 1 year ago

I think to accommodate this, we'd need to have this be configurable in the apache docker container 🤔

@georglauterbach can you post an example of what you did? Is it just an extra mount? Where do you get the config file from? I guess it should be a configmap 🤔

@provokateurin interested in your thoughts as well.

georglauterbach commented 1 year ago

I think to accommodate this, we'd need to have this be configurable in the apache docker container thinking

I knew about the fpm version, but is there another image flavor next to the Apache and fpm flavors?

@georglauterbach can you post an example of what you did? Is it just an extra mount? Where do you get the config file from? I guess it should be a configmap thinking

Indeed, it is a ConfigMap I created manually. I mount it via

extraVolumes:
  - name: extra-configuration-files
    configMap:
      name: extra-configuration-files

extraVolumeMounts:
  - name: extra-configuration-files
    subPath: ports.conf
    mountPath: /etc/apache2/ports.conf
    readOnly: true

in values.yaml.

jessebot commented 1 year ago

I knew about the fpm version, but is there another image flavor next to the Apache and fpm flavors?

There is the regular apache flavor and fpm, but there's also a fpm-alpine version as well :) You can see all the tags here: https://hub.docker.com/_/nextcloud/

For your solution, that seems ok doable 🤔 Do you want to submit a PR for that? If not, it'll probably be a few days before I have some time to write out a solution and test it quickly.

georglauterbach commented 1 year ago

I knew about the fpm version, but is there another image flavor next to the Apache and fpm flavors?

There is the regular apache flavor and fpm, but there's also a fpm-alpine version as well :) You can see all the tags here: https://hub.docker.com/_/nextcloud/

I see, thanks! 👍🏼

For your solution, that seems ok doable 🤔 Do you want to submit a PR for that? If not, it'll probably be a few days before I have some time to write out a solution and test it quickly.

I have close to zero time at the moment, so I cannot provide a PR. I'd really appreciate if you could provide a PR:) When you do, please also try to provide the ServerName part 🙈

devthejo commented 9 months ago

I got it to work as rootless using this config:

image:
  tag: 27.1.3-fpm # https://hub.docker.com/r/library/nextcloud/tags/
nextcloud:
  host: "nextcloud.mydomain.tld"
  configs:
    custom.config.php: |
      <?php
        $CONFIG = array(
          "check_data_directory_permissions"=> false, # fix data directory permissions error
          "trusted_domains" => array (
            $_ENV["NEXTCLOUD_TRUSTED_DOMAINS"], # fix probes 400 error
          ),
        );
  securityContext:
    runAsUser: 1000
    runAsGroup: 1000
    runAsNonRoot: true
    fsGroup: 1000
  podSecurityContext:
    runAsUser: 1000
    runAsGroup: 1000
    runAsNonRoot: true
    fsGroup: 1000
  containerPort: 8080
  extraVolumes:
  - name: nginx-cache
    emptyDir: {}
  extraVolumeMounts:
  - name: nginx-cache
    mountPath: "/var/cache/nginx" # fix permission denied error
nginx:
  ## You need to set an fpm version of the image for nextcloud if you want to use nginx!
  enabled: true
  image:
    repository: nginxinc/nginx-unprivileged
    tag: 1.25.3 # https://hub.docker.com/r/nginxinc/nginx-unprivileged/tags
  containerPort: 8080
georglauterbach commented 9 months ago

@devthejo that looks like you're using NGINX, not Apache. What am I missing here?

devthejo commented 9 months ago

@devthejo that looks like you're using NGINX, not Apache. What am I missing here?

I misspoke, that doesn't solve changing apache port, but the challenge of running nextcloud rootless, I thought that was the underlying objective

georglauterbach commented 9 months ago

Ah, I see. Thank you very much for posting this here!

Wopf commented 7 months ago

As we are running instances in a relatively high restricted k8s environment, where low ports are absolutely no-go. We rebuild every image by adding these layers amongst other things:

ENV APACHE_LISTEN_PORT=8080
RUN sh -c 'find /etc/apache2 -type f | xargs sed -s -i -e "s/Listen 80/Listen ${APACHE_LISTEN_PORT}/"'
EXPOSE ${APACHE_LISTEN_PORT}

The really annoying thing is set -eu and > /usr/local/etc/php/conf.d/redis-session.ini in entrypoint.sh, as it prevents start-up with non-root user.

We handle this by

georglauterbach commented 7 months ago
FYI just for @Wopf:
> `RUN sh -c 'find /etc/apache2 -type f | xargs sed -s -i -e "s/Listen 80/Listen ${APACHE_LISTEN_PORT}/"'` Piping `find` into `xargs` is usually done when people do not know about `find`'s `-exec` argument; why not write ```bash find /etc/apache2 -type f -exec sed -i -E "s/(Listen) 80/\1 ${APACHE_LISTEN_PORT}/" {} \; ``` That is more concise and definitely faster (saving the fork-exec that the pipe has to do). As a rule of thumb: piping into `xargs` can in most cases be done in a better way :)
Wopf commented 7 months ago

FYI just for @Wopf:

RUN sh -c 'find /etc/apache2 -type f | xargs sed -s -i -e "s/Listen 80/Listen ${APACHE_LISTEN_PORT}/"'

Piping find into xargs is usually done when people do not know about find's -exec argument; why not write

find /etc/apache2 -type f -exec sed -i -E "s/(Listen) 80/\1 ${APACHE_LISTEN_PORT}/" {} \;

That is more concise and definitely faster (saving the fork-exec that the pipe has to do). As a rule of thumb: piping into xargs can in most cases be done in a better way :)

Works for sure - piping is a bad habit from the days, when Solaris 2.6 had no gnu-ish shells...