search

bunkerity / bunkerweb

🛡️ Open-source and next-generation Web Application Firewall (WAF)
https://www.bunkerweb.io
GNU Affero General Public License v3.0
6.39k stars 360 forks source link

Auto disable ETag when USE_REVERSE_PROXY=yes and USE_CLIENT_CACHE=yes #70

Closed e-minguez closed 3 years ago

e-minguez commented 3 years ago

I'm trying to use bunkerized-nginx as a reverse proxy for nextcloud and for home-assistant.

Home assistant works fine but I'm not able to use it as reverse proxy for nextcloud. Let me try to ellaborate. The nextcloud client and webdav works fine, but the web ui doesn't load all the assets as it should and I cannot log in.

It looks like it is trying to serve them from /etc/nginx/html/ instead of the proper file path.

The bunkerized-nginx container logs say:

==> /var/log/access.log <==

127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET / HTTP/2.0" 302 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET /login HTTP/2.0" 200 4793 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET / HTTP/2.0" 302 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET /login HTTP/2.0" 200 4793 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET /apps/files_rightclick/css/app.css?v=46c85d58-0 HTTP/2.0" 404 548 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET /core/css/guest.css?v=a100dc34-0 HTTP/2.0" 404 548 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET /core/js/dist/main.js?v=a100dc34-0 HTTP/2.0" 404 548 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET /core/js/dist/files_fileinfo.js?v=a100dc34-0 HTTP/2.0" 404 548 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET /core/js/dist/files_client.js?v=a100dc34-0 HTTP/2.0" 404 548 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
127.0.0.1 - - nextcloud.example.com [30/Dec/2020:14:58:23 +0000] "GET /js/core/merged-template-prepend.js?v=a100dc34-0 HTTP/2.0" 404 548 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"

==> /var/log/error.log <==
2020/12/30 14:58:23 [error] 15812#15812: *3803 open() "/etc/nginx/html/apps/files_rightclick/css/app.css" failed (2: No such file or directory), client: 127.0.0.1, server: nextcloud.example.com, request: "GET /apps/files_rightclick/css/app.css?v=46c85d58-0 HTTP/2.0", host: "nextcloud.example.com"
2020/12/30 14:58:23 [error] 15812#15812: *3803 open() "/etc/nginx/html/core/css/guest.css" failed (2: No such file or directory), client: 127.0.0.1, server: nextcloud.example.com, request: "GET /core/css/guest.css?v=a100dc34-0 HTTP/2.0", host: "nextcloud.example.com"
2020/12/30 14:58:23 [error] 15812#15812: *3803 open() "/etc/nginx/html/core/js/dist/main.js" failed (2: No such file or directory), client: 127.0.0.1, server: nextcloud.example.com, request: "GET /core/js/dist/main.js?v=a100dc34-0 HTTP/2.0", host: "nextcloud.example.com"
2020/12/30 14:58:23 [error] 15812#15812: *3803 open() "/etc/nginx/html/core/js/dist/files_fileinfo.js" failed (2: No such file or directory), client: 127.0.0.1, server: nextcloud.example.com, request: "GET /core/js/dist/files_fileinfo.js?v=a100dc34-0 HTTP/2.0", host: "nextcloud.example.com"
2020/12/30 14:58:23 [error] 15812#15812: *3803 open() "/etc/nginx/html/core/js/dist/files_client.js" failed (2: No such file or directory), client: 127.0.0.1, server: nextcloud.example.com, request: "GET /core/js/dist/files_client.js?v=a100dc34-0 HTTP/2.0", host: "nextcloud.example.com"
2020/12/30 14:58:23 [error] 15812#15812: *3803 open() "/etc/nginx/html/js/core/merged-template-prepend.js" failed (2: No such file or directory), client: 127.0.0.1, server: nextcloud.example.com, request: "GET /js/core/merged-template-prepend.js?v=a100dc34-0 HTTP/2.0", host: "nextcloud.example.com"

I'm not using docker or docker-compose but podman (and I guess this is irrelevant).

The container is executed as:

/usr/bin/podman run \
  -d --restart=always \
  -p 8000:8080 \
  -p 8443:8443 \
  -v /home/edu/containers/bunkerized-nginx/letsencrypt:/etc/letsencrypt:z \
  -v /home/edu/containers/bunkerized-nginx/cache:/cache:z \
  -v /home/edu/containers/bunkerized-nginx/server-confs:/server-confs:ro,z \
  --env-file /home/edu/containers/bunkerized-nginx/scripts/podman.env \
  --name=bunkerized-nginx docker.io/bunkerity/bunkerized-nginx:latest

The environment file looks like:

# Servers configuration

SERVER_NAME=nextcloud.example.com homeassistant.example.com
nextcloud.example.com_REVERSE_PROXY_URL=/
nextcloud.example.com_REVERSE_PROXY_HOST=http://192.168.66.98:8080
nextcloud.example.com_ALLOWED_METHODS=GET|POST|HEAD|PROPFIND|DELETE|PUT|MKCOL|MOVE|COPY|PROPPATCH|REPORT
homeassistant.example.com_REVERSE_PROXY_URL=/
homeassistant.example.com_REVERSE_PROXY_HOST=http://192.168.66.98:8123

# Multisite reverse

USE_REVERSE_PROXY=yes
MULTISITE=yes
SERVE_FILES=no
DISABLE_DEFAULT_SERVER=yes
REDIRECT_HTTP_TO_HTTPS=yes
AUTO_LETS_ENCRYPT=yes
USE_PROXY_CACHE=yes
USE_GZIP=yes
USE_BROTLI=yes
PROXY_REAL_IP=yes
PROXY_REAL_IP_HEADER=X-Forwarded-For
PROXY_REAL_IP_RECURSIVE=on
PROXY_REAL_IP_FROM=192.168.0.0/16 172.16.0.0/12 10.0.0.0/8

# Remove unused features

USE_DNSBL=no
USE_FAIL2BAN=no
USE_LIMIT_REQ=no
USE_CLAMAV_SCAN=no
USE_CLAMAV_UPLOAD=no
USE_MODSECURITY=no
USE_MODSECURITY_CRS=no

# Nextcloud specific
X_FRAME_OPTIONS=SAMEORIGIN
MAX_CLIENT_SIZE=10G
USE_CLIENT_CACHE=yes

The custom server conf is just to enable the /.well-known addresses as per nextcloud documentation https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html#nginx

location /.well-known/carddav {
    return 301 $scheme://$host/remote.php/dav;
}

location /.well-known/caldav {
    return 301 $scheme://$host/remote.php/dav;
}

I've tried to remove as many as features as possible to try to debug the issue but no dice.

The current nextcloud is running in its own container and running with a regular nginx in front. The nginx.conf looks like (it is https://github.com/nextcloud/docker/blob/master/.examples/docker-compose/with-nginx-proxy/mariadb-cron-redis/fpm/web/nginx.conf):

worker_processes  auto;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    set_real_ip_from  10.0.0.0/8;
    set_real_ip_from  172.16.0.0/12;
    set_real_ip_from  192.168.0.0/16;
    real_ip_header    X-Forwarded-For;
    real_ip_recursive on;

    #gzip  on;

    upstream php-handler {
        server localhost:9000;
    }

    server {
        listen 80;

        # Add headers to serve security related headers
        # Before enabling Strict-Transport-Security headers please read into this
        # topic first.
        # add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;";
        #
        # WARNING: Only add the preload option once you read about
        # the consequences in https://hstspreload.org/. This option
        # will add the domain to a hardcoded list that is shipped
        # in all major browsers and getting removed from this list
        # could take several months.
        add_header Referrer-Policy "no-referrer" always;
        add_header X-Content-Type-Options "nosniff" always;
        add_header X-Download-Options "noopen" always;
        add_header X-Frame-Options "SAMEORIGIN" always;
        add_header X-Permitted-Cross-Domain-Policies "none" always;
        add_header X-Robots-Tag "none" always;
        add_header X-XSS-Protection "1; mode=block" always;

        # Remove X-Powered-By, which is an information leak
        fastcgi_hide_header X-Powered-By;

    # Path to the root of your installation
        root /var/www/html;

        location = /robots.txt {
            allow all;
            log_not_found off;
            access_log off;
        }

        # The following 2 rules are only needed for the user_webfinger app.
        # Uncomment it if you're planning to use this app.
        #rewrite ^/.well-known/host-meta /public.php?service=host-meta last;
        #rewrite ^/.well-known/host-meta.json /public.php?service=host-meta-json
        # last;

        location = /.well-known/carddav {
            return 301 $scheme://$host:$server_port/remote.php/dav;
        }
        location = /.well-known/caldav {
            return 301 $scheme://$host:$server_port/remote.php/dav;
        }

        # set max upload size
        client_max_body_size 10G;
        fastcgi_buffers 64 4K;

        # Enable gzip but do not remove ETag headers
        gzip on;
        gzip_vary on;
        gzip_comp_level 4;
        gzip_min_length 256;
        gzip_proxied expired no-cache no-store private no_last_modified no_etag auth;
        gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy;

        # Uncomment if your server is build with the ngx_pagespeed module
        # This module is currently not supported.
        #pagespeed off;

        location / {
            rewrite ^ /index.php;
        }

        location ~ ^\/(?:build|tests|config|lib|3rdparty|templates|data)\/ {
            deny all;
        }
        location ~ ^\/(?:\.|autotest|occ|issue|indie|db_|console) {
            deny all;
        }

        location ~ ^\/(?:index|remote|public|cron|core\/ajax\/update|status|ocs\/v[12]|updater\/.+|oc[ms]-provider\/.+)\.php(?:$|\/) {
            fastcgi_split_path_info ^(.+?\.php)(\/.*|)$;
            set $path_info $fastcgi_path_info;
            try_files $fastcgi_script_name =404;
            include fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param PATH_INFO $path_info;
            # fastcgi_param HTTPS on;

            # Avoid sending the security headers twice
            fastcgi_param modHeadersAvailable true;

            # Enable pretty urls
            fastcgi_param front_controller_active true;
            fastcgi_pass php-handler;
            fastcgi_intercept_errors on;
            fastcgi_request_buffering off;
        }

        location ~ ^\/(?:updater|oc[ms]-provider)(?:$|\/) {
            try_files $uri/ =404;
            index index.php;
        }

        # Adding the cache control header for js, css and map files
        # Make sure it is BELOW the PHP block
        location ~ \.(?:css|js|woff2?|svg|gif|map)$ {
            try_files $uri /index.php$request_uri;
            add_header Cache-Control "public, max-age=15778463";
            # Add headers to serve security related headers (It is intended to
            # have those duplicated to the ones above)
            # Before enabling Strict-Transport-Security headers please read into
            # this topic first.
            #add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;" always;
            #
            # WARNING: Only add the preload option once you read about
            # the consequences in https://hstspreload.org/. This option
            # will add the domain to a hardcoded list that is shipped
            # in all major browsers and getting removed from this list
            # could take several months.
            add_header Referrer-Policy "no-referrer" always;
            add_header X-Content-Type-Options "nosniff" always;
            add_header X-Download-Options "noopen" always;
            add_header X-Frame-Options "SAMEORIGIN" always;
            add_header X-Permitted-Cross-Domain-Policies "none" always;
            add_header X-Robots-Tag "none" always;
            add_header X-XSS-Protection "1; mode=block" always;

            # Optional: Don't log access to assets
            access_log off;
        }

        location ~ \.(?:png|html|ttf|ico|jpg|jpeg|bcmap)$ {
            try_files $uri /index.php$request_uri;
            # Optional: Don't log access to other assets
            access_log off;
        }
    }
}

The nextcloud config.php relevant lines look like:

  'trusted_domains' => 
  array (
    0 => 'nextcloud.example.com',
    1 => '192.168.66.98',
  ),
  'datadirectory' => '/var/www/html/data',
  'overwrite.cli.url' => 'http://nextcloud.example.com',
  'overwriteprotocol' => 'https',

Any help?

Thanks!

fl0ppy-d1sk commented 3 years ago

Hi @e-minguez,

Can you try to remove the USE_CLIENT_CACHE=yes env var ?

e-minguez commented 3 years ago

Hi @e-minguez,

Can you try to remove the USE_CLIENT_CACHE=yes env var ?

Aaaaaaaaaaaaaaand it works! 🏅 What kind of wizardy does that variable?

fl0ppy-d1sk commented 3 years ago

It sends the Cache-Control header to the client and enables ETag by default. I think ETag is the culprit. We should auto disable it when USE_REVERSE_PROXY=yes and USE_CLIENT_CACHE=yes.

e-minguez commented 3 years ago

Thank you

It sends the Cache-Control header to the client and enables ETag by default. I think ETag is the culprit. We should auto disable it when USE_REVERSE_PROXY=yes and USE_CLIENT_CACHE=yes.

Do you want me to close the issue or you want to keep it to make that modification?

Thanks again!

fl0ppy-d1sk commented 3 years ago

Let's keep it open as a reminder :).

fl0ppy-d1sk commented 3 years ago

Hello @e-minguez,

Since v1.2.4, the USE_CLIENT_CACHE=yes should now be safe to use in combination with USE_REVERSE_PROXY=yes. Thanks for reporting this.