nginx rewrite rule example in Install > Containers is broken

[avoinea]

The nginx rewrite rule is broken. Deploying with this configuration you will not be able to login due to backend error:

 /++api++/POST_application_json_
backend_1   | Traceback (innermost last):
backend_1   |   Module ZPublisher.WSGIPublisher, line 167, in transaction_pubevents
backend_1   |   Module ZPublisher.WSGIPublisher, line 376, in publish_module
backend_1   |   Module ZPublisher.WSGIPublisher, line 271, in publish
backend_1   |   Module ZPublisher.mapply, line 85, in mapply
backend_1   |   Module ZPublisher.WSGIPublisher, line 68, in call_object
backend_1   |   Module plone.rest.service, line 22, in __call__
backend_1   |   Module plone.restapi.services, line 19, in render
backend_1   |   Module plone.restapi.services.content.add, line 41, in reply
backend_1   | zExceptions.BadRequest: Property '@type' is required

Instead, this works as expected: https://6.dev-docs.plone.org/volto/deploying/seamless-mode.html#nginx-example-config-for-seamless-mode-deployments

[stevepiercy]

Thanks for the report.

We just reviewed and tested the configuration in https://github.com/plone/documentation/pull/1348 and it worked. It is possible that no one logged out and logged back in, but I want to make sure that the proposed fix will in fact work. I also want all examples to be consistent with one another.

# https://6.dev-docs.plone.org/volto/deploying/seamless-mode.html#nginx-example-config-for-seamless-mode-deployments
rewrite ^/\+\+api\+\+($|/.*) /VirtualHostBase/http/myservername.org/Plone/++api++/VirtualHostRoot/$1 break;

# https://github.com/plone/documentation/pull/1348/files
rewrite ^/(\+\+api\+\+\/?)+($|/.*) /VirtualHostBase/http/$server_name/Plone/++api++/VirtualHostRoot/$2 break;

@avoinea would you please double-check my work at these links?

• https://regex101.com/r/j0GUnH/1
  • https://github.com/plone/documentation/pull/1348/files
• https://regex101.com/r/E2Z6Sh/1
  • https://6.dev-docs.plone.org/volto/deploying/seamless-mode.html#nginx-example-config-for-seamless-mode-deployments

Also note that the examples for seamless mode hardcode myservername.org instead of using a server environment variable $servername, and should be updated to the latter. I would suggest that we update all of those as well.

[avoinea]

@stevepiercy As nowadays deploying a site http-only is a non-sense, we should provide the https version of the nginx conf files.

Here is what I have right now:

myserver.conf

# Security headers
add_header X-Frame-Options "SAMEORIGIN";
add_header Strict-Transport-Security "max-age=15768000; includeSubDomains";
add_header X-XSS-Protection "1; mode=block";
add_header X-Content-Type-Options "nosniff";
add_header Content-Security-Policy "default-src 'self' blob: data: https://*.myserver.com; base-uri 'self'; form-action 'self' https://*.myserver.com http://*.myserver.com; connect-src 'self' https://*.myserver.com; font-src 'self' data: https://fonts.gstatic.com/ https://*.myserver.com; frame-src 'self' https://*.myserver.com; img-src http: https: blob: data:; script-src 'self' 'unsafe-inline' 'unsafe-eval' blob: data: https://*.myserver.com; style-src 'self' 'unsafe-inline' https://*.myserver.com; frame-ancestors 'self' https://*.myserver.com; object-src 'self' https://*.myserver.com;";
add_header Referrer-Policy "strict-origin";
add_header Permissions-Policy "geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=()";

upstream backend {
  server backend:8080;
}
upstream frontend {
  server frontend:3000;
}

server {
    listen               443 ssl http2;
    server_name          www.myserver.com;
    ssl_certificate     /etc/certs/myserver.crt;
    ssl_certificate_key /etc/certs/myserver.key;
    ssl_prefer_server_ciphers on;
    ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;

    proxy_read_timeout 300;
    proxy_connect_timeout 300;
    proxy_send_timeout 300;
    client_max_body_size 100M;

    error_page 500 502 503 504 /custom_50x.html;
    location = /custom_50x.html {
      root /etc/nginx/conf.d/html;
      internal;
    }

    location ~ /\+\+api\+\+($|/.*) {
        rewrite ^/\+\+api\+\+($|/.*) /VirtualHostBase/https/www.myserver.com:443/Plone/++api++/VirtualHostRoot/$1 break;
        proxy_pass http://backend;
    }

    location ~ / {
      location ~* \.(js|jsx|css|less|swf|eot|ttf|otf|woff|woff2)$ {
          add_header Cache-Control "public";
          expires +1y;
          proxy_pass http://frontend;
      }
      location ~* static.*\.(ico|jpg|jpeg|png|gif|svg)$ {
          add_header Cache-Control "public";
          expires +1y;
          proxy_pass http://frontend;
      }

      proxy_set_header        Host $host;
      proxy_set_header        X-Real-IP $remote_addr;
      proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header        X-Forwarded-Proto $scheme;
      proxy_redirect http:// https://;
      proxy_pass http://frontend;
  }
}

server {
  listen               443 ssl http2;
  server_name          myserver.com;
  ssl_certificate     /etc/certs/myserver.crt;
  ssl_certificate_key /etc/certs/myserver.key;
  ssl_prefer_server_ciphers on;
  ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
  return 301 https://www.myserver.com$request_uri;
}

server {
  listen         80;
  listen    [::]:80;
  server_name www.myserver.com;
  return 301 https://$host$request_uri;
}

nginx.conf

user  nginx;
worker_processes  2;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    client_max_body_size 100M;
    proxy_read_timeout 300;
    proxy_connect_timeout 300;
    proxy_send_timeout 300;

    # Cache
    ##
    # common caching setup; use "proxy_cache off;" to override
    ##
    proxy_cache_path  /var/cache/nginx  levels=1:2 keys_zone=thecache:100m max_size=4000m inactive=1440m;
    proxy_temp_path   /tmp;
    proxy_redirect                  off;
    proxy_cache                     thecache;
    proxy_set_header                Host $host;
    proxy_set_header                X-Real-IP $remote_addr;
    proxy_set_header                X-Forwarded-For $proxy_add_x_forwarded_for;
    client_body_buffer_size         128k;
    proxy_buffer_size               4k;
    proxy_buffers                   4 32k;
    proxy_busy_buffers_size         64k;
    proxy_temp_file_write_size      64k;
    proxy_cache_bypass              $cookie___ac;
    proxy_http_version              1.1;
    add_header X-Cache-Status $upstream_cache_status;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  300;

    brotli on;
    brotli_static on;

    gzip on;
    gzip_disable "msie6";

    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_min_length 256;
    gzip_types
        application/atom+xml
        application/geo+json
        application/javascript
        application/x-javascript
        application/json
        application/ld+json
        application/manifest+json
        application/rdf+xml
        application/rss+xml
        application/xhtml+xml
        application/xml
        font/eot
        font/otf
        font/ttf
        image/svg+xml
        text/css
        text/javascript
        text/plain
        text/xml;

    include /etc/nginx/conf.d/*.conf;
}

docker-compose.yml

version: "3"
services:
  nginx:
    image: fholzer/nginx-brotli:v1.19.1
    restart: unless-stopped
    volumes:
    - ./nginx/conf.d:/etc/nginx/conf.d:ro
    - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
    - /etc/letsencrypt/live/myserver.com/fullchain.pem:/etc/certs/myserver.crt:ro
    - /etc/letsencrypt/live/myserver.com/privkey.pem:/etc/certs/myserver.key:ro
    ports:
    - "80:80"
    - "443:443"
    environment:
      TZ: "Europe/Bucharest"

If you don't want brotli compression, just remove it from nginx.conf and switch to the official nginx docker-image.

[avoinea]

# https://6.dev-docs.plone.org/volto/deploying/seamless-mode.html#nginx-example-config-for-seamless-mode-deployments
rewrite ^/\+\+api\+\+($|/.*) /VirtualHostBase/http/myservername.org/Plone/++api++/VirtualHostRoot/$1 break;

# https://github.com/plone/documentation/pull/1348/files
rewrite ^/(\+\+api\+\+\/?)+($|/.*) /VirtualHostBase/http/$server_name/Plone/++api++/VirtualHostRoot/$2 break;

@avoinea would you please double-check my work at these links?

:thinking: Yes, I think the culprit was the $1 instead of $2 at the end of the rewrite rule.

[stevepiercy]

🤔 Yes, I think the culprit was the $1 instead of $2 at the end of the rewrite rule.

Well, in the first example, there is one capture group as denoted by (), whereas in the second there are two, so the difference between $1 and $2 makes sense. Both "work". I would like to see what @mauritsvanrees and @Querela can test.

Regarding https, I agree, but that brings up the issue of how to generate the certificate file and key, and how to renew the certificates. That would be a good separate issue to discuss, both for local development and remote staging and production deployments. IMO, self-signed certificates for local, and Let's Encrypt for remote, is good enough, and will get most developers over that hurdle. I would welcome an improvement to the docs for this.

As I understand the current configurations, they are for local development only, and nowhere else, so only a bare minimum configuration is provided. The rest is "left to the developer" 😏 .

[Querela]

We just reviewed and tested the configuration in #1348 and it worked. It is possible that no one logged out and logged back in, but I want to make sure that the proposed fix will in fact work. I also want all examples to be consistent with one another.

# https://github.com/plone/documentation/pull/1348/files
rewrite ^/(\+\+api\+\+\/?)+($|/.*) /VirtualHostBase/http/$server_name/Plone/++api++/VirtualHostRoot/$2 break;

I only did some quick testing for the #1348 pull request. E.g. log in, create page(s), publish, log out, check etc. Clear cache and check next deployment example. (Remove any previous container, volume, ...) My issue before was that I could not log in with the wrong nginx configuration, so that was something I definitely checked. 😄 I was not aware of the other docs. When I find time next week I might be able to check it, too.

[stevepiercy]

@Querela Thanks for confirming. That's sufficient testing without writing tests.

I prefer the more simple regex that @avoinea provided. If you (or anyone else) can test that out, and you feel it is sufficient, then we can switch to using it, with the minor tweak of using a server environment variable $servername instead of a hard-coded myservername.org. A second verification is a good thing.

Also to make the docs consistent and easier to maintain, I would pull out anything that is repeated into a literalinclude. That also reduces the potential of copy-pasta mistakes.

[Querela]

I just saw that the examples bundled with plone/plone-frontend are not using a capture group and working with $1. See default.conf in https://github.com/plone/plone-frontend/tree/15.x/examples (same for 16.x branch) Those should probably also be made similar to the docs or vice-versa.

[stevepiercy]

I think you mean they are using a capture group.

https://github.com/plone/plone-frontend/blob/2e3f81b8982f2a9304c542372a3a69c5e1696e46/examples/webserver-volto-plone-postgres/default.conf#L14

The () defines a capture group, and is referenced by $1 or $# where # refers to the 1-based index of the number of captures.

Anyway, good catch! I think it would be good to make all these examples consistent. I created a new issue to track it there, and cross-referenced it here: https://github.com/plone/plone-frontend/issues/25