Nginx Gateway for Matrix

[lfom]

Hi, there. Thanks for providing a FLOSS alternative for notifications. I have successfully installed common-proxies + gotify for my Matrix homeserver, but I could not use nginx as a reverse proxy as explained here:

https://github.com/UnifiedPush/contrib/blob/main/gateways/matrix.md

It also took me a while to configure common-proxies but it is working fine after I found this:

https://github.com/UnifiedPush/common-proxies/blob/main/docs/reverse_proxy.md

I even tried to configure the reverse proxy like showed in the article below without success:

https://blog.tastytea.de/posts/set-up-unifiedpush-with-matrix-support-in-gentoo/

I still got the same error. When troubleshooting notifications in SchildChat for Android, the push notification fails with a 502 error. It works fine with common-proxies setup tho.

I wonder if something has to be updated regarding using nginx with the newer versions of Synapse?

Regards

[karmanyaahm]

Can you paste the full (including gateway and other UnifiedPush stuff) nginx config you used with the nginx matrix gateway? Also, do you still have it running (for debugging)?

[lfom]

@karmanyaahm Thank you for the quick reply. I can re-enable it using Nginx to test, since I think it would be better regarding maintenance than using common-proxies (because it needs to be updated manually).

This is the configuration that is working now with common-proxies:

server {

    server_name push.HOMESERVER;

    # access_log /var/log/nginx/push.example.org_log main;
    error_log /var/log/nginx/push.HOMESERVER_log warn;

    # this sends traffic to common-proxies
    location ~ ^/(FCM|UP|_matrix) { 
        proxy_pass      http://127.0.0.1:5000;
    }

    location / {
        proxy_pass         http://127.0.0.1:1245;
        proxy_http_version 1.1;

        # Ensuring it can use websockets
        proxy_set_header   Upgrade $http_upgrade;
        proxy_set_header   Connection "upgrade";
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto http;
        proxy_redirect     http:// $scheme://;

        # The proxy must preserve the host because gotify verifies the host with the origin
        # for WebSocket connections
        proxy_set_header   Host $http_host;

        proxy_connect_timeout   1m;
        proxy_send_timeout      1m;
        proxy_read_timeout      1m;
    }

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/push.HOMESERVER/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/push.HOMESERVER/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = push.HOMESERVER) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

        listen 80;
        listen [::]:80;

        server_name push.HOMESERVER;
    return 404; # managed by Certbot

}

This is what did not work with just nginx:

server {

    server_name push.HOMESERVER;

    # access_log /var/log/nginx/push.example.org_log main;
    error_log /var/log/nginx/push.HOMESERVER_log warn;

    location /UP {
        access_by_lua_block{
            local json=require("cjson")
            ngx.req.read_body()
            local req = ngx.req.get_body_data()
            local newreq = { ["message"] = req }
            local body = json.encode(newreq)
            ngx.req.set_body_data(body)
        }

        if ($request_method = GET ) { 
            return 200 '{"unifiedpush":{"version":1}}';
        }

        proxy_set_header    Content-Type application/json;
        proxy_pass      http://127.0.0.1:1245/message;
        proxy_set_header    Host $host;

        # Force https
        if ($scheme = http) {
            rewrite ^ https://$server_name$request_uri? permanent;
        }
    }

    resolver 127.0.0.1;
    location /_matrix/push/v1/notify {
        mirror /mx-mirror;
        mirror_request_body on; 
        if ($request_method = GET ) { 
            return 200 '{"unifiedpush":{"gateway":"matrix"}}';
        }
        proxy_pass https://$server_name/mx-upstream;
    }
    location /mx-upstream {
        return 200 '{"rejected":[]}';
    }
    location /mx-mirror {
        internal;
        set $target ''; 
        rewrite_by_lua_block {
            local cjson = require("cjson")
            ngx.req.read_body()
            local body = ngx.req.get_body_data()
            local parsedBody = cjson.decode(body)
            local accepted = "https://push.HOMESERVER/"
            ngx.var.target = parsedBody["notification"]["devices"][1]["pushkey"]
            ngx.req.set_body_data(body)
            if(string.sub(ngx.var.target,1,string.len(accepted))~=accepted) then ngx.var.target="http://0.0.0.0/"
            end 
        }
        proxy_set_header Content-Type application/json;
        proxy_set_header Host $host;
        proxy_pass $target;
    }

    location / {
        proxy_pass         http://127.0.0.1:1245;
        proxy_http_version 1.1;

        # Ensuring it can use websockets
        proxy_set_header   Upgrade $http_upgrade;
        proxy_set_header   Connection "upgrade";
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto http;
        proxy_redirect     http:// $scheme://;

        # The proxy must preserve the host because gotify verifies the host with the origin
        # for WebSocket connections
        proxy_set_header   Host $http_host;

        proxy_connect_timeout   1m;
        proxy_send_timeout      1m;
        proxy_read_timeout      1m;
    }

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/push.HOMESERVER/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/push.HOMESERVER/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = push.HOMESERVER) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

        listen 80;
        listen [::]:80;

        server_name push.HOMESERVER;
    return 404; # managed by Certbot

}

It works with SchildiChat gateway, but not with my own (this is where it shows the 502 error, I guess). Also, while using common-proxies, push.HOMESERVER/_matrix/push/v1/notify shows:

{"gateway":"matrix","unifiedpush":{"gateway":"matrix"}}

while the nginx reverse proxy prompts to download the JSON.

[karmanyaahm]

On 502 errors nginx logs to /var/log/nginx/error.log Can you post the relevant lines from that as well? It probably holds more clues as to where the error is occurring, because the config looks fine overall

It works with SchildiChat gateway, but not with my own

Hmm... that's kinda weird. The error is probably in the nginx gateway part then :thinking:

Edit: I just saw you have a custom error_log path, /var/log/nginx/push.HOMESERVER_log would be the one to post then

[lfom]

I disabled common-proxies service, stopped nginx, replaced the push.HOMESERVER nginx config with the version above, renamed the current push.HOMESERVER_log (so only new messages would be saved) and then started nginx again. Troubleshooting Notifications from SchildiChat still shows an error for Test Push:

java.lang.RunTimeExeception: HTTP 502

This is the related nginx log:

2021/12/13 17:38:52 [warn] 3766#3766: *121 an upstream response is buffered to a temporary file /var/lib/nginx/proxy/1/00/0000000001 while reading upstream, client: MYIP, server: push.HOMESERVER, request: "GET /static/js/2.038195e1.chunk.js HTTP/1.1", upstream: "http://127.0.0.1:1245/static/js/2.038195e1.chunk.js", host: "push.HOMESERVER", referrer: "https://push.HOMESERVER/"
2021/12/13 17:38:57 [warn] 3765#3765: *120 an upstream response is buffered to a temporary file /var/lib/nginx/proxy/2/00/0000000002 while reading upstream, client: MYIP, server: push.HOMESERVER, request: "GET /static/js/2.038195e1.chunk.js HTTP/1.1", upstream: "http://127.0.0.1:1245/static/js/2.038195e1.chunk.js", host: "push.HOMESERVER", referrer: "https://push.HOMESERVER/"
2021/12/13 17:40:18 [error] 3763#3763: *155 push.HOMESERVER could not be resolved (110: Operation timed out), client: MYIP, server: push.HOMESERVER, request: "POST /_matrix/push/v1/notify HTTP/1.1", host: "push.HOMESERVER"
2021/12/13 17:40:18 [error] 3763#3763: *155 push.HOMESERVER could not be resolved (110: Operation timed out) while sending to client, client: MYIP, server: push.HOMESERVER, request: "POST /_matrix/push/v1/notify HTTP/1.1", subrequest: "/mx-mirror", host: "push.HOMESERVER"

Thank you for the help. Please feel free to ask for any test or info.

[lfom]

I got it working: the problem was the resolver IP. I run resolvectl and the only DNS server in the machine was 169.254.169.254, and setting it as the resolver in the nginx configuration file (I do not want to use an external server) fixed the 502 error:

    server_name push.HOMESERVER;
    resolver 169.254.169.254;

Thank you again.