push server is not a trusted proxy

[Crocmagnon]

I guess this one is related to my config but I can't seem to make it work. I'm trying to setup notify_push using php occ notify_push:setup but I'm always getting a push server is not a trusted proxy no matter what I put into the trusted_proxies config.

Here's the relevant config:

  'trusted_proxies' =>
  array (
    0 => 'nginx-proxy-manager_app_1',
    1 => '172.18.0.8',
    2 => '172.0.0.1/8',
    3 => '127.0.0.1',
  ),

At first I only had the first value and tried adding the other ones but no luck. Here's the exact error message I get:

push server is not a trusted proxy, please add '172.18.0.8' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.

My setup involves docker and nginx proxy manager.

Here's the nginx location config I'm using:

  location /push/ {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto  $scheme;
    proxy_set_header X-Forwarded-For    $remote_addr;
    proxy_pass       http://nextcloud_notify_push_1:80/;
        proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
  }

[icewind1991]

Your nginx config overwrites the existing X-Forwarded-For header instead of appending to it.

Try replacing it with

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

[Crocmagnon]

Side note: that's not something easily overridable with nginx-proxy-manager.

I still have the same error though.

[icewind1991]

You can also try to bypass nginx for the push server -> nextcloud traffic by setting the NEXTCLOUD_URL environment variable to point directly to the docker image running your nextcloud web server.

[Dougley]

I have the same issue and I already tried setting NEXTCLOUD_URL to point directly at the webserver running nextcloud instead of passing though the reverse proxy first.

nginx config:

location /push/ {
  proxy_pass http://nextcloud-hpbf:80/;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "Upgrade";
  proxy_set_header Host $host;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

config.php

'trusted_proxies' => array (
  0 => 10.42.0.0/16',
),

Error returned: push server is not a trusted proxy, please add '10.42.0.20' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.

[icewind1991]

Have you changed the 'forwarded_for_headers' option in your config.php by any chance?

For debugging you can also try doing the request the push server makes manually

curl -H 'x-forwarded-for: 1.2.3.4' https://<nextcloud>/index.php/apps/notify_push/test/remote

which should return 1.2.3.4 if nextcloud thinks the request is coming from a trusted proxy

[Dougley]

Have you changed the 'forwarded_for_headers' option in your config.php by any chance?

I have, removing my custom headers fixed it. Thanks!

[t-aus-m]

Have you changed the 'forwarded_for_headers' option in your config.php by any chance?

I have not and my issue is the same nonetheless

[karbowiak]

Same issue here sadly :(

I run it inside a container from linuxserver.io - and try and set it up using: occ notify_push:setup https://domain.tld/push

and get:

✓ redis is configured
✓ push server is receiving redis messages
✓ push server can load mount info from database
✓ push server can connect to the Nextcloud server
🗴 push server is not a trusted proxy, please add '162.158.134.58' to the list of trusted proxies or  configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.

and 162.158.134.58 is marked as a trusted proxy in the config :(

[Crocmagnon]

You can also try to bypass nginx for the push server -> nextcloud traffic by setting the NEXTCLOUD_URL environment variable to point directly to the docker image running your nextcloud web server.

I tried that but got the same error message with a different IP address. Adding this IP address to the trusted_proxies list in the config didn't help.

Have you changed the 'forwarded_for_headers' option in your config.php by any chance?

I haven't.

I also tried disabling the rewrite IP module from apache as per https://github.com/nextcloud/docker#using-the-apache-image-behind-a-reverse-proxy-and-auto-configure-server-host-and-protocol but it didn't help either (no change in error message)

[noci2012]

for nginx also check out these configuration items: http://nginx.org/en/docs/http/ngx_http_realip_module.html

[Crocmagnon]

I'm using the nextcloud-apache image behind nginx-proxy-manager, maybe apache overrides the X-Forwarded-For header before passing it to nextcloud?

[raimund-schluessler]

For me notify_push complains that the public IP of my server is not a trusted proxy. But I cannot really add the IP as trusted proxy, since it changes regularly. And even when I added it to the trusted proxies, the error persists.

Setting

    [entryPoints.websecure.forwardedHeaders]
      insecure = true # this is not a good idea
      #trustedIPs = ["172.33.0.0/16"] # this does not help

in the static traefik configuration "fixes" the issue, but I don't want to run the server in such an insecure way.

[icewind1991]

For me notify_push complains that the public IP of my server is not a trusted proxy

You should be able to work around that by setting the NEXTCLOUD_URL environment variable to an internal url for your nextcloud instance so the notify_push->nextcloud traffic doesn't use your public domain/ip.

[PriceChild]

@icewind1991 Are you suggesting that is set in e.g. the systemd service file for the notify_push daemon?

If so, I'm not sure whether that's a good idea?

test output: [2021-02-24 16:09:32.510217 +00:00] INFO [notify_push] src/main.rs:89: shutdown signal received, shutting down
             ✓ redis is configured
             🗴 using unencrypted https for push server is strongly discouraged
             🗴 push server url is set to localhost, the push server will not be reachable from other machines
             ✓ push server is receiving redis messages
             ✓ push server can load mount info from database
             ✓ push server can connect to the Nextcloud server
             🗴 push server is not a trusted proxy, please add '10.a.b.c' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.

[icewind1991]

Are you suggesting that is set in e.g. the systemd service file for the notify_push daemon?

Yes, depending on the setup something like http://localhost if the push server and nextcloud server are running on the same machine should work, or for docker based setups the name of the nextcloud container.

You will still need to add the ip of the push server to the list of trusted proxies, but it will prevent any existing reverse proxies in the setup from messing with things.

[raimund-schluessler]

For me notify_push complains that the public IP of my server is not a trusted proxy

You should be able to work around that by setting the NEXTCLOUD_URL environment variable to an internal url for your nextcloud instance so the notify_push->nextcloud traffic doesn't use your public domain/ip.

Setting NEXTCLOUD_URL=http://app/ (with app as the docker container name) leads to error sending request for url (http://app/index.php/apps/notify_push/test/version): error trying to connect: tcp connect error: Connection refused (os error 111). Setting NEXTCLOUD_URL=https://nextcloud.domain.com/ works in that regard, but leads to the trusted proxy error.

[Monviech]

Hello, I have the same problems. I followed this issue for a while and tried all things out suggested in it, but I just can't get it to work. Hope the information I post can help.

Enviroment: Plesk Obsidian / Ubuntu 20.04 LTS / Nextcloud 21 installed with Nextcloud App from Plesk Store

Notify Push Error:

sudo -u WEBSPACEUSER /opt/plesk/php/7.4/bin/php occ notify_push:setup https://cloud.domain.tld/push
✓ redis is configured
✓ push server is receiving redis messages
✓ push server can load mount info from database
✓ push server can connect to the Nextcloud server
x push server is not a trusted proxy, please add '10.0.0.203' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.

notify_push=trace Log:

sudo -u WEBSPACEUSER env LOG=notify_push=trace ./apps/notify_push/bin/x86_64/notify_push ./config/config.php

[2021-02-24 20:28:51.727137 +01:00] TRACE [notify_push] src/main.rs:41: Running
with config: Config { database: AnyConnectOptions(MySql(MySqlConnectOptions { host: "127.0.0.1", port: 3306, socket: None, username: "XXX", password: Some("XXX"), database: Some("XXX"), ssl_mode: Preferred, ssl_ca: None, statement_cache_capacity: 100, charset: "utf8mb4", collation: None, log_settings: LogSettings { statements_level: Info, slow_statements_level: Warn,
slow_statements_duration: 1s } })), database_prefix: "oc_", redis: ConnectionInfo { addr: Tcp("127.0.0.1", 6379), db: 0, username: None, passwd: Some("XXX") }, nextcloud_url: "https://cloud.domain.tld" }

[2021-02-24 20:28:51.727850 +01:00] DEBUG [notify_push::storage_mapping] src/storage_mapping.rs:96: querying storage mapping for 1
[2021-02-24 20:28:52.038708 +01:00] TRACE [notify_push] src/main.rs:68: Listening on port 80
The application panicked (crashed).
Message:  error binding to 0.0.0.0:80: error creating server listener: Permission denied (os error 13)
Location: /home/robin/.cargo/registry/src/github.com-1ecc6299db9ec823/warp-0.3.0/src/server.rs:283
Backtrace omitted.
Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Curl Test:

curl -H 'x-forwarded-for: 1.2.3.4' https://cloud.domain.tld/index.php/apps/notify_push/test/remote
Answer: 10.0.0.203

Systemd Setup:

[Unit] 
Description = Push daemon for Nextcloud clients cloud.domain.tld 

[Service] 

Environment = PORT=7867 NEXTCLOUD_URL=https://cloud.domain.tld
ExecStart = /var/www/vhosts/domain.tld/cloud.domain.tld/nextcloud/apps/notify_push/bin/x86_64/notify_push /var/www/vhosts/domain.tld/cloud.domain.tld/nextcloud/config/config.php

User=WEBSPACEUSER
[Install] 

WantedBy = multi-user.target 

Systemd Status:

● notify_push_XXX.service - Push daemon for Nextcloud clients cloud.domain.tld 
     Loaded: loaded
(/etc/systemd/system/notify_push_XXX.service; enabled; vendor preset: enabled) 
     Active: active (running) since Wed 2021-02-24 20:32:25 CET; 5s ago 
   Main PID: 347594 (notify_push) 
      Tasks: 8 (limit: 28696) 
     Memory: 1.9M 
     CGroup: /system.slice/notify_push_XXX.service 

└─347594
/var/www/vhosts/domain.tld/cloud.domain.tld/nextcloud/apps/notify_push/bin/x86_64/notify_push /var/www/vhost>

Nextcloud config.php:

<?php 
$CONFIG = array ( 
  'passwordsalt' => 'XXX', 
  'secret' => 'XXX', 
  'trusted_domains' => 
  array ( 
    0 => 'cloud.domain.tld', 
    1 => '127.0.0.1', 
    2 => '10.0.0.203', 
  ), 
  'datadirectory' => '/var/www/vhosts/domain.tld/.nextcloud/data/XXX', 
  'dbtype' => 'mysql', 
  'version' => '21.0.0.18', 
'overwrite.cli.url' => 'https://cloud.domain.tld', 
  'dbname' => 'XXX', 
  'dbhost' => '127.0.0.1:3306', 
  'dbport' => '',
  'dbtableprefix' => 'oc_', 
  'mysql.utf8mb4' => true,
  'dbuser' => 'XXX', 
 'dbpassword' => 'XXX', 
  'installed' => true, 
  'instanceid' => 'XXX', 
'ldapIgnoreNamingRules' => false, 
'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory', 
  'defaultapp' => 'files', 
  'maintenance' => false, 
  'theme' => '', 
  'loglevel' => 2, 
'app_install_overwrite' => 
  array ( 
    0 => 'occweb', 
  ), 
  'memcache.local'
=> '\\OC\\Memcache\\Redis', 
  'redis' => 
  array ( 
    'host' => '127.0.0.1',
    'port' => 6379,
    'password' =>'XXX', 
  ), 
'updater.release.channel' => 'beta', 
  'updater.secret' => 'XXX', 
);

Nginx Config:

location /push/ { 
    proxy_pass http://127.0.0.1:7867/; 
    proxy_http_version 1.1; 
    proxy_set_header Upgrade $http_upgrade; 
    proxy_set_header Connection "Upgrade"; 
    proxy_set_header Host $host; 
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
}

[alerque]

Having the same issue getting setup with a system using an Apache proxy. This could use better documentation!

[icewind1991]

@Monviech you've added '10.0.0.203' to the trusted_domains, not the trusted_proxies

[Monviech]

@icewind1991 Thank you... that did it. I was blind. Now it works.

[alerque]

Ditto here, that was my problem too. It's really confusing that most of the time settings are mentioned it isn't mentioned whether they are PHP settings or Apache VHost settings or Nextcloud settings or what, and each of those groupings often have more than one place settings can go. "Not in trusted proxies" doesn't mean much when you don't know who isn't trusting who.

[icewind1991]

The error message will link to the nextcloud docs with the next version which will hopefully help some people.

[raimund-schluessler]

Puh, this was a long yourney, but I finally got it working without any insecure configuration hack using traefik as reverse proxy! 🎉 The trick for me was to add the nextcloud container host name app to the trusted_domains in the nextcloud config.php and to set the - NEXTCLOUD_URL=http://app environment variable for the notify_push container.

I will cleanup my configuration and post it in the original nextcloud/docker thread: https://github.com/nextcloud/docker/issues/1422.

Also the nextcloud network needs to be in the trusted proxies list, for which I gave a static IP to the nextcloud network.

[andreaslutsch]

Is a reverse proxy through nginx needed if the nextcloud instance is behind a cloudflare proxy? With cloudflare proxy, the IP always changes and can't set a fixed one as trusted.

[icewind1991]

In that case it's probably best to set the NEXTCLOUD_URL to an internal url for your instance and bypass the reverse proxy. (Don't forget to ensure that this internal url is set as a trusted_domain in the nextcloud config)

[concave-sphere]

Still not working for me.

Background: I'm running nextcloud in a Docker container alongside containers for cron, the proxy, the push app, and some other unrelated stuff. The proxy is configured to use FPM with nextcloud, and to only accept connections via HTTPS. Nextcloud works fine, aside from notify_push not being configurable.

I'm running the setup command as:

./occ notify_push:setup https://public.host.name.com/push

As noted above, I literally copied the nginx configuration block from README.md into my config file. I've been poking around log files, and here's what I have so far:

(1) x_forwarded_for is working, but the source IP is stubbornly coming through as 172.23.0.1. This is the gateway IP of a different Docker network, one that the Nextcloud container isn't even connected to. This IP doesn't show up on a request from outside the Docker network, but it does show up on any request I issue between containers.

Example:

site="_" server="public.host.name.com" dest_port="4443" dest_ip="172.23.0.2" src="172.23.0.1" src_ip="172.23.0.1" proxy_host="nextcloud_notify_push:7867" proxy_port="7867" proxy_add_x_forwarded_for="172.23.0.1" uri_query="-" uri_path="/push/test/remote/1.2.3.4"

(2) I definitely have the IP address of the proxy listed in my config.php:

$ grep -A5 trusted_proxies config/config.php
  'trusted_proxies' =>
  array (
    0 => '172.24.0.99',
    1 => '172.23.0.1',
    2 => '172.24.0.3',
  ),

Here, 0 is the static IP assigned to the proxy, 1 is the gateway address that shows up in the "not a trusted proxy" message (even with this config), and 2 is the IP address assigned to the push server (based on the comment that "You will still need the ip address of the push server added as trusted proxy"). This happens both with ./occ notify_proxy:setup https://public.host.name.com/push and if I use the name of the host within the Docker network, ./occ notify_proxy:setup https://proxy:4443/push.

(3) As far as I can tell, nextcloud is serving requests when I run notify_push:setup, and they're succeeding. With no other traffic on the server, I can run the setup command and see the following lines printed each time in my Docker logs. Note that the printed address is the 172.24.0.99 address that I expect for the proxy.

172.24.0.99 -  27/Feb/2021:03:36:45 +0000 "GET /index.php" 200
172.24.0.99 -  27/Feb/2021:03:36:45 +0000 "GET /index.php" 200

I also have lots of lines like this this in nextcloud.log:

{"reqId":"HFXraTevMD3fHFAZEfa2","level":0,"time":"2021-02-26T19:37:22-08:00","remoteAddr":"172.23.0.1","user":"--","app":"contacts","method":"GET","url":"/nextcloud/index.php/apps/notify_push/test/cookie","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"21.0.0.18"}
{"reqId":"HFXraTevMD3fHFAZEfa2","level":0,"time":"2021-02-26T19:37:22-08:00","remoteAddr":"172.23.0.1","user":"--","app":"documentserver_community","method":"GET","url":"/nextcloud/index.php/apps/notify_push/test/cookie","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"21.0.0.18"}
{"reqId":"HFXraTevMD3fHFAZEfa2","level":0,"time":"2021-02-26T19:37:22-08:00","remoteAddr":"172.23.0.1","user":"--","app":"files_external","method":"GET","url":"/nextcloud/index.php/apps/notify_push/test/cookie","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"21.0.0.18"}
{"reqId":"HFXraTevMD3fHFAZEfa2","level":0,"time":"2021-02-26T19:37:22-08:00","remoteAddr":"172.23.0.1","user":"--","app":"files_sharing","method":"GET","url":"/nextcloud/index.php/apps/notify_push/test/cookie","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"21.0.0.18"}
{"reqId":"eJtHPF9nq19OhSL7As9B","level":0,"time":"2021-02-26T19:37:22-08:00","remoteAddr":"172.23.0.1","user":"--","app":"contacts","method":"GET","url":"/nextcloud/index.php/apps/notify_push/test/remote","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"21.0.0.18"}
{"reqId":"eJtHPF9nq19OhSL7As9B","level":0,"time":"2021-02-26T19:37:22-08:00","remoteAddr":"172.23.0.1","user":"--","app":"documentserver_community","method":"GET","url":"/nextcloud/index.php/apps/notify_push/test/remote","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"21.0.0.18"}
{"reqId":"eJtHPF9nq19OhSL7As9B","level":0,"time":"2021-02-26T19:37:22-08:00","remoteAddr":"172.23.0.1","user":"--","app":"files_external","method":"GET","url":"/nextcloud/index.php/apps/notify_push/test/remote","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"21.0.0.18"}
{"reqId":"eJtHPF9nq19OhSL7As9B","level":0,"time":"2021-02-26T19:37:22-08:00","remoteAddr":"172.23.0.1","user":"--","app":"files_sharing","method":"GET","url":"/nextcloud/index.php/apps/notify_push/test/remote","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"21.0.0.18"}

which shows that the 172.23.0.1 remote address is making it through to the backend and fetching a URL having to do with notify_push.

So, to summarize:

1. I still don't understand where the 172.23.0.1 address is coming from.
2. But it looks like it's the remote address, not the proxy address.
3. And Nextcloud is accepting the requests and returning 200 OK.
4. So where is this error about "not a trusted proxy" coming from?

My best guess at this point is that something else is going wrong, and occ notify_push:setup is hiding+misdiagnosing the actual error, and printing the "not a trusted proxy" message instead.

[concave-sphere]

After some experimentation, I managed to get debugging output from occ when it sends this request. This is the output of the client request with debug=true, as well as the response object as stringified by print_r.

client request:

* Expire in 0 ms for 6 (transfer 0x55b1276ad180)              
* Expire in 30000 ms for 8 (transfer 0x55b1276ad180)    
* Expire in 150000 ms for 2 (transfer 0x55b1276ad180)                                       
* Found bundle for host public.domain.name.com: 0x55b12764c480 [can pipeline]
* Re-using existing connection! (#0) with host public.domain.name.com
* Connected to public.domain.name.com (192.168.254.19) port 443 (#0)
* Expire in 0 ms for 6 (transfer 0x55b1276ad180)
> GET /push/test/remote/1.2.3.4 HTTP/1.1                      
Host: public.domain.name.com                                                   
User-Agent: Nextcloud Server Crawler          
Accept-Encoding: gzip                                                           

< HTTP/1.1 200 OK                                                 
< Server: nginx/1.19.7                                            
< Date: Sat, 27 Feb 2021 03:55:41 GMT                             
< Content-Type: text/plain; charset=utf-8                             
< Content-Length: 10                                                        
< Connection: keep-alive                         
< Referrer-Policy: no-referrer
< X-Content-Type-Options: nosniff        
< X-Download-Options: noopen
< X-Frame-Options: SAMEORIGIN
< X-Permitted-Cross-Domain-Policies: none     
< X-Robots-Tag: none     
< X-XSS-Protection: 1; mode=block              
<                        
* Connection #0 to host public.domain.name.com left intact

and the contents of the response. Note that the response code is 200 -- I believe this means the request succeeded. While I'm not a PHP expert, it looks to me like the code that produces the "not a trusted proxy" message is taking whatever string it receives in the body of the request, and if it isn't equal to 1.2.3.4, claiming that the proxy wasn't trusted.

OC\Http\Client\Response Object                                  
(
    [response:OC\Http\Client\Response:private] => GuzzleHttp\Psr7\Response Object
        (                
            [reasonPhrase:GuzzleHttp\Psr7\Response:private] => OK
            [statusCode:GuzzleHttp\Psr7\Response:private] => 200
            [headers:GuzzleHttp\Psr7\Response:private] => Array
                (                      
                    [Server] => Array
                        (
                            [0] => nginx/1.19.7
                        )

                    [Date] => Array
                        (
                            [0] => Sat, 27 Feb 2021 03:55:41 GMT
                        )

                    [Content-Type] => Array
                        (
                            [0] => text/plain; charset=utf-8
                        )

                    [Content-Length] => Array
                        (
                            [0] => 10
                        )

                    [Connection] => Array
                        (
                            [0] => keep-alive
                        )

                    [Referrer-Policy] => Array
                        (
                            [0] => no-referrer
                        )

                    [X-Content-Type-Options] => Array
                        (
                            [0] => nosniff
                        )

                    [X-Download-Options] => Array
                        (
                            [0] => noopen
                        )

                    [X-Frame-Options] => Array
                        (
                            [0] => SAMEORIGIN
                        )

                    [X-Permitted-Cross-Domain-Policies] => Array
                        (
                            [0] => none
                        )

                    [X-Robots-Tag] => Array
                        (
                            [0] => none
                        )

                    [X-XSS-Protection] => Array
                        (
                            [0] => 1; mode=block
                        )

                )

            [headerNames:GuzzleHttp\Psr7\Response:private] => Array
                (
                    [server] => Server
                    [date] => Date
                    [content-type] => Content-Type
                    [content-length] => Content-Length
                    [connection] => Connection
                    [referrer-policy] => Referrer-Policy
                    [x-content-type-options] => X-Content-Type-Options
                    [x-download-options] => X-Download-Options
                    [x-frame-options] => X-Frame-Options
                    [x-permitted-cross-domain-policies] => X-Permitted-Cross-Domain-Policies
                    [x-robots-tag] => X-Robots-Tag
                    [x-xss-protection] => X-XSS-Protection
                )

            [protocol:GuzzleHttp\Psr7\Response:private] => 1.1
            [stream:GuzzleHttp\Psr7\Response:private] => GuzzleHttp\Psr7\Stream Object
                (
                    [stream:GuzzleHttp\Psr7\Stream:private] => Resource id #1446
                    [size:GuzzleHttp\Psr7\Stream:private] =>
                    [seekable:GuzzleHttp\Psr7\Stream:private] => 1
                    [readable:GuzzleHttp\Psr7\Stream:private] => 1
                    [writable:GuzzleHttp\Psr7\Stream:private] => 1
                    [uri:GuzzleHttp\Psr7\Stream:private] => php://temp
                    [customMetadata:GuzzleHttp\Psr7\Stream:private] => Array
                        (
                        )

                )

        )

    [stream:OC\Http\Client\Response:private] =>
)

[concave-sphere]

Additional findings:

1. If I run curl http://public.host.name.com/push/test/remote/1.2.3.4, I get back 172.23.0.1. This is what I expected given the above. It happens from both a container, the Docker host, and the remote host.
2. If I enable debugging (via occ notify_push:log debug), I get this message:

[2021-02-27 04:09:03.944873 +00:00] DEBUG [notify_push] src/lib.rs:282: got remote 172.23.0.1 when trying to set remote 1.2.3.4 

Haven't tracked down yet where this comes from in the server. I'm guessing it may be the root problem.

[concave-sphere]

This seems to be what happens. Apologies if I got anything wrong -- I'm not familiar with the warp library that's in use here.

1. The test/remote URL is served by this Rust code. It invokes test_set_remote on the IP address that's the last component of the URL, and if it succeeds, returns the result.
2. test_set_remote performs an HTTP GET on self.base_url.join("index.php/apps/notify_push/test/remote"). At least I assume that's what reqwest::Client::get does. It includes an x-forwarded-for header set to the addr that it received in its URL (1.2.3.4 in this case). It then waits for the response and tries to ... parse it as an IP, I think?
3. Some PHP thing then returns 172.23.0.1.

I can't figure out how the PHP part of the code is put together, but my guess is that the notify_push server is expecting the recipient to echo back what it got in x-forwarded-for. However, this is not happening: either the proxy is overwriting it, or Nextcloud isn't able to interpret it. My problem has nothing to do with whether the proxy is trusted or not, that is a total red herring.

I see that an early comment in the thread pointed this out. I assumed this wasn't my problem because I do have this line in my nginx config, as suggested:

                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

I also (after some configuration & fiddling with logs) can see logs showing that nginx at least claims to be appending to X-Forwarded-For, as instructed:

site="_" server="public.host.name.com" dest_port="4443" dest_ip="172.23.0.2" src="172.23.0.1" src_ip="172.23.0.1" http_x_forwarded_for="1.2.3.4" proxy_host="-" proxy_port="-"
proxy_add_x_forwarded_for="1.2.3.4, 172.23.0.1" uri_query="-" uri_path="/nextcloud/index.php"

I believe that this shows that 1.2.3.4 was the original value of the header, and the new value is 1.2.3.4, 172.23.0.1. I believe this is the expected behavior of $proxy_add_x_forwarded_for.

My next goal is to get Nextcloud to show me the value of the header that it sees. I can see that remoteAddr in the Nextcloud logs is set to 172.23.0.1, but I don't know if that's because the header is never reaching Nextcloud, or if the comma separated list prevents it from being interpreted as expected.

[benhartwich]

I´m running nextcloud in a very default debian 10 php7.4 / nginx setup and I don´t get it working. I have tried trusted proxy, set up an own subdomain reverse proxy for the push server. I´ve tried the config parameter overwritehost and overwriteprotocoll. It doesn´t simply work at all. I´m always getting:

🗴 push server is not a trusted proxy, please add '127.0.0.1' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.

[concave-sphere]

This is surprising.

curl -H 'X-Forwarded-For: 1.2.3.4' https://public.name.of.host.com/nextcloud/index.php/apps/notify_push/test/remote

and on the Nextcloud server (adding %{HTTP_X_FORWARDED_FOR} to php-fpm's www.conf):

172.24.0.99 -  27/Feb/2021:05:50:12 +0000 "GET /index.php" 200 /var/www/html/index.php 80.728 8192 86.71% X_FORWARDED_FOR=1.2.3.4

It's surprising to me because after my earlier debugging and after reading earlier parts of this thread, I was sure that I'd either see 172.23.0.1, or 1.2.3.4, 172.23.0.1 there. Instead, I have ... exactly the value that I think I "should" have. Yet the curl command still outputs 172.23.0.1. That's not the peer address (the front of the log line shows the peer is 172.24.0.99); it can only come from the X-Forwarded-For header.

Looking again at the PHP code, I have a strong suspicion that this line generates the remote address to return. As best I can tell, that calls over here. And the intent of that code seems to be to return the first IP address in X-Forwarded-For that is permitted by FILTER_VALIDATE_IP. Just in case 1.2.3.4 was getting rejected by the filter, I changed the address to 192.168.0.1 and 8.8.8.8, but I got the same behavior with both of them.

At this point, I think I may be stumped. I hope some of my flailing around in public was useful to someone, anyway.

[icewind1991]

Thanks for the research @concave-sphere, your understanding of the code is correct.

9dc0ea1 adds a bit more diagnostics, I'm not going to push a release with it now but you can apply it manually to see if it helps with debugging has been released with 0.1.6

[Marcwa19197]

I also had an long night fighting with this issue. For me there were some problems regarding proxy_http2 and SELinux. (Running Fedora 33) And last but not least, my config.php included the "trusted_proxies" array two times, which i did not noticed... I wrote a few steps which guide me to success in https://github.com/nextcloud/notify_push/issues/44

[benhartwich]

I also had an long night fighting with this issue. For me there were some problems regarding proxy_http2 and SELinux. (Running Fedora 33) And last but not least, my config.php included the "trusted_proxies" array two times, which i did not noticed... I wrote a few steps which guide me to success in #44

Doesn´t seem to be the issue at my server. Can anyone name, what the NEXTCLOUD_URL environment variable is? I´m not using any docker setup, so I´m not sure, if I can use it?

[much-doge]

@andreaslutsch Push server works behind Cloudflare, just make sure to add lists of Cloudflare IPs to nextcloud trusted proxy

The IPs are listed here: https://www.cloudflare.com/ips/

[ghost]

I'm not Sure, but I also got a Strange Error Message. The IP Adress is listed in the Nextcloud config.php in the Trusted Proxy Array.

my Console Output

user_n@server:~$ sudo -u www-data php /var/www/nextcloud/occ notify_push:setup https://cloud.abc.de/push`

✓ redis is configured ✓ push server is receiving redis messages ✓ push server can load mount info from database ✓ push server can connect to the Nextcloud server 🗴 push server is not a trusted proxy, please add '11.22.333.444' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server. See https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html#defining-trusted-proxies for how to set trusted proxies. The following trusted proxies are currently configured: 127.0.0.1, ::1 The following x-forwarded-for header was received by Nextcloud: 1.2.3.4 from the following remote: 11.22.333.444

If you're having issues getting the trusted proxy setup working, you can try bypassing any existing reverse proxy in your setup by setting the NEXTCLOUD_URL environment variable to point directly to the internal Nextcloud webserver url (You will still need the ip address of the push server added as trusted proxy)

my notify_push Service Unit

[Unit] Description = Push daemon for Nextcloud clients After = nginx.service php7.4-fpm.service postgresql.service redis.service smbd.service

[Service] Environment = SOCKET_PATH=/var/run/notify_push/notify_push.sock

ExecStart = /var/www/nextcloud/apps/notify_push/bin/aarch64/notify_push /var/www/nextcloud/config/config.php

Environment = DATABASE_URL=postgres://nextcloud:"password"@localhost:5432/nextcloud_db?sslmode=disable Environment = DATABASEPREFIX=oc Environment = NEXTCLOUD_URL=https://cloud.abc.de

RuntimeDirectory = notify_push RuntimeDirectoryMode = 750

User = www-data Group = www-data

[Install] WantedBy = multi-user.target

my Webserver Config

location /push/ { proxy_pass http://unix:/var/run/notify_push/notify_push.sock:/; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } }

As you can see below the notify_push service is working but i dont know why I'm not able to finish the Setup.

user_n@server:~$ sudo systemctl status notify_push.service ● notify_push.service - Push daemon for Nextcloud clients Loaded: loaded (/etc/systemd/system/notify_push.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2021-03-20 22:14:47 CET; 1 day 14h ago Main PID: 4607 (notify_push) Tasks: 5 (limit: 4432) CGroup: /system.slice/notify_push.service └─4607 /var/www/nextcloud/apps/notify_push/bin/aarch64/notify_push /var/www/nextcloud/config/config.php

Mar 20 22:14:47 server systemd[1]: Started Push daemon for Nextcloud clients.

[buehlefs]

I also ran into this issue and found a workaround not mentioned here. The underlying issue (as far as I can tell) is, that the notyfy_push service resolves the configured nextcloud URL to the public IP of the server. For my setup this public IP cannot be included as a trusted proxie because it changes regularly. I also do not use a local docker container for nextcloud which means that the notify_push server cannot use localhost as the URL because the certificate will not match.

My workaround is to edit the /etc/hosts file to force the dns resolution to use the loopback IP that is configured as a trusted proxy:

127.0.1.1       your.nextcloud.domain
::1             your.nextcloud.domain

[ghost]

Thanks @buehlefs

your Hint with editing the /etc/hosts File helped me out. Now it seems to work.

[Crocmagnon]

I'm running nextcloud in Docker, and it turns out that environment variables override entries of the config file. The new error messages in the latest versions helped me understand that my trusted proxies config was not applied. Thanks to the devs for including helpful error messages!

My setup now works after setting the trusted proxies in the env var.

[lukbukkit]

I've tried to configure the notify_push application running in a Docker container with additional containers for traefik, Nextcloud with PHP-FPM and NGINX. I've got the same error and could fix it by allowing traefik to relay headers from inside the Docker network. --entryPoints.websecure.forwardedHeaders.trustedIPs=172.16.0.0/12 See https://doc.traefik.io/traefik/v2.3/routing/entrypoints/#forwarded-headers for more information.

[cloudwindy]

Spent my whole day dealing with this issue, still with no luck.

[cloudwindy]

Too painful to set up.

🗴 push server is not a trusted proxy, please add '172.30.0.10' to the list of trusted proxies or configure any existing reverse proxy to forward the 'x-forwarded-for' send by the push server.
  See https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html#defining-trusted-proxies for how to set trusted proxies.
  The following trusted proxies are currently configured: "172.0.0.0/8"
  The following x-forwarded-for header was received by Nextcloud: 172.30.0.10
    from the following remote: 172.30.0.10

[icewind1991]

172.30.0.10 is sending the wrong x-forwarded-for, so while it is set as a trusted proxy there is still a break in the proxy chain

[despens]

For everybody who is struggling to get the notify_push server into the trusted proxy list because using docker-compose and the IP number of the notify container can be different on each start of the setup, and it is not possible to use host names in the trusted_proxies array: I was able to solve this by querying for the container's IP number in the configuration file:

<?php
// file proxy.config.php
$CONFIG = array (

    'trusted_proxies' => array(
        0 => "1.1.1.1", // static IP of your web facing reverse proxy
        1 => dns_get_record("notify", DNS_A)[0]["ip"],
    ),
);

This code asks for the IP number of the container running the notify_push service (labeled "notify" in my docker-compose.yaml, but you can use any name of course) which must be running in the same network as the main nextcloud app.

Make sure that the notify container is already running when nextcloud is starting up, so add notify to the main app's depends_on list in docker-compose.yaml.

[jolly-jump]

1 => dns_get_record("notify", DNS_A)[0]["ip"],

Although this seems to be a very elegant solution to the problem, that the IP of the notify_push backend is dynamic in any docker setup. Unfortunately:

 2023-12-22 10:12:33 starting notify_push
nextcloud-hpb      | Error: php_literal_parser::unexpected_token
nextcloud-hpb      | 
nextcloud-hpb      |   × Error while parsing nextcloud config.php
nextcloud-hpb      |   ╰─▶ Error while parsing '/var/www/html/config/config.php':
nextcloud-hpb      |       No valid token found, expected one of boolean literal, integer literal,
nextcloud-hpb      |       float literal, string literal, 'null', 'array' or '['
nextcloud-hpb      |     ╭─[12:1]
nextcloud-hpb      |  12 │     0 => '157.90.6.38',
nextcloud-hpb      |  13 │     1 => dns_get_record("nextcloud-hpb", DNS_A)[0]["ip"],
nextcloud-hpb      |     ·          ┬
nextcloud-hpb      |     ·          ╰── Expected boolean literal, integer literal, float literal, string literal, 'null', 'array' or '['
nextcloud-hpb      |  14 │   ),
nextcloud-hpb      |     ╰────
nextcloud-hpb      | 
nextcloud-hpb      | 2023-12-22 10:12:33 notify_push stopped

[S0ulf3re]

So I'm having this issue as well with the official docker image. I had to manually set a static IP for one of push's networks because it was insisting on using that specific address, and then use that address as a trusted proxy. But I am still getting an error about The following x-forwarded-for header was received by Nextcloud: "1.2.3.4". I am using Traefik v2.11, however, I'm not sure how to set the x-forwarded-for header. As far as I know, Traefik only has these headers for entrypoints. How can I get the right headers forwarded?