search

crazy-max / docker-firefox-syncserver

Firefox Sync Server Docker image
MIT License
195 stars 14 forks source link

forwarded_allow_ips variable not applied to gunicorn launch #61

Open rainforest1155 opened 2 years ago

rainforest1155 commented 2 years ago

Behaviour

As my Nginx server with the reverse proxy is on another machine than the docker, I'm in need of the "forwarded_allow_ips" setting set to * (or the IP of my Nginx server), but I can't get the setting to be applied to Gunicorn. It always keeps the default of 127.0.0.1 - see line from the debug log: firefox_syncserver | forwarded_allow_ips: ['127.0.0.1']

Steps to reproduce this issue

  1. docker pull
  2. set FF_SYNCSERVER_FORWARDED_ALLOW_IPS = *
  3. run docker

Expected behaviour

gunicorn should display set forward_allow_ips variable instead of the default. This would prevent gunicorn from mistrusting the https protocol from the proxy and allowing the syncserver to properly work.

Actual behaviour

gunicorn always uses default forwarded_allow_ips = 127.0.0.1 and per the note from the official syncserver docs:

Note

If you see errors about a mismatch between public_url and application_url, you may need to tell gunicorn that it should trust the X-Forwarded-Proto header being sent by nginx. Add the following to the gunicorn configuration in syncserver.ini:

forwarded_allow_ips = *

this prevents the sync server from actually syncing as the sync process then attempts to use http:// instead of the secured one, which fails as my server isn't set to accept insecure requests.

Configuration

services: firefox-syncserver: image: crazymax/firefox-syncserver container_name: firefox_syncserver ports:

[app:main] use = egg:syncserver

[syncserver]

This must be edited to point to the public URL of your server,

i.e. the URL as seen by Firefox.

public_url = https://[myserver]/moz-sync

This defines the database in which to store all server data.

sqluri = sqlite:///data/syncserver.db

This is a secret key used for signing authentication tokens.

It should be long and randomly-generated.

The following command will give a suitable value on *nix systems:

#

head -c 20 /dev/urandom | sha1sum

#

If not specified then the server will generate a temporary one at startup.

secret = d0a07aea2425a116f688aed63bd60e1278fcc512

Set this to "false" to disable new-user signups on the server.

Only request by existing accounts will be honoured.

allow_new_users = true

Set this to "true" to work around a mismatch between public_url and

the application URL as seen by python, which can happen in certain reverse-

proxy hosting setups. It will overwrite the WSGI environ dict with the

details from public_url. This could have security implications if e.g.

you tell the app that it's on HTTPS but it's really on HTTP, so it should

only be used as a last resort and after careful checking of server config.

force_wsgi_environ = false

Uncomment and edit the following to use a local BrowserID verifier

rather than posting assertions to the mozilla-hosted verifier.

Audiences should be set to your public_url without a trailing slash.

[browserid]

backend = tokenserver.verifiers.LocalVerifier

audiences = https://localhost:5000

If you are running Nginx on a different host than the ff sync server the ff snyc server have to trust the X-Forwarded-* headers sent by Nginx.

forwarded_allow_ips = *

Include all necessary configuration files : `docker-compose.yml`, `.env`, ...

### Docker info

Client: Context: default Debug Mode: false Plugins: app: Docker App (Docker Inc., v0.9.1-beta3) buildx: Docker Buildx (Docker Inc., v0.8.2-docker) compose: Docker Compose (Docker Inc., v2.6.0)

Server: Containers: 9 Running: 8 Paused: 0 Stopped: 1 Images: 7 Server Version: 20.10.17 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc Default Runtime: runc Init Binary: docker-init containerd version: 0197261a30bf81f1ee8e6a4dd2dea0ef95d67ccb runc version: v1.1.3-0-g6724737 init version: de40ad0 Security Options: apparmor seccomp Profile: default cgroupns Kernel Version: 5.15.43-sunxi64 Operating System: Debian GNU/Linux 11 (bullseye) OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 1.943GiB Name: pine64 ID: KWYO:YKZ5:RMQF:347Q:QUYU:XDYJ:36VS:ZUVK:SI57:2EIF:F4FX:W5XW Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false


### Logs

Attaching to firefox_syncserver firefox_syncserver | Setting timezone to Europe/Paris... firefox_syncserver | Checking prerequisites... firefox_syncserver | Generating configuration... firefox_syncserver | Fixing perms... firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [DEBUG] Current configuration: firefox_syncserver | secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'} firefox_syncserver | proxy_protocol: False firefox_syncserver | worker_connections: 1000 firefox_syncserver | statsd_host: None firefox_syncserver | max_requests_jitter: 0 firefox_syncserver | post_fork: <function post_fork at 0xffff9cfc2f50> firefox_syncserver | pythonpath: None firefox_syncserver | enable_stdio_inheritance: False firefox_syncserver | worker_class: sync firefox_syncserver | ssl_version: 3 firefox_syncserver | suppress_ragged_eofs: True firefox_syncserver | syslog: False firefox_syncserver | syslog_facility: user firefox_syncserver | when_ready: <function when_ready at 0xffff9cfc2c50> firefox_syncserver | pre_fork: <function pre_fork at 0xffff9cfc2dd0> firefox_syncserver | cert_reqs: 0 firefox_syncserver | preload_app: False firefox_syncserver | keepalive: 2 firefox_syncserver | accesslog: None firefox_syncserver | group: 1000 firefox_syncserver | graceful_timeout: 30 firefox_syncserver | do_handshake_on_connect: False firefox_syncserver | spew: False firefox_syncserver | workers: 1 firefox_syncserver | proc_name: None firefox_syncserver | sendfile: None firefox_syncserver | pidfile: None firefox_syncserver | umask: 0 firefox_syncserver | on_reload: <function on_reload at 0xffff9cfc2ad0> firefox_syncserver | pre_exec: <function pre_exec at 0xffff9cfc85d0> firefox_syncserver | worker_tmp_dir: None firefox_syncserver | post_worker_init: <function post_worker_init at 0xffff9cfc8150> firefox_syncserver | limit_request_fields: 100 firefox_syncserver | on_exit: <function on_exit at 0xffff9cfc8cd0> firefox_syncserver | config: None firefox_syncserver | logconfig: None firefox_syncserver | check_config: False firefox_syncserver | statsd_prefix: firefox_syncserver | proxy_allow_ips: ['127.0.0.1'] firefox_syncserver | pre_request: <function pre_request at 0xffff9cfc8750> firefox_syncserver | post_request: <function post_request at 0xffff9cfc8850> firefox_syncserver | user: 1000 firefox_syncserver | forwarded_allow_ips: ['127.0.0.1'] firefox_syncserver | worker_int: <function worker_int at 0xffff9cfc82d0> firefox_syncserver | threads: 1 firefox_syncserver | max_requests: 0 firefox_syncserver | limit_request_line: 4094 firefox_syncserver | access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s" firefox_syncserver | certfile: None firefox_syncserver | worker_exit: <function worker_exit at 0xffff9cfc89d0> firefox_syncserver | chdir: / firefox_syncserver | paste: /opt/syncserver/syncserver.ini firefox_syncserver | default_proc_name: /opt/syncserver/syncserver.ini firefox_syncserver | errorlog: - firefox_syncserver | loglevel: debug firefox_syncserver | capture_output: False firefox_syncserver | syslog_addr: udp://localhost:514 firefox_syncserver | syslog_prefix: None firefox_syncserver | daemon: False firefox_syncserver | ciphers: TLSv1 firefox_syncserver | on_starting: <function on_starting at 0xffff9cfc2950> firefox_syncserver | worker_abort: <function worker_abort at 0xffff9cfc8450> firefox_syncserver | bind: ['0.0.0.0:5000'] firefox_syncserver | raw_env: [] firefox_syncserver | reload: False firefox_syncserver | limit_request_field_size: 8190 firefox_syncserver | nworkers_changed: <function nworkers_changed at 0xffff9cfc8b50> firefox_syncserver | timeout: 30 firefox_syncserver | ca_certs: None firefox_syncserver | django_settings: None firefox_syncserver | tmp_upload_dir: None firefox_syncserver | keyfile: None firefox_syncserver | backlog: 2048 firefox_syncserver | logger_class: gunicorn.glogging.Logger firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [INFO] Starting gunicorn 19.6.0 firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [DEBUG] Arbiter booted firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1) firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [INFO] Using worker: sync firefox_syncserver | [2022-09-08 11:00:32 +0000] [16] [INFO] Booting worker with pid: 16 firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [DEBUG] 1 workers firefox_syncserver | /usr/local/lib/python2.7/site-packages/pyramid/path.py:341: PkgResourcesDeprecationWarning: Parameters to load are deprecated. Call .resolve and .require separately. firefox_syncserver | 'x=%s' % value).load(False) firefox_syncserver | /usr/local/lib/python2.7/site-packages/jwt/utils.py:8: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. firefox_syncserver | from cryptography.hazmat.primitives.asymmetric.utils import (

nginx reverse proxy config:
location /moz-sync {
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_redirect off;
    proxy_read_timeout 120;
    proxy_connect_timeout 10;
    proxy_pass http://[DockerMachineIP]:5000/;
}
rainforest1155 commented 2 years ago

I noticed one potential issue in the docker container. While it's running I opened a shell and looked at the app/docker-entrypoint.sh file, which starts off with:

#!/bin/sh

cd $(dirname $0)
case "$1" in
    server)
        export SYNCSERVER_SQLURI="${SYNCSERVER_SQLURI:-sqlite:///tmp/syncserver.db}"
        exec gunicorn \
            --bind ${HOST-0.0.0.0}:${PORT-5000} \
            --forwarded-allow-ips="${SYNCSERVER_FORWARDED_ALLOW_IPS:-127.0.0.1,172.17.0.1}" \
            syncserver.wsgi_app
        ;;

It lists this variable SYNCSERVER_FORWARDED_ALLOW_IPS while I believe the variable name should actually be FF_SYNCSERVER_FORWARDED_ALLOW_IPS.

That said, I'm not versed in docker and wasn't actually sure how to make changes to that file to test this as each time the docker is stopped and restarted, the change seems to revert.