fix(syncserver): Configure syncserver to use localhost verifier.

[rfk]

Fixes #112.

[shane-tomlinson]

@rfk - I see this when I apply this patch and start the sync server:

| ConnectionError: HTTPConnectionPool(host='host.docker.internal', port=3030): Max retries exceeded with url: /.well-known/fxa-client-configuration (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f7ca3fc8750>: Failed to establish a new connection: [Errno 111] Connection refused',)) 13|sync se | [2018-07-17 14:45:33 +0000] [12] [INFO] Worker exiting (pid: 12) 13|sync se | [2018-07-17 14:45:33 +0000] [7] [INFO] Shutting down: Master 13|sync se | [2018-07-17 14:45:33 +0000] [7] [INFO] Reason: Worker failed to boot. 13|sync se | [2018-07-17 14:46:01 +0000] [7] [INFO] Starting gunicorn 19.6.0 13|sync se | [2018-07-17 14:46:01 +0000] [7] [INFO] Listening at: http://0.0.0.0:5000 (7) 13|sync se | [2018-07-17 14:46:01 +0000] [7] [INFO] Using worker: sync 13|sync se | [2018-07-17 14:46:01 +0000] [12] [INFO] Booting worker with pid: 12 13|sync se | [2018-07-17 14:46:02 +0000] [12] [ERROR] Exception in worker process 13|sync se | Traceback (most recent call last): 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 557, in spawn_worker 13|sync se | worker.init_process() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 126, in init_process 13|sync se | self.load_wsgi() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 136, in load_wsgi 13|sync se | self.wsgi = self.app.wsgi() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi 13|sync se | self.callable = self.load() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load 13|sync se | return self.load_wsgiapp() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp 13|sync se | return util.import_app(self.app_uri) 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app 13|sync se | import(module) 13|sync se | File "/app/syncserver/wsgi_app.py", line 2, in 13|sync se | application = syncserver.main() 13|sync se | File "/app/syncserver/init.py", line 245, in main 13|sync se | config = get_configurator(global_config, settings) 13|sync se | File "/app/syncserver/init.py", line 237, in get_configurator 13|sync se | config.include(includeme) 13|sync se | File "/usr/local/lib/python2.7/site-packages/pyramid/config/init.py", line 754, in include 13|sync se | c(configurator) 13|sync se | File "/app/syncserver/init.py", line 137, in includeme 13|sync se | config.include("tokenserver", route_prefix="/token") 13|sync se | File "/usr/local/lib/python2.7/site-packages/pyramid/config/init.py", line 754, in include 13|sync se | c(configurator) 13|sync se | File "/usr/local/lib/python2.7/site-packages/tokenserver/init.py", line 46, in includeme 13|sync se | load_and_register("oauth", config) 13|sync se | File "/usr/local/lib/python2.7/site-packages/mozsvc/plugin.py", line 85, in load_and_register 13|sync se | plugin = load_from_settings(section_name, settings) 13|sync se | File "/usr/local/lib/python2.7/site-packages/mozsvc/plugin.py", line 132, in load_from_settings 13|sync se | return klass(kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/tokenserver/verifiers.py", line 201, in init 13|sync se | r = requests.get(server_url[:-3] + '/config') 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 70, in get 13|sync se | return request('get', url, params=params, kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 56, in request 13|sync se | return session.request(method=method, url=url, kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 488, in request 13|sync se | resp = self.send(prep, send_kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 609, in send 13|sync se | r = adapter.send(request, kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 487, in send 13|sync se | raise ConnectionError(e, request=request) 13|sync se | ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=9010): Max retries exceeded with url: /config (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fd6c73f9590>: Failed to establish a new connection: [Errno 111] Connection refused',)) 13|sync se | [2018-07-17 14:46:02 +0000] [12] [INFO] Worker exiting (pid: 12) 13|sync se | [2018-07-17 14:46:02 +0000] [7] [INFO] Shutting down: Master 13|sync se | [2018-07-17 14:46:02 +0000] [7] [INFO] Reason: Worker failed to boot.

[shane-tomlinson]

@rfk - I see this when I apply this patch and start the sync server:

I expect this is because I need to download the docker image for https://github.com/mozilla-services/syncserver/pull/127

[vladikoff]

Checking logs on travis, it says:

[rfk]

Checking logs on travis, it says:

:-( I wonder if the version of docker on travis does not support the special host.docker.internal name, IIRC it's been in docker-for-windows and docker-for-mac for a while but only recently added to docker-for-linux.

[rfk]

$ docker --version Docker version 18.05.0-ce, build f150324

[vladikoff]

$ docker --version Docker version 18.05.0-ce, build f150324

Which one do we need?

[rfk]

Oh, hrm, a comment went missing here - it turns out I was wrong, it doesn't look like "host.docker.internal" has landed in docker-for-linux yet.

[rfk]

I fixed travis with a bit of if-then-else to choose the right localhost addr. I removed the setting of SYNCSERVER_IDENTITY_PROVIDER which was causing it to block waiting for content-server to start, because I don't think this should be necessary for development. (The syncserver will accept assertions from any browserid IdP in this case, which is fine for dev purposes).

[vladikoff]

We are Syncing yet again!

[shane-tomlinson]

:( Am I doing something wrong? I'm not.

13|sync se | WARNING: Published ports are discarded when using host network mode 13|sync se | [2018-07-18 14:12:10 +0000] [7] [INFO] Starting gunicorn 19.6.0 13|sync se | [2018-07-18 14:12:10 +0000] [7] [INFO] Listening at: http://0.0.0.0:5000 (7) 13|sync se | [2018-07-18 14:12:10 +0000] [7] [INFO] Using worker: sync 13|sync se | [2018-07-18 14:12:10 +0000] [12] [INFO] Booting worker with pid: 12 13|sync se | [2018-07-18 14:12:11 +0000] [12] [ERROR] Exception in worker process 13|sync se | Traceback (most recent call last): 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 557, in spawn_worker 13|sync se | worker.init_process() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 126, in init_process 13|sync se | self.load_wsgi() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 136, in load_wsgi 13|sync se | self.wsgi = self.app.wsgi() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi 13|sync se | self.callable = self.load() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load 13|sync se | return self.load_wsgiapp() 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp 13|sync se | return util.import_app(self.app_uri) 13|sync se | File "/usr/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app 13|sync se | import(module) 13|sync se | File "/app/syncserver/wsgi_app.py", line 2, in 13|sync se | application = syncserver.main() 13|sync se | File "/app/syncserver/init.py", line 257, in main 13|sync se | config = get_configurator(global_config, settings) 13|sync se | File "/app/syncserver/init.py", line 249, in get_configurator 13|sync se | config.include(includeme) 13|sync se | File "/usr/local/lib/python2.7/site-packages/pyramid/config/init.py", line 754, in include 13|sync se | c(configurator) 13|sync se | File "/app/syncserver/init.py", line 62, in includeme 13|sync se | r = requests.get(urljoin(idp, '/.well-known/fxa-client-configuration')) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 70, in get 13|sync se | return request('get', url, params=params, kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 56, in request 13|sync se | return session.request(method=method, url=url, kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 488, in request 13|sync se | resp = self.send(prep, send_kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 609, in send 13|sync se | r = adapter.send(request, **kwargs) 13|sync se | File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 487, in send 13|sync se | raise ConnectionError(e, request=request) 13|sync se | ConnectionError: HTTPConnectionPool(host='localhost', port=3030): Max retries exceeded with url: /.well-known/fxa-client-configuration (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f2e2713a990>: Failed to establish a new connection: [Errno 111] Connection refused',)) 13|sync se | [2018-07-18 14:12:11 +0000] [12] [INFO] Worker exiting (pid: 12) 13|sync se | [2018-07-18 14:12:11 +0000] [7] [INFO] Shutting down: Master 13|sync se | [2018-07-18 14:12:11 +0000] [7] [INFO] Reason: Worker failed to boot.

I have git pulled, & run update_all

[rfk]

Hrm...

ConnectionError: HTTPConnectionPool(host='localhost', port=3030): Max retries exceeded with url: /.well-known/fxa-client-configuration

So this is trying to connect to http://localhost:3030/.well-known/fxa-client-configuration, but I don't know why. The latst version of ./_scripts/syncserver.sh should not be setting the env var for that. Can you double-check any references to localhost in that script. I would expect only the following:

$> grep 'localhost' syncserver.sh
  -e SYNCSERVER_PUBLIC_URL=http://localhost:5000 \

[shane-tomlinson]

grep localhost syncserver.sh

-e SYNCSERVER_PUBLIC_URL=http://localhost:5000 \