Closed mileswwatkins closed 6 years ago
There's some server complaining that I've seen as I try to get this to work, including these tracebacks:
$ docker-compose up daemon
...
daemon_1 | INFO:fabfile.daemons:2018-09-11 16:08:53,766: updating get-caught-up text (runs in parallel)
daemon_1 | INFO:fabfile.daemons:2018-09-11 16:08:54,270: loading results
daemon_1 |
daemon_1 | Warning: local() encountered an error (return code 64) while executing 'elex results --results-level ru --officeids H,S,G,I 2016-11-08 > .data/results---results-level_ru_--officeids_HSGI.csv'
daemon_1 |
daemon_1 | INFO:fabfile.data:2018-09-11 16:09:01,403: results loaded
daemon_1 | INFO:fabfile.daemons:2018-09-11 16:09:01,403: results loaded: 7.637657165527344 seconds
daemon_1 | INFO:fabfile.daemons:2018-09-11 16:09:18,144: results rendered and published: 24.378236532211304 seconds
daemon_1 | INFO:fabfile.daemons:2018-09-11 16:09:18,245: updating get-caught-up text (runs in parallel)
daemon_1 | Traceback (most recent call last):
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/main.py", line 763, in main
daemon_1 | *args, **kwargs
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/tasks.py", line 427, in execute
daemon_1 | results['<local-only>'] = task.run(*args, **new_kwargs)
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/tasks.py", line 174, in run
daemon_1 | return self.wrapped(*args, **kwargs)
daemon_1 | File "/opt/elections18/fabfile/daemons.py", line 40, in main
daemon_1 | execute('text.update_in_parallel')
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/tasks.py", line 427, in execute
daemon_1 | results['<local-only>'] = task.run(*args, **new_kwargs)
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/tasks.py", line 174, in run
daemon_1 | return self.wrapped(*args, **kwargs)
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/decorators.py", line 184, in inner
daemon_1 | return func(*args, **kwargs)
daemon_1 | File "/opt/elections18/fabfile/text.py", line 27, in update_in_parallel
daemon_1 | update_calendar()
daemon_1 | File "/opt/elections18/fabfile/text.py", line 46, in update_calendar
daemon_1 | app_config.CALENDAR_PATH
daemon_1 | File "/opt/elections18/oauth.py", line 119, in get_document
daemon_1 | raise KeyError("Error! Google returned a %s error" % response.status)
daemon_1 | KeyError: 'Error! Google returned a 500 error'
elections18-general_daemon_1 exited with code 1
and
$ docker-compose up daemon
elections18-general_database_1 is up-to-date
Starting elections18-general_daemon_1 ... done
Attaching to elections18-general_daemon_1
daemon_1 | INFO:fabfile.daemons:2018-09-11 15:56:20,504: updating get-caught-up text (runs in parallel)
daemon_1 | Traceback (most recent call last):
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/main.py", line 763, in main
daemon_1 | *args, **kwargs
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/tasks.py", line 427, in execute
daemon_1 | results['<local-only>'] = task.run(*args, **new_kwargs)
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/tasks.py", line 174, in run
daemon_1 | return self.wrapped(*args, **kwargs)
daemon_1 | File "/opt/elections18/fabfile/daemons.py", line 40, in main
daemon_1 | execute('text.update_in_parallel')
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/tasks.py", line 427, in execute
daemon_1 | results['<local-only>'] = task.run(*args, **new_kwargs)
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/tasks.py", line 174, in run
daemon_1 | return self.wrapped(*args, **kwargs)
daemon_1 | File "/usr/local/lib/python3.6/site-packages/fabric/decorators.py", line 184, in inner
daemon_1 | return func(*args, **kwargs)
daemon_1 | File "/opt/elections18/fabfile/text.py", line 27, in update_in_parallel
daemon_1 | update_calendar()
daemon_1 | File "/opt/elections18/fabfile/text.py", line 46, in update_calendar
daemon_1 | app_config.CALENDAR_PATH
daemon_1 | File "/opt/elections18/oauth.py", line 111, in get_document
daemon_1 | credentials = get_credentials()
daemon_1 | File "/opt/elections18/oauth.py", line 92, in get_credentials
daemon_1 | credentials.refresh()
daemon_1 | File "/usr/local/lib/python3.6/site-packages/authomatic/core.py", line 772, in refresh
daemon_1 | return self.provider_class(self, None, self.provider_name).refresh_credentials(self)
daemon_1 | File "/usr/local/lib/python3.6/site-packages/authomatic/providers/oauth2.py", line 251, in refresh_credentials
daemon_1 | method='POST')
daemon_1 | File "/usr/local/lib/python3.6/site-packages/authomatic/providers/oauth2.py", line 162, in create_request_elements
daemon_1 | 'are required to create OAuth 2.0 refresh token request elements!')
daemon_1 | authomatic.exceptions.OAuth2Error: Credentials with valid refresh_token, consumer_key, consumer_secret are required to create OAuth 2.0 refresh token request elements!
elections18-general_daemon_1 exited with code 1
But I can look into these more later, handling them as separate bugs. Much more important come AP tests in October.
This will power the pipeline that @freejoe76 is working on now; cc https://github.com/nprapps/elections18-graphics/issues/48
Make sure this task is included (as a background process?) in the main Fabric
daemon
task.Also, make sure that the production server that runs this daemon will have OAuth/Google credentials that'll be able to run
fab text.update
to properly update this file.