freedomofpress / securethenews

An automated scanner and web dashboard for tracking TLS deployment across news organizations
https://securethe.news
GNU Affero General Public License v3.0
102 stars 25 forks source link

Forces JSON log format in prod settings #203

Closed conorsch closed 5 years ago

conorsch commented 5 years ago

Two changes here, both intended to standardize the production logging story:

Upon merge, the new container will be pushed to the testing cluster, and we can evaluate the logging change there.

Testing

Using the prod docker-compose environment, you can observe that on current master, each request generates at least two events, on JSON and one plaintext:

django_1 | [2019-08-12 16:26:10 +0000] [21] [DEBUG] GET /sites/ django_1 | {"created": 1565627171, "levelname": "INFO", "request": {"data": {}, "meta": {"http_host": "localhost:8000", "http_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0", "path_info": "/sites/", "remote_addr": "172.18.0.1"}, "method": "GET", "path": "/sites/", "scheme": "http", "user": "AnonymousUser"}, "response": {"charset": "utf-8", "headers": {"Content-Type": "text/html; charset=utf-8"}, "reason": "OK", "status": 200}}

Whereas on this branch, after running docker-compose -f prod-docker-compose.yaml up --build, you'll see only one log event per request:

django_1 | {"created": 1565628214, "levelname": "INFO", "request": {"data": {}, "meta": {"http_host": "localhost:8000", "http_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0", "path_info": "/sites/", "remote_addr": "172.18.0.1"}, "method": "GET", "path": "/sites/", "scheme": "http", "user": "AnonymousUser"}, "response": {"charset": "utf-8", "headers": {"Content-Type": "text/html; charset=utf-8"}, "reason": "OK", "status": 200}} django_1 | {"created": 1565628218, "levelname": "INFO", "request": {"data": {}, "meta": {"http_host": "localhost:8000", "http_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0", "path_info": "/sites/the-intercept", "remote_addr": "172.18.0.1"}, "method": "GET", "path": "/sites/the-intercept", "scheme": "http", "user": "AnonymousUser"}, "response": {"charset": "utf-8", "headers": {"Content-Type": "text/html; charset=utf-8"}, "reason": "OK", "status": 200}} django_1 | {"created": 1565628224, "levelname": "INFO", "request": {"data": {}, "meta": {"http_host": "localhost:8000", "http_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0", "path_info": "/sites/", "remote_addr": "172.18.0.1"}, "method": "GET", "path": "/sites/", "scheme": "http", "user": "AnonymousUser"}, "response": {"charset": "utf-8", "headers": {"Content-Type": "text/html; charset=utf-8"}, "reason": "OK", "status": 200}} django_1 | {"created": 1565628243, "levelname": "INFO", "request": {"data": {}, "meta": {"http_host": "localhost:8000", "http_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0", "path_info": "/sites/the-moscow-times", "remote_addr": "172.18.0.1"}, "method": "GET", "path": "/sites/the-moscow-times", "scheme": "http", "user": "AnonymousUser"}, "response": {"charset": "utf-8", "headers": {"Content-Type": "text/html; charset=utf-8"}, "reason": "OK", "status": 200}} django_1 | {"created": 1565628250, "exception": {"message": "", "traceback": [{"file": "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", "line": 124, "method": "_get_response"}, {"file": "/usr/local/lib/python3.6/site-packages/wagtail/core/views.py", "line": 17, "method": "serve"}, {"file": "/usr/local/lib/python3.6/site-packages/wagtail/core/models.py", "line": 610, "method": "route"}], "type": "django.http.response.Http404"}, "levelname": "ERROR", "request": {"data": {}, "meta": {"http_host": "localhost:8000", "http_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0", "path_info": "/sites/the-moscow-times/asdlkfjsadf/", "remote_addr": "172.18.0.1"}, "method": "GET", "path": "/sites/the-moscow-times/asdlkfjsadf/", "scheme": "http", "user": "AnonymousUser"}} django_1 | {"created": 1565628251, "levelname": "WARNING", "request": {"data": {}, "meta": {"http_host": "localhost:8000", "http_user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0", "path_info": "/sites/the-moscow-times/asdlkfjsadf/", "remote_addr": "172.18.0.1"}, "method": "GET", "path": "/sites/the-moscow-times/asdlkfjsadf/", "scheme": "http", "user": "AnonymousUser"}, "response": {"charset": "utf-8", "headers": {"Content-Type": "text/html; charset=utf-8"}, "reason": "Not Found", "status": 404}}