mozilla-services / shavar

Tracking Protection update service for Firefox based on Safe Browsing protocol
Mozilla Public License 2.0
20 stars 10 forks source link

Reduce debug output message #138

Closed say-yawn closed 4 years ago

say-yawn commented 4 years ago

About this Issue

Currently there are too much debug output message that is not useful, such as the 200 responses:

ar 10 19:31:07 ip-172-31-30-24 docker-shavar[8789]: [pid: 1|app: 0|req: 128787/290012] 172.31.28.224 () {52 vars in 877 bytes} [Tue Mar 10 19:31:07 2020] POST /downloads?client=navclient-auto-ffox&appver=73.0&pver=2.2 => generated 7 bytes in 1 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
Mar 10 19:31:07 ip-172-31-30-24 docker-shavar[8789]: [pid: 1|app: 0|req: 128788/290013] 172.31.14.23 () {52 vars in 930 bytes} [Tue Mar 10 19:31:07 2020] POST /downloads?client=navclient-auto-ffox&appver=52.9&pver=2.2 => generated 110 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 80 bytes (1 switches on core 1)
Mar 10 19:31:07 ip-172-31-30-24 docker-shavar[8789]: [pid: 1|app: 0|req: 128790/290014] 172.31.9.81 () {52 vars in 915 bytes} [Tue Mar 10 19:31:07 2020] POST /downloads?client=navclient-auto-ffox&appver=52.9&pver=2.2 => generated 320 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 80 bytes (1 switches on core 1)
Mar 10 19:31:07 ip-172-31-30-24 docker-shavar[8789]: [pid: 1|app: 0|req: 128791/290015] 172.31.14.96 () {52 vars in 866 bytes} [Tue Mar 10 19:31:07 2020] POST /downloads?client=navclient-auto-ffox&appver=73.0&pver=2.2 => generated 7 bytes in 2 msecs (HTTP/1.1 200) 2 headers in 78 bytes (1 switches on core 0)
Mar 10 19:31:07 ip-172-31-30-24 docker-shavar[8789]: [pid: 1|app: 0|req: 128791/290016] 172.31.28.99 () {54 vars in 889 bytes} [Tue Mar 10 19:31:07 2020] POST /downloads?client=navclient-auto-ffox&appver=68.5&pver=2.2 => generated 123 bytes in 1 msecs (HTTP/1.1 200) 2 headers in 80 bytes (1 switches on core 1)

We should create a flag to enable/disable the suppression of the 200 responses so that logs like:

Mar 10 19:21:17 ip-172-31-30-24 docker-shavar[8532]: AttributeError: 'LogRecord' object has no attribute 'message'
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: Top level Sentry exception caught - failed creating log record
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: Skipping 75.0 version support for google-trackwhite-digest256 since the file does not exist in S3
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: {"time": "2020-03-10T19:21:18.047933Z", "v": 1, "message": "Skipping 75.0 version support for google-trackwhite-digest256 since the file does not exist in S3", "hostname": "ip-172-31-30-24", "pid": 1, "op": "shavar", "name": "shavar"}
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: Traceback (most recent call last):
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: File "/usr/local/lib/python2.7/site-packages/raven/handlers/logging.py", line 66, in emit
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: return self._emit(record)
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: File "/usr/local/lib/python2.7/site-packages/raven/handlers/logging.py", line 145, in _emit
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: handler_kwargs['formatted'] = text_type(record.message)
Mar 10 19:21:18 ip-172-31-30-24 docker-shavar[8532]: AttributeError: 'LogRecord' object has no attribute 'message

are surfaced.

Acceptance Criteria

say-yawn commented 4 years ago

Potential solution to disable the logging from uwsgi is run: exec /usr/local/bin/uwsgi --ini-paste shavar.ini --paste-logger --uid 10001 --gid 10001 --disable-logging --log-4xx --log-5xx instead of exec /usr/local/bin/uwsgi --ini-paste shavar.ini --paste-logger --uid 10001 --gid 10001 in startup.sh

LilyLME commented 4 years ago

Hello @skim1102 this is related to #125 which I am currently working on in shavar-creation-list. please assign this to me as well.

say-yawn commented 4 years ago

Thanks for expressing your interest @LilyLME. This debug output message suppression issue is different from the shavar-list-creation repo's debug message suppression. After completing the shavar-list-creation issue if you think you can pick this one up as well I am happy to discuss further then.