search

internetarchive / warcprox

WARC writing MITM HTTP/S proxy
378 stars 54 forks source link

Adds logging for failed connections #145

Closed adam-miller closed 3 years ago

nlevitt commented 4 years ago

I refactored a bit to share some the code you had copied and pasted. Also there was a bug where this kind of exception could happen:

Traceback (most recent call last):
  File "/Users/nlevitt/src/warcprox/warcprox/mitmproxy.py", line 364, in do_CONNECT
    self.send_error(500, str(e))
  File "/Users/nlevitt/src/warcprox/warcprox/warcproxy.py", line 385, in send_error
    exception=exception)
  File "/Users/nlevitt/src/warcprox/warcprox/warcproxy.py", line 459, in __init__
    referer=referer, payload_digest=None, truncated=None, warc_records=None, do_not_archive=do_not_archive)
  File "/Users/nlevitt/src/warcprox/warcprox/warcproxy.py", line 407, in __init__
    self.url = url.encode('ascii')
AttributeError: 'NoneType' object has no attribute 'encode'

That's with a logging tweak committed, the actual manifestation looked like this for example:

2020-05-06 14:35:18,379 56341 WARNING MitmProxyHandler(tid=n/a,started=2020-05-06T21:35:18.315977,client=127.0.0.1:55088) warcprox.warcprox.WarcProxyHandler.do_CONNECT(mitmproxy.py:366) failed to send error response ([SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1076)) to proxy client: 'NoneType' object has no attribute 'encode'
nlevitt commented 4 years ago

If you can test with my changes to make sure it still works properly for you, I give my stamp of approval to merge. A test case would be nice to have but I'm ok with merging without.