search

chromium / web-page-replay

DEPRECATED - Use WebPageReplayGo instead:
https://github.com/catapult-project/catapult/blob/master/web_page_replay_go/README.md
Apache License 2.0
233 stars 75 forks source link

Unsuccessfully trying to record https://facebook.com #94

Open armenzg opened 7 years ago

armenzg commented 7 years ago

Command being used:

$ ./replay.py --record archive.wpr --should_generate_certs I've also tried: $ ./replay.py --record archive.wpr --no-ssl but it does not load the page. It seems I need to add a certificate: https://cl.ly/2H1m2j1n0U1c

I would have used pyOpenSSL==0.13.0 but it does not build Mac 10.12 (MacOS Sierra)

(wpr) armenzg@armenzg-mbp web-page-replay$ pip freeze appdirs==1.4.3 asn1crypto==0.22.0 cffi==1.10.0 cryptography==1.8.1 enum34==1.1.6 idna==2.5 ipaddress==1.0.18 packaging==16.8 pycparser==2.17 pyOpenSSL==17.0.0 pyparsing==2.2.0 six==1.10.0

(ERROR) 2017-04-24 14:03:41,986 sslproxy._SetUpUsingDummyCert:61 Dropping request without SNI (ERROR) 2017-04-24 14:03:41,987 sslproxy.handle_servername:51 Exception in SNI handler: [('SSL routines', 'SSL_shutdown', 'shutdown while in init')] Exception in thread Thread-55: Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in bootstrap_inner self.run() File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run self.target(*self.args, **self.kwargs) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 599, in process_request_thread self.handle_error(request, client_address) File "/Users/armenzg/repos/web-page-replay/httpproxy.py", line 419, in handle_error _HandleSSLCertificateError() File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 596, in process_request_thread self.finish_request(request, client_address) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 331, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 650, in init self.setup() File "/Users/armenzg/repos/web-page-replay/sslproxy.py", line 82, in setup _SetUpUsingDummyCert(self) File "/Users/armenzg/repos/web-page-replay/sslproxy.py", line 63, in _SetUpUsingDummyCert raise certutils.Error('SSL handshake error %s: %s' % (host, str(v))) Error: SSL handshake error www.facebook.com: [('SSL routines', 'tls_post_process_client_hello', 'no shared cipher')] Exception in thread Thread-56: Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in bootstrap_inner self.run() File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run self.target(*self.args, **self.kwargs) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 599, in process_request_thread self.handle_error(request, client_address) File "/Users/armenzg/repos/web-page-replay/httpproxy.py", line 419, in handle_error _HandleSSLCertificateError() File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 596, in process_request_thread self.finish_request(request, client_address) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 331, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 654, in init self.finish() File "/Users/armenzg/repos/web-page-replay/sslproxy.py", line 86, in finish self.connection.shutdown() File "/Users/armenzg/venv/wpr/lib/python2.7/site-packages/OpenSSL/SSL.py", line 1715, in shutdown self._raise_ssl_error(self._ssl, result) File "/Users/armenzg/venv/wpr/lib/python2.7/site-packages/OpenSSL/SSL.py", line 1378, in _raise_ssl_error _raise_current_error() File "/Users/armenzg/venv/wpr/lib/python2.7/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue raise exception_type(errors) Error: [('SSL routines', 'SSL_shutdown', 'shutdown while in init')]

armenzg commented 7 years ago

@nedn any suggestions? ^

nedn commented 7 years ago

Hmhh, I would recommend bypassing the certificate stuffs by running Chrome with "--ignore-certificate-error". The whole things about WPR handles certificate needs to be fixed, but I have no expertise on net security :-(

armenzg commented 7 years ago

Thanks ned! In this case I'm actually trying to use it with Firefox. Do you know if anyone else has been successful?

sleevi commented 7 years ago

This isn't related to certificates. It's related to TLS ciphersuites:

Error: SSL handshake error www.facebook.com: [('SSL routines', 'tls_post_process_client_hello', 'no shared cipher')]

I'm not sure what ciphersuites w-p-r advertises when using the pyOpenSSL version you are using, but it seems to be that you have zero compatibility with the default-enabled Firefox suites. You could fiddle with Firefox's about:flags, as they may have disabled one or more ciphersuites w-p-r is using.

You could also see what ciphersuites OpenSSL negotiates with openssl s_client -connect [your ip]:[wpr port] to see what ciphersuites it negotiates.