search

torpyorg / torpy

Pure python Tor client implementation
Apache License 2.0
411 stars 53 forks source link

'Can't deserialize #3 cell' #9

Closed quadrismegistus closed 4 years ago

quadrismegistus commented 4 years ago

Hi there,

First, thanks for this repo, it's fantastic. I'm hoping to compile it into a mobile application using Kivy.

However, I keep getting an error when I use torpy as opposed to a socks proxy to Tor Browser. It'll often work the first time in a while, but then fail after that.

Here's the code:

from torpy.http.requests import TorRequests
with TorRequests() as tor_requests:
    with tor_requests.get_session() as s:
        res = s.get('http://'+hostname+':'+str(port))
        print(res,res.text)

It works initially:

ryan@ember:~/github/Komrade/client$ python3 torconn.py 
<Response [200]> 404 go home friend
Stream #1: closed already

But then keeps failing with this error:

ryan@ember:~/github/Komrade/client$ python3 torconn.py 
Can't deserialize 3 cell: '04 44 42 f1 6f 00 00 0e 10'
Some handle errors
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torpy/circuit.py", line 236, in _do_recv
    self._handler_mgr.handle(cell)
  File "/usr/local/lib/python3.6/dist-packages/torpy/circuit.py", line 289, in handle
    handler(cell)
  File "/usr/local/lib/python3.6/dist-packages/torpy/guard.py", line 43, in wrapped
    return func(*args_new, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torpy/guard.py", line 121, in _on_relay
    circuit.handle_relay(cell)
  File "/usr/local/lib/python3.6/dist-packages/torpy/circuit.py", line 449, in handle_relay
    circuit_node, inner_cell = self._decrypt(cell)
  File "/usr/local/lib/python3.6/dist-packages/torpy/circuit.py", line 509, in _decrypt
    circuit_node.decrypt_backward(relay_cell)
  File "/usr/local/lib/python3.6/dist-packages/torpy/circuit.py", line 116, in decrypt_backward
    self._crypto_state.decrypt_backward(relay_cell)
  File "/usr/local/lib/python3.6/dist-packages/torpy/crypto_state.py", line 98, in decrypt_backward
    relay_cell.set_decrypted(**header)
  File "/usr/local/lib/python3.6/dist-packages/torpy/cells.py", line 302, in set_decrypted
    inner_cell = TorCell.deserialize(cell_type, 0, relay_payload, 0)
  File "/usr/local/lib/python3.6/dist-packages/torpy/cells.py", line 74, in deserialize
    kwargs = cell_type._deserialize_payload(payload, proto_version)
  File "/usr/local/lib/python3.6/dist-packages/torpy/cells.py", line 517, in _deserialize_payload
    reason = struct.unpack('!B', payload)[0]
struct.error: unpack requires a buffer of 1 bytes
TimeoutError
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torpy/http/adapter.py", line 124, in connect
    self._tor_stream = self._circuit.create_stream((self.host, self.port))
  File "/usr/local/lib/python3.6/dist-packages/torpy/circuit.py", line 330, in wrapped
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torpy/circuit.py", line 591, in create_stream
    tor_stream.connect(address)
  File "/usr/local/lib/python3.6/dist-packages/torpy/stream.py", line 260, in connect
    self._connect(address)
  File "/usr/local/lib/python3.6/dist-packages/torpy/stream.py", line 287, in _connect
    self._wait_connected(address, self._conn_timeout)
  File "/usr/local/lib/python3.6/dist-packages/torpy/stream.py", line 266, in _wait_connected
    raise TimeoutError('Could not connect to %r' % (address,))
TimeoutError: Could not connect to ('komrades.net', 5555)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.6/http/client.py", line 1264, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1310, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1259, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 976, in send
    self.connect()
  File "/usr/local/lib/python3.6/dist-packages/torpy/http/adapter.py", line 132, in connect
    self, 'Connection to %s timed out. (connect timeout=%s)' % (self.host, self.timeout)
urllib3.exceptions.ConnectTimeoutError: (<torpy.http.adapter.MyHTTPConnection object at 0x7feb4cd547b8>, 'Connection to komrades.net timed out. (connect timeout=None)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: MyHTTPConnectionPool(host='komrades.net', port=5555): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<torpy.http.adapter.MyHTTPConnection object at 0x7feb4cd547b8>, 'Connection to komrades.net timed out. (connect timeout=None)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "torconn.py", line 47, in <module>
    try_torpy()
  File "torconn.py", line 20, in try_torpy
    res = s.get('http://'+hostname+':'+str(port))
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 504, in send
    raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: MyHTTPConnectionPool(host='komrades.net', port=5555): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<torpy.http.adapter.MyHTTPConnection object at 0x7feb4cd547b8>, 'Connection to komrades.net timed out. (connect timeout=None)'))

Thanks for looking at this!

jbrown299 commented 4 years ago

Thanks for reporting. There was a bug in RELAY_END cell parsing for EXIT_POLICY reason. But in your case it wont help because some tor nodes block connections to non standard ports (for example 5555). But it is possible to implement filtering such nodes in circuits manager...

quadrismegistus commented 4 years ago

Wow, thanks so much! I pip-reinstalled torpy from the repo, and also changed the port for my server from 5555 to 80, but now I get this error -- which may be the same as this issue?

ryan@ember:~/github/Komrade/client$ python3 main.py 
[INFO   ] [Logger      ] Record log in /home/ryan/.kivy/logs/kivy_20-08-09_75.txt
[INFO   ] [Kivy        ] v1.11.1
[INFO   ] [Kivy        ] Installed at "/usr/local/lib/python3.6/dist-packages/kivy/__init__.py"
[INFO   ] [Python      ] v3.6.9 (default, Jul 17 2020, 12:50:27) 
[GCC 8.4.0]
[INFO   ] [Python      ] Interpreter at "/usr/bin/python3"
[INFO   ] [Factory     ] 184 symbols loaded
[INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_pil, img_gif (img_ffpyplayer ignored)
[INFO   ] [KivyMD      ] v0.104.1
[INFO   ] [Window      ] Provider: sdl2(['window_egl_rpi'] ignored)
[INFO   ] [GL          ] Using the "OpenGL" graphics system
[INFO   ] [GL          ] Backend used <sdl2>
[INFO   ] [GL          ] OpenGL version <b'3.0 Mesa 20.0.8'>
[INFO   ] [GL          ] OpenGL vendor <b'Intel Open Source Technology Center'>
[INFO   ] [GL          ] OpenGL renderer <b'Mesa DRI Intel(R) UHD Graphics 630 (CFL GT2)'>
[INFO   ] [GL          ] OpenGL parsed version: 3, 0
[INFO   ] [GL          ] Shading version <b'1.30'>
[INFO   ] [GL          ] Texture max size <16384>
[INFO   ] [GL          ] Texture max units <32>
[INFO   ] [Window      ] auto add sdl2 input provider
[INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
[INFO   ] [Text        ] Provider: sdl2
[INFO   ] [GL          ] NPOT texture support is available
xclip version 0.12
Copyright (C) 2001-2008 Kim Saunders et al.
Distributed under the terms of the GNU GPL
[INFO   ] [Clipboard   ] Provider: xclip
[INFO   ] [CutBuffer   ] cut buffer support enabled
[WARNING] [Lang        ] The file /home/ryan/github/Komrade/client/main.kv is loaded multiples times, you might have unwanted behaviors.
[INFO   ] [Base        ] Start application main loop
[ERROR  ] [ignored]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torpy/utils.py", line 78, in newfn
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 175, in renew
    if not self.verify(new_doc):
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 200, in verify
    pubkey = self._get_pubkey(sign['identity'], sign['signing_key_digest'])
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 207, in _get_pubkey
    key_certificate = provider.download_fp_sk(identity, signing_key_digest)
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 80, in download_fp_sk
    return http_get('{}/{}-{}'.format(self.fp_sk_url, identity, keyid))
  File "/usr/local/lib/python3.6/dist-packages/torpy/utils.py", line 190, in http_get
    with opener.open(url, timeout=timeout) as response:
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 503: Directory busy, try again later
 Retry with another authority...
[ERROR  ] [ignored]
Traceback (most recent call last):
  File "/usr/lib/python3.6/urllib/request.py", line 1325, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.6/http/client.py", line 1264, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1310, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1259, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 976, in send
    self.connect()
  File "/usr/lib/python3.6/http/client.py", line 948, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torpy/utils.py", line 78, in newfn
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 163, in renew
    raw_string = authority.download('consensus', prev_hash=prev_hash)
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 70, in download
    return http_get('{}/{}'.format(self.status_url, doc_type), headers=headers)
  File "/usr/local/lib/python3.6/dist-packages/torpy/utils.py", line 190, in http_get
    with opener.open(url, timeout=timeout) as response:
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1353, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.6/urllib/request.py", line 1327, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error timed out>
 Retry with another authority...
[ERROR  ] [ignored]
Traceback (most recent call last):
  File "/usr/lib/python3.6/urllib/request.py", line 1325, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.6/http/client.py", line 1264, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1310, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1259, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 976, in send
    self.connect()
  File "/usr/lib/python3.6/http/client.py", line 948, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torpy/utils.py", line 78, in newfn
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 175, in renew
    if not self.verify(new_doc):
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 200, in verify
    pubkey = self._get_pubkey(sign['identity'], sign['signing_key_digest'])
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 207, in _get_pubkey
    key_certificate = provider.download_fp_sk(identity, signing_key_digest)
  File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 80, in download_fp_sk
    return http_get('{}/{}-{}'.format(self.fp_sk_url, identity, keyid))
  File "/usr/local/lib/python3.6/dist-packages/torpy/utils.py", line 190, in http_get
    with opener.open(url, timeout=timeout) as response:
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1353, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.6/urllib/request.py", line 1327, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error timed out>
 Retry with another authority...
[INFO   ] [Base        ] Leaving application in progress...
 Traceback (most recent call last):
   File "/usr/lib/python3.6/urllib/request.py", line 1325, in do_open
     encode_chunked=req.has_header('Transfer-encoding'))
   File "/usr/lib/python3.6/http/client.py", line 1264, in request
     self._send_request(method, url, body, headers, encode_chunked)
   File "/usr/lib/python3.6/http/client.py", line 1310, in _send_request
     self.endheaders(body, encode_chunked=encode_chunked)
   File "/usr/lib/python3.6/http/client.py", line 1259, in endheaders
     self._send_output(message_body, encode_chunked=encode_chunked)
   File "/usr/lib/python3.6/http/client.py", line 1038, in _send_output
     self.send(msg)
   File "/usr/lib/python3.6/http/client.py", line 976, in send
     self.connect()
   File "/usr/lib/python3.6/http/client.py", line 948, in connect
     (self.host,self.port), self.timeout, self.source_address)
   File "/usr/lib/python3.6/socket.py", line 724, in create_connection
     raise err
   File "/usr/lib/python3.6/socket.py", line 713, in create_connection
     sock.connect(sa)
 socket.timeout: timed out

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
   File "main.py", line 251, in <module>
     App.run()
   File "/usr/local/lib/python3.6/dist-packages/kivy/app.py", line 855, in run
     runTouchApp()
   File "/usr/local/lib/python3.6/dist-packages/kivy/base.py", line 504, in runTouchApp
     EventLoop.window.mainloop()
   File "/usr/local/lib/python3.6/dist-packages/kivy/core/window/window_sdl2.py", line 747, in mainloop
     self._mainloop()
   File "/usr/local/lib/python3.6/dist-packages/kivy/core/window/window_sdl2.py", line 479, in _mainloop
     EventLoop.idle()
   File "/usr/local/lib/python3.6/dist-packages/kivy/base.py", line 342, in idle
     self.dispatch_input()
   File "/usr/local/lib/python3.6/dist-packages/kivy/base.py", line 327, in dispatch_input
     post_dispatch_input(*pop(0))
   File "/usr/local/lib/python3.6/dist-packages/kivy/base.py", line 293, in post_dispatch_input
     wid.dispatch('on_touch_up', me)
   File "kivy/_event.pyx", line 707, in kivy._event.EventDispatcher.dispatch
   File "/usr/local/lib/python3.6/dist-packages/kivymd/uix/behaviors/ripplebehavior.py", line 245, in on_touch_up
     return super().on_touch_up(touch)
   File "/usr/local/lib/python3.6/dist-packages/kivymd/uix/button.py", line 969, in on_touch_up
     return super().on_touch_up(touch)
   File "/usr/local/lib/python3.6/dist-packages/kivy/uix/behaviors/button.py", line 179, in on_touch_up
     self.dispatch('on_release')
   File "kivy/_event.pyx", line 703, in kivy._event.EventDispatcher.dispatch
   File "kivy/_event.pyx", line 1214, in kivy._event.EventObservers.dispatch
   File "kivy/_event.pyx", line 1098, in kivy._event.EventObservers._dispatch
   File "/usr/local/lib/python3.6/dist-packages/kivy/lang/builder.py", line 64, in custom_callback
     exec(__kvlang__.co_value, idmap)
   File "/home/ryan/github/Komrade/client/main.kv", line 236, in <module>
     app.login(username.text, password.text)
   File "main.py", line 226, in login
     with self.get_session() as sess:
   File "main.py", line 181, in get_session
     return get_tor_python_session()
   File "main.py", line 164, in get_tor_python_session
     with TorRequests() as tor_requests:
   File "/usr/local/lib/python3.6/dist-packages/torpy/http/requests.py", line 35, in __enter__
     self._tor = TorClient(auth_data=self._auth_data)
   File "/usr/local/lib/python3.6/dist-packages/torpy/client.py", line 37, in __init__
     self._consensus = consensus or TorConsensus()
   File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 145, in __init__
     self.renew()
   File "/usr/local/lib/python3.6/dist-packages/torpy/utils.py", line 78, in newfn
     return func(*args, **kwargs)
   File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 175, in renew
     if not self.verify(new_doc):
   File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 200, in verify
     pubkey = self._get_pubkey(sign['identity'], sign['signing_key_digest'])
   File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 207, in _get_pubkey
     key_certificate = provider.download_fp_sk(identity, signing_key_digest)
   File "/usr/local/lib/python3.6/dist-packages/torpy/consesus.py", line 80, in download_fp_sk
     return http_get('{}/{}-{}'.format(self.fp_sk_url, identity, keyid))
   File "/usr/local/lib/python3.6/dist-packages/torpy/utils.py", line 190, in http_get
     with opener.open(url, timeout=timeout) as response:
   File "/usr/lib/python3.6/urllib/request.py", line 526, in open
     response = self._open(req, data)
   File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
     '_open', req)
   File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
     result = func(*args)
   File "/usr/lib/python3.6/urllib/request.py", line 1353, in http_open
     return self.do_open(http.client.HTTPConnection, req)
   File "/usr/lib/python3.6/urllib/request.py", line 1327, in do_open
     raise URLError(err)
 urllib.error.URLError: <urlopen error timed out>
jbrown299 commented 4 years ago

Both bugs was fixed