search

nskinkel / oppy

A Tor client (onion proxy) implementation in Python
BSD 3-Clause "New" or "Revised" License
24 stars 3 forks source link

Handle different compression formats and zlib errors #5

Closed nskinkel closed 10 years ago

nskinkel commented 10 years ago

When trying to decompress a downloaded consensus, we'll occassionally get:

zlib.error: Error -5 while decompressing data: incomplete or truncated stream

It's not clear to me if this is an error with the consensus doc itself, or if I'm somehow parsing it wrong.

Tor dir-spec states:

Clients MUST handle compressed concatenated information in two forms:

  • A concatenated list of zlib-compressed objects.
  • A zlib-compressed concatenated list of objects.

Investigate what zlib.decompress supports in Python (both types? just one? is this causing the error?).

Whatever the root cause, we need to handle errors here sanely.

dwtj commented 10 years ago

I haven't looked at the context in which this is being generated, and I'm just going on the zlib error message, but it seems like this problem would be caused by an incomplete or malformed download. I don't think that this has anything to do with your parsing.

If this is indeed the case, it seems to me that the way to handle such an error is to try to download, decompress, and parse a consensus document again. In the mean time, you could either block further accesses to the NetStatusDB by our other components, if the current state is considered invalid according to the Tor specification; alternatively, you could just serve up whatever old state you have in the DB.

nskinkel commented 10 years ago

Fixed in feeda12170