torproject / stem

Python controller library for Tor
https://stem.torproject.org/
GNU Lesser General Public License v3.0
260 stars 75 forks source link

Incremental descriptor queries #30

Open atagar opened 4 years ago

atagar commented 4 years ago

Migrated from ticket 8248.

We presently have a get_server_descriptor() and get_network_status() method, but it's horribly inefficient. Our present messaging infrastructure provided by the BaseController class sends or receives complete messages rather than streaming the message as it comes in. This means that we load the compete "GETINFO desc/all" response into memory before providing our caller with even the first descriptor. This eats up memory and it means an unnecessarily long latency before we provide results. Unfortunately this will take a bit of re-architecting in the BaseController to allow for streamed messages.