meejah / txtorcon

Twisted-based asynchronous Tor control protocol implementation. Includes unit-tests, examples, state-tracking code and configuration abstraction.
http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/
MIT License
250 stars 72 forks source link

regression: microdescriptor performance #224

Closed meejah closed 7 years ago

meejah commented 7 years ago

As part of "switching to Automat for state-machines", the microdescriptor parsing state-machine was switched over. This, however, has made TorState start-up times very slow.

meejah commented 7 years ago

To quantify this a little, on Python3:

meejah commented 7 years ago

on pypy, this is:

meejah commented 7 years ago

(of interest: stem's microdescriptor parser does ~7300/s on py2 and ~8700/s on py3)

meejah commented 7 years ago

I'm marking this for 0.19.0 -- I want at least revert to previous (or different) code for the microdescriptor parser before the next release. Building TorState objects has gone from "dozens of milliseconds" to "a couple seconds" which is very noticeable and way too slow for this.

meejah commented 7 years ago

For the 0.19.0 release, I put back the old parser code (using the spaghetti.FSM state-machine code). See ba96e8ba167882ecce4fdb3789701a723c9a8f7c