search

MaddieM4 / EJTP-lib-python

Encrypted JSON Transport Protocol library for Python 2.6-3.3.
GNU Lesser General Public License v3.0
23 stars 10 forks source link

Performance bug hunt #120

Closed MaddieM4 closed 11 years ago

MaddieM4 commented 11 years ago

This is an in-progress effort to optimize EJTP based on the benchmarking utility. I'm going to use this pull request as a place to document the process as I go.

MaddieM4 commented 11 years ago

Currently the biggest bottleneck on my system is actually the encryption library. For some reason PyCrypto seems to be using Python-heavy implementations rather than the highly efficient compiled ones. I'm interested in seeing what the numbers look like after diagnosing that.

I find that the most useful way of sorting cProfile output is by total time, at least for figuring out which pain points desperately need optimization.

cProfile doesn't accurately capture pong and done results, because all the work is carried out in a secondary thread (the one belonging to the UDP jack). I'm still trying to figure out how exactly to remedy this. It definitely requires rethinking some assumptions I made when setting up cProfile support. Probably the best option here is to finally try out gevent monkey-patching (something I've always intended to support), such that all activity ends up in a single thread per process. Not sure that this will work, but it's worth a shot.

MaddieM4 commented 11 years ago

I'm going to merge this for now, because the performance gains are important. But I'll start another performance quest for 0.9.5, now that we can actually collect cProfile stats about pong and done.