Open itamarst opened 1 year ago
Oh, and I should add that profiling indicates the bulk of the additional time is inside OpenSSL code, it's not about Python function overhead.
I fixed this in Twisted with a buffering layer, but it may well be an issue in other frameworks or applications.
Consider the following program:
You can run it with lots of small writes, or with bigger writes. Here's the CPU usage:
I also measured bandwidth usage (bytes received in the TCP layer of Twisted) of the two options:
Discussion
Connection.send()
returns how many bytes were successfully written. And pyOpenSSL sets SSL_MODE_ENABLE_PARTIAL_WRITE. This means futuresend()
should not be called with same bytes, but rather those past whatever the result ofsend()
is. Initially I thought this was part of the issue, but disabling it doesn't really make a difference.So to solve this, either pyOpenSSL has to grow a buffering layer, or this would have to be something handled in libraries using pyOpenSSL. If the latter, the performance and bandwidth implications of small send()/write() should be documented, at least.