internetarchive / warcprox

WARC writing MITM HTTP/S proxy
380 stars 54 forks source link

Increase proxy_client socket send buffer and use TCP_NODELAY #78

Closed vbanos closed 6 years ago

vbanos commented 6 years ago

Python socket.send buffer has default value 8192. We receive data in big chunks with prox_rec_res.read(65536) but we send them in much smaller chunks (8192) to the client. We should use the same size to improve performance.

We also didn't use TCP_NODELAY for the proxy client and enabling it helps performance.