search

cherrypy / cheroot

Cheroot is the high-performance, pure-Python HTTP server used by CherryPy. Docs -->
https://cheroot.cherrypy.dev
BSD 3-Clause "New" or "Revised" License
185 stars 90 forks source link

Memory leak when using BuiltinSSLAdapter #647

Open bmourgues opened 9 months ago

bmourgues commented 9 months ago

❓ I'm submitting a ...

🐞 Describe the bug. What is the current behavior? When enabling ssl_adapter on WSGIServer, there is a memory leak each time a connection is received

❓ What is the motivation / use case for changing the behavior?

💡 To Reproduce Using example code from cheroot with added ssl_adapter allows to reproduce the bug

Steps to reproduce the behavior:

  1. Run this server: 

    from cheroot.wsgi import WSGIServer
from cheroot.server import get_ssl_adapter_class

def my_crazy_app(environ, start_response):
    status = '200 OK'
    response_headers = [('Content-type','text/plain')]
    start_response(status, response_headers)
    return [b'Hello world!']

addr = '0.0.0.0', 8078
ssl_cert='/etc/shinken/certs/server.cert'
ssl_key='/etc/shinken/certs/server.key'

server = WSGIServer(addr, my_crazy_app)

server.ssl_adapter = get_ssl_adapter_class()(certificate=ssl_cert, private_key=ssl_key)

server.start()

  2. Make some requests: for I in $(seq 2000); do wget --quiet -O /dev/null --no-check-certificate https://localhost:8078 ; done

  3. See error: read RSS value from following command each time requests are made to server ps wu -p $(pgrep cheroot_test) RSS value increases and never goes down.

💡 Expected behavior Stable memory consumption for a server usage

📋 Details

📋 Environment

📋 Additional context

OpenSSL has been compiled from https://www.openssl.org/source/openssl-3.0.13.tar.gz without any special option (except install prefix)

Python has been compiled from source https://www.python.org/ftp/python/3.11.8/Python-3.11.8.tgz with following options

./configure --prefix=/opt/shinken/python311 --enable-optimizations --with-openssl=/opt/shinken/openssl/ --with-openssl-rpath=/opt/shinken/openssl/lib64 --with-ssl-default-suites=openssl --without-static-libpython --disable-test-modules --enable-shared LDFLAGS="-L/opt/shinken/openssl/lib64 -Wl,-rpath=/opt/shinken/python311/lib"

If you need more information or if I can help anyway tell me

webknjaz commented 9 months ago

Hello. Is this reproducible with a distro-default Python in any official/public container image? I'd rather not have a repro dependent on compiling CPython from scratch.

Is there a console log output that you can share?

bmourgues commented 9 months ago

Hello,

you can reproduce this issue on Debian bookworm (12.5)

I also managed to reproduce the same issue with cheroot version 10.0.0 in a virtual environment on the same system.

No log or console log, as everything works fine, but this memory leak

If needed, you can generate your certificate with following command : openssl req -x509 -newkey rsa:2048 -nodes -keyout server.key -days 365 -out server.crt

bmourgues commented 9 months ago

After further investigation the leak appeared in version 8.3.0 Older versions do not seem to leak.

We suspect BuiltinSSLAdapter.get_environ(self, sock) method to be the root cause. Removing all code in this method except the first dict creation, seems to solve leak issue.

I hope this may help you to target the problem ...

jeffvanvoorst commented 3 months ago

Please see the corresponding issue in the cpython repo: https://github.com/python/cpython/issues/116810

webknjaz commented 3 months ago

Thanks for the hint! Also linking the corresponding PR: https://github.com/python/cpython/pull/123249.