sprin / pg-discuss

A comment system backend in Python with PostgreSQL
http://pg-discuss.sprin.io/
10 stars 1 forks source link

Diagnose higher than expected process memory , create memory profiler extension #10

Closed sprin closed 8 years ago

sprin commented 9 years ago

When the "got six weeks" thread with 30K+ comments is rendered, the uwsgi process memory can jump from a steady state value of about 40MB RSS to about 110MB. The memory use does seem to be bounded though, as it does not increase beyond about 110MB.

We can create a trivial extension for the Dozer memory profiler middleware to inspect memory use. Hopefully, this will reveal areas for improvement.

sprin commented 8 years ago

The Dozer extension was trivial to implement:

class DozerExt(ext.AppExtBase):
    def init_app(self, app):
        app.wsgi_app = Dozer(app.wsgi_app)

However, Dozer is limited to just reporting object counts. In my case, the issue was with a single list object. I won't be including a Dozer extension in the bundled extensions.