memory leak from psycopg2.connect on macos

[asottile-sentry]

Please complete the following information:

• OS: macos 13.5.2
• Psycopg version: psycopg2-binary==2.9.9
• Python version: 3.11.0, 3.10.13, 3.11.6 (reproduced on each of these)
• PostgreSQL version: 14.9
• pip version 23.3.2

Describe the bug

there appears to be a leak in connect which is not freed by close. here's a simple example (increase the number of iterations to get a more significant leak):

import psycopg2

def main():
    for _ in range(10000):
        psycopg2.connect('user=postgres host=127.0.0.1 dbname=sentry').close()

main()

running this through memray as follows (sidestepping pymalloc to get more accurate memory usage):

$ rm memray-*.bin memray*.html
$ PYTHONMALLOC=malloc memray run --quiet t2.py
$ memray flamegraph --leaks memray-*.bin

here's a screenshot of the memray information (the rise in heap usage is more evident because it starts from a rougly ~0 scale -- but RSS also increased from 40MB to 64MB across the same timeframe):

we have many long running celery tasks which open and close connections via django -- over time this contributes to a significant rise in memory usage

will try and look into what's not being freed, but wanted to throw up what I've found so far at least

[asottile-sentry]

I don't see the same leak on linux so I suspect this is specific to macos

here's some more information when expanding native frames

[dvarrazzo]

Possibly related to https://github.com/psycopg/psycopg/issues/678

And I can't find it anymore, but someone was complaining of slow connection on macOS, and it turned out to be their badly configured gssapi.

gssapi is a transitive dependency of psycopg, brought in by the libpq. You can check if disabling its use helps, see related libpq docs for details.

If you find a bug, please report it upstream (to the postgres or to the kerberos/gssapi projects).

Feel free to reopen the bug if we can change something our side. I don't have a macOS to experiment with.

[asottile-sentry]

indeed -- I narrowed it down to krb5 -- this is sufficient to leak memory on macos:

#include <stdbool.h>
#include <stdio.h>
#include <gssapi.h>

int main(void) {
    for (int i = 0; i < 10; i += 1) {
        gss_cred_id_t cred = 0;
        OM_uint32 minor = 0;
        OM_uint32 ret = gss_acquire_cred(
            &minor, GSS_C_NO_NAME, 0, GSS_C_NO_OID_SET,
            GSS_C_INITIATE, &cred, NULL, NULL
        );
        if (ret == GSS_S_COMPLETE) {
            printf("no error\n");
            gss_release_cred(&minor, &cred);
        } else {
            printf("got error: %d\n", ret);
        }
    }
}

reported it to the krb5-bugs mailing list... hasn't shown up yet though or I'd link it here

[asottile-sentry]

and a small patch I'm sending their way to fix it too :D

$ diff -u lib/krb5/ccache/cc_api_macos.c{.old,}
--- lib/krb5/ccache/cc_api_macos.c.old  2024-01-10 16:14:19
+++ lib/krb5/ccache/cc_api_macos.c  2024-01-10 16:14:43
@@ -218,8 +218,10 @@
         xpc_release(request);
     if (reply != NULL)
         xpc_release(reply);
-    if (conn != NULL)
+    if (conn != NULL) {
         xpc_connection_cancel(conn);
+        xpc_release(conn);
+    }
     return ret;
 }

[dvarrazzo]

Hello @asottile-sentry that's great, thank you for the finding!

If they fix the problem upstream and release a new version with the fix I will make sure to include it in the next binaries packages.

Keep us informed, cheers!

[asottile-sentry]

fix is committed so it should land in the next krb5 release!

[dvarrazzo]

Thank you for your help, @asottile-sentry

[dvarrazzo]

@asottile-sentry Is there a link for your report upstream we can link in our docs and which I can report in https://github.com/psycopg/psycopg/issues/678?

[asottile-sentry]

here's the commit in krb5 https://github.com/krb5/krb5/commit/52fe67623b7205d91ceac855651e8c17f56b10c8

here's the mailing thread https://mailman.mit.edu/pipermail/krb5-bugs/2024-January/013352.html