search

long2ice / asyncmy

A fast asyncio MySQL/MariaDB driver with replication protocol support
https://github.com/long2ice/asyncmy
Apache License 2.0
230 stars 25 forks source link

TCP connection leaks #98

Open dkang-quora opened 2 weeks ago

dkang-quora commented 2 weeks ago

I am having fd leaks in mysql connections.

From the runtime inspection of GC, all Python objects are properly garbage-collected. However the actual TCP connections are still alive.

objgraphql.by_type('socket.socket') only lists the 6 connections:

"<socket.socket fd=742, family=2, type=1, proto=6, laddr=('10.0.0.1', 45160), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=752, family=2, type=1, proto=6, laddr=('10.0.0.1', 42152), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=753, family=2, type=1, proto=6, laddr=('10.0.0.1', 41736), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=762, family=2, type=1, proto=6, laddr=('10.0.0.1', 36052), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=737, family=2, type=1, proto=6, laddr=('10.0.0.1', 52222), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=778, family=2, type=1, proto=6, laddr=('10.0.0.1', 36620), raddr=('10.0.0.2', 3306)>"

but ss or lsof shows many more connections including the old connections: 

...
python  2601911 root  715u  IPv4 305130877      0t0  TCP myserver:43446->dbserver:mysql (ESTABLISHED)
python  2601911 root  718u  IPv4 305132513      0t0  TCP myserver:44024->dbserver:mysql (ESTABLISHED)
python  2601911 root  722u  IPv4 305162297      0t0  TCP myserver:55118->dbserver:mysql (ESTABLISHED)
python  2601911 root  724u  IPv4 305159638      0t0  TCP myserver:52336->dbserver:mysql (ESTABLISHED)
python  2601911 root  729u  IPv4 305159665      0t0  TCP myserver:52374->dbserver:mysql (ESTABLISHED)
python  2601911 root  733u  IPv4 305162876      0t0  TCP myserver:53728->dbserver:mysql (ESTABLISHED)
python  2601911 root  737u  IPv4 305413976      0t0  TCP myserver:52222->dbserver:mysql (ESTABLISHED)
python  2601911 root  742u  IPv4 305169194      0t0  TCP myserver:45160->dbserver:mysql (ESTABLISHED)
python  2601911 root  752u  IPv4 305306885      0t0  TCP myserver:42152->dbserver:mysql (ESTABLISHED)
python  2601911 root  753u  IPv4 305305260      0t0  TCP myserver:41736->dbserver:mysql (ESTABLISHED)
python  2601911 root  762u  IPv4 305400262      0t0  TCP myserver:36052->dbserver:mysql (ESTABLISHED)
python  2601911 root  778u  IPv4 305679092      0t0  TCP myserver:36620->dbserver:mysql (ESTABLISHED)

note the last fd numbers (742, 752, 753, 762, 778) are the same.

I guess there is a remaining refcount to the underlying os socket. Should asyncmy call transport.shutdown() before transport.close() in Connection.close() ?