coleifer / peewee

a small, expressive orm -- supports postgresql, mysql, sqlite and cockroachdb
http://docs.peewee-orm.com/
MIT License
11.06k stars 1.37k forks source link

Does db.close() exactly close the database connection in network layer? #2896

Closed lizheng2018CC closed 4 months ago

lizheng2018CC commented 4 months ago

When I use a database connection pool, I naturally expect that once a database connection has been used by one thread and is then idle, it can be reused by other threads to speed up connection times and avoid the overhead of establishing new network connections. I've seen the standard usage pattern of connection pools in many places, which is:

def my_thread():
    db.connect()
    # Some operations here
    TempModel.create(data='test data').save()
    db.close()

However, the explanation for the close() API is as follows:

close()[](https://docs.peewee-orm.com/en/latest/peewee/api.html#Database.close)
Returns
Whether a connection was closed. If the database was already closed, this returns False.

Return type
bool

Close the connection to the database.

This easily leads one to mistakenly believe that the close method will close the database connection at the network layer. However, I conducted some tests, including capturing packets with tcpdump and using netstat to query the number of TCP connections to the database, as well as printing len(db._in_use) and len(db._connections). The evidence proved that close() actually recycles the connection back to the connection pool, rather than closing the network connection.

The close() method belongs to the Database class, but I am using PooledMySQLDatabase, which is a subclass of both MySQLDatabase and PooledDatabase. This means I need to look at the grandparent class of PooledMySQLDatabase (Database) to know that the close method exists. The documentation could be clearer about this.

Is my understanding correct?

coleifer commented 4 months ago

The documentation is clear, in my opinion:

The pool works by overriding the methods on the Database class that open and close connections to the backend.

In a single-threaded application, only one connection will be created. It will be continually recycled until either it exceeds the stale timeout or is closed explicitly (using .manual_close()).

By default, all your application needs to do is ensure that connections are closed when you are finished with them, and they will be returned to the pool.

So the pool subclasses explicitly use the same APIs as the regular database implementation, so that they can function as a drop-in replacement. However when you call close() on the pool it will attempt to recycle the connection for later use.