pudo / dataset

Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.
https://dataset.readthedocs.org/
MIT License
4.76k stars 297 forks source link

Make sure to also close connection #367

Closed TheGU closed 3 years ago

TheGU commented 3 years ago

After test and investigate why dataset not release db connection, even after call close() or engine.dispose(), from below note.

https://dataset.readthedocs.io/en/latest/api.html#notes

Note: dataset uses SQLAlchemy connection pooling when connecting to the database. There is no way of explicitly clearing or shutting down the connections, other than having the dataset instance garbage collected.

I found that because it still has connection open. From sqlalchemy document.

The Engine.connect() method returns a Connection object, and by using it in a Python context manager (e.g. the with: statement) the Connection.close() method is automatically invoked at the end of the block.

and code comment sqlalchemy/engine/base.py#L48

The Connection object represents a single DBAPI connection checked out
from the connection pool. In this state, the connection pool has no affect
upon the connection, including its expiration or timeout state. For the
connection pool to properly manage connections, connections should be
returned to the connection pool (i.e. ``connection.close()``) whenever the
connection is not in use.

it's need to call .close() for any open connection to make sure connection close.

TheGU commented 3 years ago

Try to solve the same issue with #364 but different approach. Not sure which one should be use in which case. In this PR, I just want to close local connection to make sure dispose process work like expect. However, #364 track all connection then close all of them when call close() function.