search

wireservice / agate-sql

agate-sql adds SQL read/write support to agate.
https://agate-sql.readthedocs.io
MIT License
18 stars 15 forks source link

Ensure closing connection and disposing SQLAlchemy engine if required #29

Closed smtakeda closed 6 years ago

smtakeda commented 6 years ago

Snowflake DB (https://www.snowflake.net/) is a cloud DB, which requires explicit closing connection and disposing the SQLAlchemy engine object for the clients to tell the server that the connection is no longer required or leaks sessions. Although the orphaned sessions will be purged eventually in the server side, it would be better for the clients to close them explicitly if no longer required.

This PR is an improvements in agatesql/table to ensure the connections are closed and the engine is destroyed if they are allocated within the methods.

smtakeda commented 6 years ago

Additional change in the same branch was accidentally added... https://github.com/wireservice/agate-sql/pull/29/commits/5c75987d4f9252062e4c7c0a9a6ebf0fe10410b6

That is to add a support of chunksize parameter to to_sql so that the large set of rows can be inserted by multiple batches.

This change will mitigate https://github.com/snowflakedb/snowflake-connector-python/issues/37

jpmckinney commented 6 years ago

Thanks!