- What I did
Improved the way the database session is created using a contextmanager, that way the session is automatically closed on each paginated query, and added a final dispose call to make sure all connections have been closed.
Also increased the connection poll size.
- How I did it
Used the contextmanager library and session.close and dispose logic.
- How to verify it
Run the connector in a hive metastore with 15000+ tables.
- Description for the changelog
This is a follow up optimization from the PR #23. Improves session clean up and connection pool handling.
- What I did Improved the way the database session is created using a contextmanager, that way the session is automatically closed on each paginated query, and added a final
disposecall to make sure all connections have been closed. Also increased the connection poll size.- How I did it Used the
contextmanagerlibrary andsession.closeanddisposelogic.- How to verify it Run the connector in a hive metastore with 15000+ tables.
- Description for the changelog This is a follow up optimization from the PR #23. Improves session clean up and connection pool handling.