Recommendation: Pin the SDK <=0.19.0 in your pyproject.toml. Looks like you're currently pulling in from main, which will allow updates to happen automatically past 0.19.0; I'd recommend altering that spec to <=0.19.0.
Please take a look at the PR description for a deeper discussion. Here is a summary of what's changed.
Overview:
The SQLAlchemy connection on the SQLConnector is now protected, and belongs only to the SQLConnector or it's subclasses. Streams and other objects/functions should stop accessing the SQLConnector's engine or connection.
Within the SQLConnector, DB connections are opened in only one way: by using the _connect() method as a context manager.
The SQLConnector uses a single engine throughout its lifetime. Connections are opened as needed and closed after each operation.
Specific code changes:
_connection is gone altogether. You cannot pass it to SQLConnector and you cannot access it off of the connector. We are not caching or passing around an open DB connection anywhere anymore.
create_sqlalchemy_engine and create_sqlalchemy_connection have been altered to use the single cached engine, and are also deprecated. Ideally, subclasses & other objects would stop using those methods.
Suggestions:
If you have any database-interacting logic that happens outside of the SQLConnector (eg if your SQLStream subclass has a get_records method that gets the SQLConnector.connection and does something with it), please move that logic onto a method on your subclass of SQLConnector and call that method from the Stream/other object.
Wherever you're accessing the connection on SQLConnector/a subclass, please do so by using the _connect() method as a context manager. It's directly equivalent to SQLAlchemy.Engine.connect(). Here's an example of its use.
This PR https://github.com/meltano/sdk/pull/1394 makes some small breaking changes to SQLConnector.
Recommendation: Pin the SDK <=0.19.0 in your pyproject.toml. Looks like you're currently pulling in from main, which will allow updates to happen automatically past 0.19.0; I'd recommend altering that spec to
<=0.19.0.Please take a look at the PR description for a deeper discussion. Here is a summary of what's changed.
Overview:
_connect()method as a context manager.Specific code changes:
_connectionis gone altogether. You cannot pass it to SQLConnector and you cannot access it off of the connector. We are not caching or passing around an open DB connection anywhere anymore.create_sqlalchemy_engineandcreate_sqlalchemy_connectionhave been altered to use the single cached engine, and are also deprecated. Ideally, subclasses & other objects would stop using those methods.Suggestions:
_connect()method as a context manager. It's directly equivalent to SQLAlchemy.Engine.connect(). Here's an example of its use.