A data pipeline orchestration library for rapid iterative development with automatic cache invalidation allowing users to focus writing their tasks in pandas, polars, sqlalchemy, ibis, and alike.
When using pipedag_config.get('instance') function with an IBM-DB2 that needs to be connected to. There is the following error, when SQLAlchemy is below version 2.0.0:
I only logged the actuall error outputs.
TypeError: Invalid argument(s) 'isolation_level' sent to create_engine(), using configuration DB2Dialect_ibm_db/QueuePool/Engine. Please check that the keyword arguments are appropriate for this combination of components.
RuntimeError: Failed loading table_store
RuntimeError: Error while creating backend objects from pipedag config (instance=mini, flow=None): /home/ubuntu/hf4-claims-exploration/src/hf4_claims_exploration/pipeline/pipedag.yaml
To my understanding, when trying to call the create_engine() function of sqlalchemy, with an specific isolation_level for versions below 2.0.0 it can lead to incompatibility issues. This is fixed in versions > 2.0.0 as noted here. I simply upgraded the sqlalchemy version and then it worked. Does this incompatibility maybe warrant setting the SQLAlchemy dependency to > 2.0.0 in the pyproject.toml?
When using
pipedag_config.get('instance')function with an IBM-DB2 that needs to be connected to. There is the following error, when SQLAlchemy is below version 2.0.0:I only logged the actuall error outputs.
To my understanding, when trying to call the create_engine() function of sqlalchemy, with an specific
isolation_levelfor versions below 2.0.0 it can lead to incompatibility issues. This is fixed in versions > 2.0.0 as noted here. I simply upgraded the sqlalchemy version and then it worked. Does this incompatibility maybe warrant setting the SQLAlchemy dependency to > 2.0.0 in the pyproject.toml?