databricks / databricks-sql-python

Databricks SQL Connector for Python
Apache License 2.0
152 stars 87 forks source link

sqlalchemy.types has no attribute 'Uuid' #291

Closed dfinchdb closed 9 months ago

dfinchdb commented 9 months ago

After release of databricks-sql-connector 3.0.0 receiving error sqlalchemy.types has no attribute 'Uuid' when using:

from langchain.sqldatabase import SQLDatabase db = SQLDatabase.from_databricks()

Downgrading to version 2.9.3 resolves the error. Multiple reports of this issue. Additional report that downgrading sqlalchemy to version 2.0.12 resolved error.

susodapop commented 9 months ago

What version of SQLAlchemy do you have installled?

dfinchdb commented 9 months ago

Reporting issue on behalf of a customer, and I don't have that information. You're asking for the sqlalchemy version for the working environment?

I did notice that: databricks-sql-connector==3.0.0 + langchain==0.0.341 results in sqlalchemy==2.0.23 databricks-sql-connector==2.9.3 + langchain==0.0.341 results in sqlalchemy==1.4.50; databricks-sql-connector==2.9.3 alone results in sqlalchemy==1.4.50

My guess is that the working version for them was: databricks-sql-connector==2.9.3 langchain==0.0.341 sqlalchemy==1.4.50

The underlying difference seems to be the sqlalchemy version. Another user reported that they had success with sqlalchemy==2.0.12. I know that this seems to indicate the actual problem is outside of the databricks-sql-connector library, but I'm hoping that we can find some compromise since langchain is a popular library right now.

susodapop commented 9 months ago

Okay I've researched this a bit more. SQLAlchemy inroduced the Uuid type in sqlalchemy==2.0.0. The databricks dialect in databricks-sql-connector==3.0.0 is compatible with SQLAlchemy>=2.0.0. So the ImportError here happens because the environment you're running in uses SQLAlchemy 1.x.

I can reproduce this with the following:

from langchain.sql_database import SQLDatabase

uri = "databricks://token:dapi****@****.cloud.databricks.com?http_path=/sql/1.0/warehouses/****&catalog=***&schema=***"
dbt = SQLDatabase.from_uri(uri)
print("it worked!")

With SQLAlchemy==1.4.48 installed, I receive:

AttributeError: module 'sqlalchemy.types' has no attribute 'Uuid'

But with any SQLAlchemy>2.0.0 I receive no such error. I see no evidence that there is a difference between 2.0.12 and 2.0.23.

but I'm hoping that we can find some compromise since langchain is a popular library right now.

Totally agree. langchain has no specific SQLAlchemy dependency. So if you run pip install langchain databricks-sql-connector you will end up with compatible versions of every dependency. The root of the problem here, I suspect, is that users are trying to run langchain in an environment with a pre-existing SQLAlchemy installation that is too old (below version 2.0.0).

Fallinggator commented 9 months ago

Thank you for going the extra mile Jesse.