databricks / databricks-sql-python

Databricks SQL Connector for Python
Apache License 2.0
153 stars 89 forks source link

Unable to write list/array type data #377

Open akhileshpachipulusu-inviz-DataScience opened 6 months ago

akhileshpachipulusu-inviz-DataScience commented 6 months ago

I am trying to save a pandas dataframe to a table which contains columns of type list of strings (array). I am doing it using df.to_sql() using sqlalchemy engine. It doesn't seem to be working. I am getting the following error (databricks.sql.exc.NotSupportedError) Could not infer parameter type from value: ['11th gen laptop', 'natural silver laptop', '1tb and 256gb laptop', '6n038pa#acj laptop', 'intel core i3 laptop', 'fhd display laptop', 'intel uhd laptop', '15s-dr3506tu laptop', '8gb ram laptop', 'hp laptop', 'windows 11 home laptop'] - <class 'list'>

When I use use_inline_params=True, I am getting the following error (databricks.sql.exc.ServerOperationError) [UNBOUND_SQL_PARAMETER] Found the unbound parameter

Details: databricks-sql-connector[sqlalchemy]==3.1.1 Python 3.8.10 Databricks Serverless SQL hive_metastore catalog

kravets-levko commented 5 months ago

Hi @akhileshpachipulusu-inviz-DataScience! Can you please provide some more details to help us with debugging:

Thank you!

susodapop commented 3 months ago

When I use use_inline_params=True, I am getting the following error (databricks.sql.exc.ServerOperationError) [UNBOUND_SQL_PARAMETER] Found the unbound parameter

FYI this won't work since SQLAlchemy always uses server bound parameters. If you attempt this with connector version 2.9.3 it may work better for you, since that version of the SQLAlchemy dialect used inline parameters.