Open edwardsdm opened 1 month ago
At least part of the problem appears to be that the extension is trying to convert column types to an internal type:
But it does not have a mapping for "varchar(xxxx)", so it throws an exception:
Scanning our (databricks) system.information_schema.columns for unique full_data_types
select distinct data_type, full_data_type from system.information_schema.columns where data_type not in ('DECIMAL', 'STRUCT', 'ARRAY', 'MAP') order by data_type, full_data_type
It looks like you might also need to map:
varchar(xxxx)
char(xxx)
Maybe also the 'unsized' versions for extra diligence?
char
varchar
Hi @edwardsdm, thanks for reporting this. We will take a look at this at some point. Currently we have prioritized the development of the Delta extension over the uc_catalog extension so I can not give a timeline yet!
The information below has been share with the Databricks UC engineering team. They suggested the issue is likely with the DuckDB UC extension.
Credential serving is enabled on the Databricks instance the code is connecting to.
Duckdb CLI installed on MacOS - Retry SHOW TABLES query
R - Retry SHOW TABLES query
Python - Retry SHOW TABLES query
Additionally even after a successful SHOW TABLES query no tables are found and the table queried successfully in R cannot be queried using Python.