relferreira / metabase-sparksql-databricks-driver

GNU Affero General Public License v3.0
31 stars 32 forks source link

Error trying to browse data in tables with UC #34

Open dgpoulet opened 9 months ago

dgpoulet commented 9 months ago

I am trying to connect metabase to a Unity Catalog enabled workspace via a SQL Warehouse. I can connect to Databricks okay, and use JDBC params to select the correct catalog/schema. The sync process reads the metadata from all the tables in the schema correctly, but any attempt to browse the data in those tables throws an error. e.g. for an address table:

[Databricks]DatabricksJDBCDriver ERROR processing query/statement. Error Code: 0, SQL state: org.apache.hive.service.cli.HiveSQLException: Error running query: [PARSE_SYNTAX_ERROR] org.apache.spark.sql.catalyst.parser.ParseException: [PARSE_SYNTAX_ERROR] Syntax error at or near '.'.(line 2, pos 380)

== SQL == -- Metabase:: userID: 1 queryType: MBQL queryHash: 06e705add30da5f0a6d7ad3f3460a662e536478e4698889d4bbcc399178f05a1 SELECT `t1`.`AddressID` AS `AddressID`, `t1`.`AddressLine1` AS `AddressLine1`, `t1`.`AddressLine2` AS `AddressLine2`, `t1`.`City` AS `City`, `t1`.`StateProvince` AS `StateProvince`, `t1`.`CountryRegion` AS `CountryRegion`, `t1`.`PostalCode` AS `PostalCode`, `t1`.`rowguid` AS `rowguid`, `t1`.`ModifiedDate` AS `ModifiedDate` FROM `saleslt`.`address` AS `metabase.util.honey_sql_2`.`identifier` `table-alias` (decode(unhex('7431'), 'utf-8')) LIMIT 2000

The part it seems to be complaining about a syntax error is below, but I'm not sure if that's relevant or not:

... `metabase.util.honey_sql_2`.`identifier`

I've defined the connector params like so:

;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/warehouses/\<WAREHOUSE>; ConnCatalog=\<CATALOG>;ConnSchema=\<SCHEMA>;UID=token;PWD=\<TOKEN>;

Any clues as to the issue here?

shrodingers commented 9 months ago

@dgpoulet , there was an issue at some place in the connector (fixed in the spark one), causing this error. Made a fork that seems to work with some changes tied with UC (making it non-backwards compatible per se). It's here if you wanna take a look

AlexVialaBellander commented 9 months ago

@shrodingers, you should enable discussions or issues on your repo <3

shrodingers commented 9 months ago

@shrodingers, you should enable discussions or issues on your repo <3

Wow thanks the advice thought it was 🤦🏻 ^^