search

relferreira / metabase-sparksql-databricks-driver

GNU Affero General Public License v3.0
30 stars 32 forks source link

Use multiple databases at the same as different "Schemas" #13

Open Esya opened 1 year ago

Esya commented 1 year ago

Hi, is it possible to use a single connection (Or register databricks only once) in metabase, but then use the databases inside Databricks just like Schemas in PostgreSQL for example ?

Other BI tools do this, even when connecting to Databricks, allowing for joins between different databases. We split our databases per usecases, but also have a "models" database that has all of our key business entities (while the aggregates and key metrics live in other databases).

alxsbn commented 1 year ago

I have almost the same question but when using Unity catalog

bjgbeelen commented 1 year ago

@Esya To my knowledge this is not possible. Perhaps, I could imagine having multiple databases, one per department, which contains all the tables for their use cases. That would solve the need to join from multiple databases.

Multiple connections is indeed annyoing. I hope in the future to overcome this with a Metabase terraform provider to organize multiple connections (including automatic token rotation).

@alxsbn I haven't had any experience yet with Unity catalog, so can't help you

alxsbn commented 1 year ago

@alxsbn I haven't had any experience yet with Unity catalog, so can't help you

It's working with ConnCatalog=<catalog-name>;ConnSchema=<schema-name> in the jdbc URL.

Details here https://docs.databricks.com/integrations/jdbc-odbc-bi.html#default-catalog-and-schema

bjgbeelen commented 1 year ago

@alxsbn so just to get it clear, you have configured a connection to unity catalog in metabase using the above moentioned extra parameters and this allows you to access tables from different schemas using a single connection? Because then I'm definitely going to further investigate the unity catalog!

alxsbn commented 1 year ago

@bjgbeelen Yes you just need to setup the parameter then you will be able to do cross-schema queries through Metabase. It's not working among multiple catalogs fyi

harlemmuniz commented 7 months ago

@alxsbn I couldn't bring all schemas and tables from a catalog, I need to specify here the ConnSchema. The database was successfully saved, but the database is empty. Checking the logs the metabase was in a looping doing some GET requests in the database until stopped it.