databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
447 stars 385 forks source link

[FEATURE] Export JDBC and HTTP path parameters for `databricks_cluster` #1896

Open fuselessmatt opened 1 year ago

fuselessmatt commented 1 year ago

For SQL Warehouse it is easy to export HTTP, JDBC, ODBC directly from the resource.

databricks_sql_endpoint.tableau_sql_warehouse.jdbc_url
databricks_sql_endpoint.tableau_sql_warehouse.odbc_params[0]

This is very useful for exporting tokens to Secret Manager etc.

Similar to SQL Warehouses, the connection parameters for each cluster are unique. It would be nice if cluster parameters could be exported in a similar fashion to SQL Warehouse parameters.

Use-cases

The end goal is to enable exporting these parameters to services like Secret Manager and Parameter Store. It would be nice if it could be done in a similar fashion for clusters as for warehouses.

Attempted Solutions

To be honest, there does not seem to be a way to handle this in Terraform. You have to find another way to do this. What is worse is that the API does not seem to include the information we need.

https://docs.databricks.com/dev-tools/api/latest/clusters.html#get

Proposal

Add it to the backend and then add it to the resource

References

Did not find any similar issues

nfx commented 1 year ago

Thank you for the feature request! Currently, the team operates in a limited capacity, carefully prioritizing, and we cannot provide a timeline to implement this feature. Please make a Pull Request if you'd like to see this feature sooner, and we'll guide you through the journey.

fuselessmatt commented 1 year ago

@nfx : But we can't create in the Terraform provider alone as this isn't part of the API? Otherwise it would be a fun exercise for me to implement it as a PR.

Can you CC this to the API team?

pstrasser commented 1 year ago

I think you can get all the necessary information for the JDBC url by querying the clusters url and its cluster_id and combining it to a JDBC connection string

alexott commented 1 year ago

One of the challenges with supporting this on clusters is that workspace ID is a part of the connection string and we can't easily get this information from APIs