search

databricks / cli

Databricks CLI
Other
132 stars 50 forks source link

Feature Request: Please add SQL Query and SQL Alert support to Asset Bundle #1615

Open litan1106 opened 2 months ago

litan1106 commented 2 months ago

Hi, please add SQL query and SQL alert support to the asset bundle in addition to the job, and pipeline type.

With sql_task, it makes more sense to use alert creation and query creation. The asset bundle provides the best DEVOPS experience while keeping everything in source control.

image

pietern commented 2 months ago

Thanks for creating the issue.

We do not plan to add these in the next ~2 months, but we will consider it beyond that.

Let's leave the issue open to gather +1s from other folks who would like to see this happen.

Lonache commented 3 weeks ago

Frequently we build (materialized) views in conjunction with jobs, and the refresh cycles are easiest to set up if this is done in the same repo and coordinated. I would, if I could, write idempotent SQL queries for the views, and I'd like to use sql_task syntax that @litan1106 suggested.

This is something I think a lot of us could use - or would use. Or should use. I mean, creating the views by hand would be possible. But it's messy to be doing some things by hand and others not.

@pietern We're all onboard with following best DevOps practice, right?

pietern commented 3 weeks ago

@Lonache The SQL queries and alerts that @litan1106 is referring to are proper resources.

If you're looking to manage SQL queries in text files, you can do that today with the sql_file_task.

You could check out the default-sql bundle template for an example on how to structure this.

To update views you can use CREATE OR REPLACE in the query text.

litan1106 commented 2 weeks ago

@Lonache The SQL queries and alerts that @litan1106 is referring to are proper resources.

If you're looking to manage SQL queries in text files, you can do that today with the sql_file_task.

You could check out the default-sql bundle template for an example on how to structure this.

To update views you can use CREATE OR REPLACE in the query text.

even if we can use the sql_file_task to create the view for the SQL alert, we still want to create the SQL alert with the asset bundle so we can deploy all at once instead of manually create alert in the UI.

Escoto commented 2 weeks ago

Is there any timeline already to incorporate SQL Alerts into Bundles?.. My team is evaluating if we need to invest time into Terraform or is this already in the roadmap?

pietern commented 2 weeks ago

There is no timeline for SQL Alerts yet. First, we're working on adding support for AI/BI dashboards.

We'll definitely consider adding this and I agree it makes sense to do so. It is a matter of prioritization.

hashemi-soroush commented 1 week ago

Can you share the timeline on the dashboards, @pietern ?