search

CodyAustinDavis / dbsql_sme

DBSQL SME Repo contains demos, tutorials, blog code, advanced production helper functions and more!
40 stars 16 forks source link

Refactor SQL queries out of the JSON dashboard into individual files #5

Open yegorski opened 2 months ago

yegorski commented 2 months ago

Big fan of this repo @CodyAustinDavis. We're using this repo to create per-warehouse costs monitoring as well as globally in one master dashboard across all warehouses in our workspace.

Diffs like this one https://github.com/CodyAustinDavis/dbsql_sme/commit/e5d6a7000c5d49f296001a4d85fa72741aaa69ac and others make difficult to see what changed.

Perhaps you can have process that stitches the JSON together from individual files, grabbing the "query" blocks from separate SQL files. What do you think?

yegorski commented 2 months ago

Should have clarified that I'm talking about this dashboard https://github.com/CodyAustinDavis/dbsql_sme/blob/main/Observability%20Dashboards%20and%20DBA%20Resources/Observability%20Lakeview%20Dashboard%20Templates/DBSQL%20Warehouse%20Advisor%20Observability%20Dashboard.lvdash.json

CodyAustinDavis commented 2 months ago

@yegorski Glad youre finding these dashboards helpful! I saw you have put in a few issues, would you have some time next week to meet to discuss potential solutions?

CodyAustinDavis commented 2 months ago

We likely wont be putting in all the extra work to reverse engineer the JSON files to decompose and then recompose. However, I am open to the idea of just having the dataset queries as separate entities so you can track the Diffs, but thats harder to do in the UI part of the dashboard.

yegorski commented 2 months ago

@CodyAustinDavis I'd love to meet up and go over how we use the system tables and the queries you put together. If only as feedback on how we use it. Please reach out! yegor.ius@enigma.com