uc-04-system-tables uses hardcoded UUID

[zcking]

Every time I attempt to install the demo uc-04-system-tables I receive the following exception:

Installation error: '49187002-01a8-4abf-b140-7795957779b3'
Couldn't create or update a dashboard.
If this is a permission error, we recommend you to search the existing dashboard and delete it manually.
You can skip the dashboard installation with skip_dashboards = True:
dbdemos.install('uc-04-system-tables', skip_dashboards = True)

Of course skip_dashboards=True does allow the rest of the demo to install successfully, but the dashboards are the most important part of this demo. I tried deleting the entire folder for the dashboards multiple times.

It seems that the ID 49187002-01a8-4abf-b140-7795957779b3 is hard-coded somewhere in the demo?

---

Tail stacktrace:

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-fe1d2c16-4416-444d-be18-03dc0e27f7a5/lib/python3.10/site-packages/dbsqlclone/utils/load_dashboard.py:234, in duplicate_dashboard.<locals>.load_widget(widget)
    232     query_id = widget["visualization"]["query"]["id"]
    233     visualization_id = widget["visualization"]["id"]
--> 234     visualization_id_clone = dashboard_state["queries"][query_id]["visualizations"][visualization_id]
    235 data = {
    236     "dashboard_id": new_dashboard["id"],
    237     "visualization_id": visualization_id_clone,
   (...)
    240     "width": widget["width"]
    241 }
    242 time.sleep(sleep_between_queries)

KeyError: '49187002-01a8-4abf-b140-7795957779b3'

---

I viewed the dashboard JSON from within the site-packages bundles and confirmed the 4918... UUID appears as the ID of one of the queries:

...
          "id": "49187002-01a8-4abf-b140-7795957779b3",
          "name": "System Tables - top workspaces",
...

[zcking]

This may be a bug with the JSON for two of the queries in the dashboard JSON file. I had to create the following two queries by hand, and after doing so, the dbdemos.install() worked.

• System Tables - top workspaces
  • Note: In the JSON, it contains visualizations that reference columns which are not present in the query's actual result set
• System Tables - Forecast DBUs

[chhayavishwakarma]

I have exact same issue.

@zcking - how did you getting working ? could you help with more detailed steps ?

[QuentinAmbard]

hi there - sorry I was off these last days. Looking at that now...

[QuentinAmbard]

@zcking @chhayavishwakarma could you try again and let me know?

[zcking]

@QuentinAmbard thanks, it works now. I ran it on a different account and workspace for a clean slate.

And I noticed the following output during installation, which makes sense:

Installing demo uc-04-system-tables under /Users/.../dbdemos, please wait... Your dashboard has widgets. Widget queries need to be run first to be able to load other queries. DBSQL-CLONE will now submit a job run to start the param queries. This will take a few minutes without serverless (few sec with serverless)... You can check the progress in https://REDACTED.cloud.databricks.com#job/runs Waiting for parameter queries to run as they're needed to import the next queries ({'run_id': 636})... Waiting for parameter queries to run as they're needed to import the next queries ({'run_id': 636})... Waiting for parameter queries to run as they're needed to import the next queries ({'run_id': 636})... Waiting for parameter queries to run as they're needed to import the next queries ({'run_id': 636})... Waiting for parameter queries to run as they're needed to import the next queries ({'run_id': 636})... Waiting for parameter queries to run as they're needed to import the next queries ({'run_id': 636})... Param queries initialization successful. Resume dashboard import...

[chhayavishwakarma]

@zcking @chhayavishwakarma could you try again and let me know?

Thanks @QuentinAmbard I was able to run the demo, but many of the dashboard queries ain't running due init_job failure

notebook - 02-forecast-billing-tables

@zcking how's it for you ?

[zcking]

@chhayavishwakarma ah yes you're right there are still some issues with the queries. Here is another example of an incorrect query from the dashboard I encountered:

Total $DBU consumption for selected date and Workspace

[UNRESOLVED_COLUMN.WITH_SUGGESTION] A column or function parameter with name ds cannot be resolved. Did you mean one of the following? [main.billing_forecast.billing_forecast.sku, main.billing_forecast.billing_forecast.workspace_id].; line 4, pos 52

[QuentinAmbard]

Hi @zcking , can you try with the fix in #45 and let me know if this fixes your issue?