microsoft / semantic-link-labs

Early access to new features for Microsoft Fabric's Semantic Link.
MIT License
178 stars 37 forks source link

run_model_bpa_bulk fails on a semantic model with no tables #259

Closed plorenc closed 1 week ago

plorenc commented 2 weeks ago

run_model_bpa_bulk function fails when there is a semantic model with no tables in a Workspace.

To Reproduce Steps to reproduce the error:

  1. Create empty semantic model in any workspace
  2. run run_model_bpa_bulk with that workspace as a parameter
  3. See error

Expected behavior Function should omit semantic models with no tables.

Version: 0.8.4

m-kovalsky commented 1 week ago
  1. Why do you have a model with no tables? A semantic model with no tables is nonsensical.
  2. Please provide the full error message in text.
plorenc commented 1 week ago
  1. It's true but It's not me, it's the clients who will use the tool by calling the notebooks / pipelines we will provide. It's enough that one of them, in the shared Workspace, creates something that they either haven't developed yet or forgot about altogether and you can no longer call the function with the workspace=None parameter. Now I created a workaround where I iterate through the Workspaces and then Semantic Models (datasets) inside and check myself for the dataset.empty. BTW. From what I see, the parameter that provided the list of semantic models to scan has been removed from the function parameters (but documentation still says it exists - it is another issue reported by me). Thanks to my workaround, I can still limit the function to selected semantic models or filter them out.

I really like your function, but I think it could use a bit more flexibility in how it's called. What do you think? What if we want to skip a semantic model with a given name only in one Workspace and not in others? Wouldn't a structure like {"Workspace1": ["dataset1", "dataset2" ...], "Workspace2": None} be more appropriate, where both the Workspace list of datasets could take the value None? If you want I can help with the solution.

  1. The error log:
⌛ Collecting Model BPA stats for the 'PL_SQL_Database' semantic model within the 'Sandbox_PL' workspace.
🔴 Model BPA failed for the 'PL_SQL_Database' semantic model within the 'Sandbox_PL' workspace.
An error occurred when running AdomdCommand. AdomdCommandActivityId: '158bd7f3-5af6-4f5f-9f68-bd205a9b57bf'

Caused by AdomdErrorResponseException:
The database is empty. The DISCOVER_CALC_DEPENDENCY operation cannot be performed on an empty database.

Technical Details:
RootActivityId: 74752382-3496-400e-8c41-c8182a01608c
Date (UTC): 11/10/2024 9:50:03 AM
   at Microsoft.AnalysisServices.AdomdClient.XmlaClient.CheckForSoapFault(XmlReader reader, XmlaResult xmlaResult, Boolean throwIfError)
   at Microsoft.AnalysisServices.AdomdClient.XmlaClient.CheckForError(XmlReader reader, XmlaResult xmlaResult, Boolean throwIfError)
   at Microsoft.AnalysisServices.AdomdClient.XmlaClient.SendMessage(Boolean endReceivalIfException, Boolean readSession, Boolean readNamespaceCompatibility)
   at Microsoft.AnalysisServices.AdomdClient.XmlaClient.ExecuteStatement(String statement, IDictionary connectionProperties, IDictionary commandProperties, IDataParameterCollection parameters, Boolean isMdx)
   at Microsoft.AnalysisServices.AdomdClient.AdomdConnection.XmlaClientProvider.Microsoft.AnalysisServices.AdomdClient.IExecuteProvider.ExecuteTabular(CommandBehavior behavior, ICommandContentProvider contentProvider, AdomdPropertyCollection commandProperties, IDataParameterCollection parameters)
   at Microsoft.AnalysisServices.AdomdClient.AdomdCommand.ExecuteReader(CommandBehavior behavior)
   at Microsoft.AnalysisServices.AdomdClient.AdomdCommand.ExecuteReader()
   at Microsoft.Fabric.SemanticLink.DAXToParquetWriter.Write(String dax, String fileName, Int32 batchSize, AdomdConnection adomdConnection, Int32 verbose, Nullable`1 maxNumRows, Nullable`1 activityId)

⌛ Saving the Model BPA results of the 'Sandbox_PL' workspace to the 'modelbparesults' within the 'BPA' lakehouse within the 'BPA' workspace...
🟢 The dataframe has been saved as the 'modelbparesults' table in the 'BPA' lakehouse within the 'BPA' workspace.
🟢 Saved BPA results to the 'modelbparesults' delta table.
m-kovalsky commented 1 week ago

Fixed in 0.8.5.