Interactions between pbi-tools and Power BI REST APIs with public XMLA endpoint restrictions

[bradsmucker]

Hello,

In our current Power BI embedded capacity design, we effectively have "source" copies of semantic models as .pbip in TMDL format.

Our application calls the Reports - Export Report In Group API to export the semantic model to a local file system as a .pbix file, then publish that model only, not the report, to a customer workspace.

My confusion comes from this line on restrictions and limitations when downloading .pbix semantic models:

Semantic models modified by using the XMLA endpoint can't be downloaded to a .pbix file.

Additionally, from the Datasets - Update Parameters API, the following limitation is also noted:

Datasets created or modified using the public XMLA endpoint aren't supported. To make changes to those datasets, the admin must use the Azure Analysis Services client library for Tabular Object Model.

In our CI/CD process, would using pbi-tools with a deployment manifest to update a dataset parameter in a "source" semantic model and deploy it to a customer workspace prevent the Reports - Export Report In Group API from exporting the "source" model as a .pbix or the Datasets - Update Parameters API from updating a dataset parameter, even if the semantic model is a .pbip in TMDL format?

Thank you for any information you that you can provide.

[mthierba]

Hi @bradsmucker ,

that's a great question - let me clarify. With pbi-tools in place, you'd generally adopt a different approach for your lifecycle management of model and report assets.

Implementing pbi-tools means your git repositories become the source of truth for all your PBI/Fabric items. Power BI/Fabric workspaces, in that setup, are merely deployment targets, and you'll never (need to) download anything from a workspace - that's where your git repository comes it.

With your chosen git workflow, you ensure that a particular branch and/or tag in your source control system, by definition, always represents "Production", i.e. the official, approved version of all those items. "Downloading", in that context, means for someone to clone/check out the repository to a local folder (which comes with no technical restrictions, merely your security setup).

Hence, any download limitations that may exist in any of your PBI/Fabric workspaces become completely irrelevant and are no longer a limitation.

That's one of the (many) advantages of switching to a pbi-tools workflow.

[bradsmucker]

Hi @mthierba , thank you for your response!

Are there any other PBI/Fabric workspace limitations that I should be aware of when it comes to deploying semantic models? For example, we need to update a PowerQuery parameter, but the dataset is owned by a service principal profile and not the service principal itself as defined in the deployment manifest.

Datasets - Update Parameters API requires the user to be the owner of the dataset, and I wasn't sure if that applied to pbi-tools/XMLA endpoints as well.

After adding compatibilityMode: powerBI to my model.tmdl file and giving my database a name in the database.tmdl file, as described in #326 , I am now encountering the following error that appears to happen when trying to update the parameter. It says that the model has been successfully deployed, but the dataset parameter remains the old value in the service.

Model deployment succeeded. An unhandled exception occurred. System.AggregateException: One or more errors occurred. (Encountered malformed markup tag at position 10.) ---> System.InvalidOperationException: Encountered malformed markup tag at position 10. at Spectre.Console.MarkupTokenizer.ReadMarkup() in //src/Spectre.Console/Internal/Text/Markup/MarkupTokenizer.cs:line 192 at Spectre.Console.MarkupTokenizer.MoveNext() in //src/Spectre.Console/Internal/Text/Markup/MarkupTokenizer.cs:line 27 at Spectre.Console.MarkupParser.Parse(String text, Style style) in //src/Spectre.Console/Internal/Text/Markup/MarkupParser.cs:line 17 at Spectre.Console.Markup..ctor(String text, Style style) in //src/Spectre.Console/Widgets/Markup.cs:line 42 at Spectre.Console.TableExtensions.<>c.b__60(String column) in //src/Spectre.Console/Extensions/TableExtensions.cs:line 160 at System.Linq.Enumerable.SelectArrayIterator`2.ToArray() at Spectre.Console.TableExtensions.AddRow(Table table, String[] columns) in /_/src/Spectre.Console/Extensions/TableExtensions.cs:line 160 at PbiTools.Deployments.DeploymentManager.ReportPartitionStatus(Model model, ConsoleOptions consoleOptions) in X:\pbi-tools\BRANCHES\rc.8\src\PBI-Tools\Deployments\DeploymentManager.Dataset.cs:line 469 at PbiTools.Deployments.DeploymentManager.DeployDatasetAsync(PbiDeploymentManifest manifest, String label, String environment) in X:\pbi-tools\BRANCHES\rc.8\src\PBI-Tools\Deployments\DeploymentManager.Dataset.cs:line 294 at PbiTools.Deployments.DeploymentManager.DeployAsync(String profileName, String environment) in X:\pbi-tools\BRANCHES\rc.8\src\PBI-Tools\Deployments\DeploymentManager.cs:line 91 --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at PbiTools.Cli.CmdLineActions.Deploy(String folder, String label, String environment, String basePath, Boolean whatIf) in X:\pbi-tools\BRANCHES\rc.8\src\PBI-Tools\Cli\Deploy.cs:line 41 --- End of stack trace from previous location --- at PowerArgs.ArgAction.Invoke() at PbiTools.Program.Main(String[] args) in X:\pbi-tools\BRANCHES\rc.8\src\PBI-Tools\Program.cs:line 104

Is this issue related to limitations on the Update Parameters API or PBI/Fabric, or is this a different, unrelated issue?

[mthierba]

The error you quoted is very unfortunate as it is a bug in the logging code, not the deployment (which actually succeeded, as you can see). I'd love to fix that, might need to come back to you for some clarification how to reproduce the bug. - I'm assuming one of your tables/partitions contains some special characters in their name?

pbi-tools currently does not have explicit support for SPN profiles. However, I've done a bit of research into the tech behind, and think it wouldn't be hard to add. What is your use case here? How would you want to use pbi-tools in a multi-tenancy setup?

[bradsmucker]

Hi Mathias,

We are deploying an app-owns-data, embedded capacity solution where each customer that we onboard has their own copy of our standardized "source" semantic models in their individual workspaces. In order to work around the service principal limit of 1000 workspace assignments, we are using service principal profiles to create and populate these workspaces so that the customer's unique service principal profile owns their datasets and the service principal itself isn't assigned their workspace.

Additionally, we are using a unique dataset parameter that needs set individually for each customer to implement row-level security. Based on my understanding of the Datasets - Update Parameters API, only the dataset owner can update dataset parameters. I don't know if this limitation applies when using pbi-tools, but we currently need to make additional calls to change ownership of semantic models when we deploy them via Power BI deployment pipelines in order to set the parameter.

With the limitation of one workspace to one Power BI deployment pipeline stage, and therefore one workspace per pipeline, our ability to broadcast a dataset out to hundreds of customers is severely limited and heavily serialized. Our hope is that pbi-tools could help us parallelize the deployments, but the current design means that we would need to add and remove the service principal performing the pbi-tools deployment to each customer workspace instead of using the service principal profile associated with that workspace.