microsoft / coe-starter-kit

Other
742 stars 218 forks source link

[CoE Starter Kit - BUG] Power BI -- Custom connectors and App Connection Reference #1048

Closed FacilyDev closed 3 years ago

FacilyDev commented 3 years ago

Describe the issue

  1. In the August release of PBI CoE dashboard (CDS version) on the Custom connectors page there is no table containing... [Custom connectors]
  2. Custom connectors are not loaded into [admin_connectionreferences] that ultimately becomes App Connection Reference (e.g. in my environment Connectors table shows 18 custom connectors but do not have any references to the apps, however, when read from PowerShell: Get-AdminPowerApp and loop through Internal.properties.connectionReferences the same custom connector is bound to 4 apps; those Apps are visible in CoE Dashboard and this custom connector is also visible in Connector table but the link (row containing both) is missing in App Connection Reference).

Expected Behavior

  1. Custom connectors detail table should be added to [Custom Connectors] report page in CoE PowerBI dashboard.
  2. Data load to admin_connectionreferences should be corrected to include custom connectors/apps links (if you could tell which flow loads app/connector references I could also take a look and find the cause).

What solution are you experiencing the issue with?

Core

What solution version are you using?

2.75

What app or flow are you having the issue with?

Power BI dashboard

Steps To Reproduce

No response

Anything else?

No response

JeneferM-MSFT commented 3 years ago

Hello. I'm unsure if you are experiencing issues at the data or Power BI level. Can you please go to Power Platform Admin View, the Model Driven App in Core, and look up a custom connector, and let me know if the apps and flows are associated? image

FacilyDev commented 3 years ago

Hi, It's data -- in the app you mentioned for the custom connector there are no flows nor apps. I guess I found the step, please see below -- maybe there was a reason behind it...

image

JeneferM-MSFT commented 3 years ago

We have to do the regular and custom connectors separately, below the forall in your picture you should see another for custom connectors:

image

I do not repro, so can you please trigger this flow for the environment you expect to see an app associated with a custom connector and share what this flow looks like?

FacilyDev commented 3 years ago

Hi Jenefer, you're right steps below also custom connectors should be also included in admin_connectionreferences. 1) I think the issue must be connected to some ordinary technical thing like throttling restrictions because in my "parallel" BI report (made on PowerShell exports) I can see that in AppConnector table I have 84k rows while in CoE "only" 3k... Additionally, in CoE those specific Apps have the field admin_usescustomapi=true. Do you know how to verify it? No flow error received...

2) Sorry but I don't know how to do what you asked for 🤷‍♂️ if you could guide me (or indicate a link) how to troubleshoot this flow -- how can I see what data are through (and trigger only individual environment) I'd appreciate. Secondly, should I somehow clear the current CoE App table (I see that this flow checks if there are any new or edited app)? I tried yesterday with FullInventory=yes but it didn't bring anything...

JeneferM-MSFT commented 3 years ago

I feel a little lost at this point, so let's take a step back.

You issue is that in the PBI and in the Model Driven App, you see some custom connectors that don't show their apps as connected. Can you validate that this is the issue you are seeing?

We do not repro this, but we did fix an issue in this space a while ago, but to see it applied to your tenant you would need to do a "full inventory" run. Can you please try that? Here's how: Incomplete Inventory

FacilyDev commented 3 years ago

Ok, so now it's more clear. I made the full refresh and there is one custom connector linked to the PowerApp. So it's not the issue of custom connectors being filtered out from App References but incompleteness of App References. Again, after the refresh there is exact the same number of rows in App References table so I presume the flow cuts/stops by whatever reason and not import all App References. Strangely, all Apps and all Connectors are in. Any hint where to look for, appreciated.

JeneferM-MSFT commented 3 years ago

I'm still unclear, can you validate that this is the issue you are seeing? Your issue is that in the PBI and in the Model Driven App, you see some custom connectors that don't show their apps as connected.

JeneferM-MSFT commented 3 years ago

Or are you missing custom connectors all together?

FacilyDev commented 3 years ago

"you see some custom connectors that don't show their apps as connected." -- exactly but I don't think the custom connectors are the issue. It was what I checked and noticed that after initial load NONE custom connector was present. After second load one custom connector appeared in references. So it's not custom connectors issue but app-connector references incompleteness. The process of filling the table admin_connectionreferences do not contain all app-connector references; the load stops after about 4k while in reality it should have 20 times more...

FacilyDev commented 3 years ago

Ok, I think I'm getting closer -- I downloaded the data directly from dataverse table (as .csv) admin_connectionreferences and it has 13351 rows (all custom connectors are there :-) ) while in PowerBI before any filters first step importing data admin_connectionreferences = entities{[EntitySetName="admin_connectionreferences"]}[Data], -- gives only 1000 rows... So looks like some restriction during PBI data import...

JeneferM-MSFT commented 3 years ago

Except you said that they dont appear in the Model Driven App either. Can you confirm that is still the case after you did the full inventory? I am also doubtful that a full inventory would have run that quickly for you given what it sounds like is the size of your tenant

FacilyDev commented 3 years ago

After the Full Inventory data load they appear in the Power Platform Admin View app. So it's only issue in PowerBI now.

JeneferM-MSFT commented 3 years ago

What version of the PBI are you using?

I suggest you do the following 1) Run the (Check Deleted) flow in Core. It will remove a bunch of the dead connection references that may be clogging things 1) Try the latest version of the Power BI released in August. I know she did some work to simplify the relationships and made them more streamlined.

FacilyDev commented 3 years ago

Looks like after (Check Deleted) and loading FullInventory all Custom Connectors' connections are now visible in CoE PowerBI. Also, # Apps for specific connectors seems correct. Thanks for help -- this solution really rocks (even though can still have some bugs and sometimes is really complex to troubleshoot ;-) ). I close this issue now.

JeneferM-MSFT commented 3 years ago

Wonderful news. Thanks for using CoE