microsoft / coe-starter-kit

Other
751 stars 223 forks source link

[CoE Starter Kit - BUG] Incorrect Reports - Connectors Used #8414

Closed BP-WS closed 5 months ago

BP-WS commented 5 months ago

Does this bug already exist in our backlog?

Describe the issue

I currently have an issue where the Connector Deep Dive tab on the CoE Dashboard does not show all connectors that are actually in use across the org. This was determined, following a deployment of a new DLP policy which then caused other connectors which were actually being used (but not visible in this report) to stop working.

Expected Behavior

Would expect the 'Connectors used in apps and flows' report table to show all connectors used across the org. Not sure why there's a difference in what's displayed in here and what's actively used.

What solution are you experiencing the issue with?

Core

What solution version are you using?

4.29

What app or flow are you having the issue with?

PowerBI Report/Underlying Data

What method are you using to get inventory and telemetry?

None

Steps To Reproduce

  1. Open CoE Dashboard report
  2. go to 'Connector Deep Dive' tab
  3. View the 'Connectors used in app or flows' graph.

Anything else?

No response

AB#3477

Jenefer-Monroe commented 5 months ago

I think the visual just isnt that great to see all connectors, and likely you are just not able to see the lesser used ones as they are dwarfed by the larger.

Likely issue

Here is mine for example. By default what I see: image

Filter out all the top ones: image

Better tool for you

The better tool for this will be the Power Platform Admin View which is more tabular in nature. You can easily sort by either apps or flows and see all numbers, even if they are dwarfed. image

BP-WS commented 5 months ago

@Jenefer-Monroe , Thank you very much for your response.

I suppose a more broad concern as a result of this would be how much trust can be put into the reports generally that are provided, as with this example, there is no indication that any data may be missing/not showing, so can be quite dangerous if under the illusion that all data is being presented?

Jenefer-Monroe commented 5 months ago

I'm not sure what to say, that's the nature of charts. When there are giant numbers smaller ones appear to be zero when they really are not. Can you confirm that is the case for you?

BP-WS commented 5 months ago

@Jenefer-Monroe I'm not sure I understand/agree with that sentiment.

Metrics, charts and the like have no value if we cannot trust the data that they display? Certainly in the case of the graph I've specifically raised this issue for, there were connectors that were in use that the chart did not display.

Jenefer-Monroe commented 5 months ago

I'm not sure how to improve the chart to show values for both large and small numbers at the same time. Open to ideas. Otherwise I would suggest using the tabular view in the Power Platform Admin View app.

None of us that made this starter kit are PBI experts, that product lies with another team, and we are not visualization experts either. So with that in mind, you may be more comfortable working just within the Power Platform Admin view and not the charts if you feel these are not up to par.

BP-WS commented 5 months ago

@Jenefer-Monroe Would this therefore mean that it's quite likely we could have similar issues with other charts within the CoE Dashboard more broadly? Just would like to confirm for awareness more than anything - as I'll bear in mind in the future.

If so, I think this should be something that is much more visibly displayed throughout the Power BI tabs/graphs. I'd personally much rather just keeping a small fraction of the graphs currently available rather than the currently-huge set of graphs if it would mean these could be checked/improved for better data accuracy. As I say, from my perspective, I struggle to understand what value any number of graphs provide if they're simply not correct - especially when informing the likes of strategies and policies.

Jenefer-Monroe commented 5 months ago

Some charts are top-20 style, the rest show all data.

You are the first person to raise this and this chart has been in the PBI for a few years, so I think others recognize that the large data dwarfs the small. I do see where you are coming from but do believe that is just the nature of charts. If people have better ideas we are open to them.

But please remember that we are a starter kit not a product. So if you are dissatisfied with our PBI please do feel free to make your own. And if you come up with a clever way to show both large and small data here, please let us know, we are happy to take contributions from users.

BP-WS commented 5 months ago

@Jenefer-Monroe Understood, appreciate you taking the time to provide helpful responses throughout this query.

Will certainly let you know if so - thank you :)

BP-WS commented 5 months ago

@Jenefer-Monroe I can confirm that this report does in fact not show all data following some testing.

We have spent some time trying to test our theory and have been able to work out that due to the bidirectional relationship of the model for this graph, it's essentially filtering connectors and showing all connectors where they exist for a power app AND a flow, not either - which is implied and should therefore be the case.

Upon quickly altering this and building a new graph, we have been able to confirm that there are specific connectors (quite a few!) that have been completely missed as a result.

Jenefer-Monroe commented 5 months ago

Thank you for testing, likely I will just remove this chart from the next release then. And point people to the Power Platform Admin View app.

Jenefer-Monroe commented 5 months ago

looks like we can split it into two instead and keep the report

Image Image

Thanks for pressing here and let me know your thoughts after upgrade!

BP-WS commented 5 months ago

@Jenefer-Monroe Thank you