microsoft / coe-starter-kit

Other
763 stars 225 forks source link

[CoE Starter Kit - BUG] Audit Logs/ PowerBi Production dashboard #9282

Open brigidaanne94 opened 6 days ago

brigidaanne94 commented 6 days ago

Does this bug already exist in our backlog?

Describe the issue

I am having issues with setting up CoE audit logs and the production coe dashboard for powerbi. I followed the instructions that are listed on Microsoft (https://learn.microsoft.com/en-us/power-platform/guidance/coe/setup-auditlog-http) and as you can see in the attached images, everything is empty. I went through the CoE setup wizard and audit log wizard multiple times. The power platform admin center is completely empty, audit logs empty, and same with the powerbi dashboard. Can anyone offer some solution?

Image Image Image

Expected Behavior

No response

What solution are you experiencing the issue with?

Core

What solution version are you using?

4.45

What app or flow are you having the issue with?

Production_CoEDashboard

What method are you using to get inventory and telemetry?

Cloud flows

Steps To Reproduce

No response

Anything else?

No response

Jenefer-Monroe commented 6 days ago
  1. (O365) Go to your app registration and validate that you have the correct API permissions. Note that it requires application permissions not delegated and they need to be Granted with Admin Consent. image

  2. Are you storing the secret in Azure or in plain text?

  3. Validate that you have entered all the env variables image

  4. Turn the flows off and back on. Sometimes env vars get stuck in them with old values

  5. Validate that you have started the subscription

brigidaanne94 commented 6 days ago

Yes I have performed all these steps multiple times and I am still not seeing any data... I restarted the subscription flow and received this message: "The subscription is already enabled. No property change."

brigidaanne94 commented 6 days ago

Also not seeing any runs here: Image

Jenefer-Monroe commented 6 days ago

Is this flow turned on, running, and passing? Admin | Audit Logs | Sync Audit Logs (V2)

If so, find one of your launches here and look at the output: Image

What is shown for the RecordType Image

brigidaanne94 commented 6 days ago

It is so long that I have to click to download, so there is data in there. However, the recordtype comes back as 25, 83 and 84. Also, the action after this is blank and then the following step doesn't run: Image

Jenefer-Monroe commented 6 days ago

You will need to find an instance where you have a launch to test this.

My suspicion is that you are impacted by this: https://github.com/microsoft/coe-starter-kit/issues/9268

So if you prefer you can just do the workaround for now that is described there.

brigidaanne94 commented 6 days ago

I just did the workaround instructions and same problem, no data:

Image

Jenefer-Monroe commented 5 days ago

What I'm trying to do is see if your Purview schema is correct.

The easiest way to check is to download the logs manually but you may not have permissions. Lets check.

  1. Go launch some canvas apps and then wait ~15 minutes
  2. Go to https://purview.microsoft.com/audit/ and create a search for launch eventsHave Start and End date be for the last two days or so For activities chose Launched app from both PowerApps app activities and PowerApps app activities V2 (if you see both) Image
  3. Wait for it to complete and then download the results Image Image Image
  4. Do you see both these record types shown here? (PowerAppsApp and PowerPlatformAdministratorActivity) Export the results Image
  5. Open exported file in Excel, do you see both Record Types here? (45 and 256) Image
brigidaanne94 commented 5 days ago

So before I do that, I went into another bug discussion and saw that you mentioned to make sure these 2 flows were turned on. However, I was facing a Policy error. In my attempt to fix it, I am now unable to even edit these flows. I am getting this error message: "The XRM workflow table row could not be found."

I was thinking maybe I should try to delete them and reimport them?

Image

brigidaanne94 commented 5 days ago

I did the export, and yes the record types are 45 and 256:

Image

Jenefer-Monroe commented 5 days ago

ok the issue should not be schema then. You can delete that second solution (auditlogsbackupwithoutpurview) and return to the flow in core : Admin | Audit Logs | Sync Audit Logs (V2)

You will need to find cases where you have Power Apps activities which are 45 ad 256 Image

brigidaanne94 commented 5 days ago

So I did what you said and then I launched an app and ran the sync audit log flow. I went to the filter event action and there are 67 tries. On the 2nd try and 26th try, there was data but in the launchpowerapp case, it says that the app does not exist:

Image

Jenefer-Monroe commented 5 days ago

There are duplicate records for every launch that comes through, both types 45 and 256.

What you see there is likely a type 45, can you check? And are there any that come through with the type 256?

Find one of your launches here and look at the output: Image

What is shown for the RecordType Image

brigidaanne94 commented 4 days ago

That flow just ran 40 minutes ago and I see recordtypes as: 84,83,25,210,66. However, my powerbi production dashboard now has data.

I did receive an email regarding one of the flows: "Your flow has used more than 80% of its Power Platform Requests limit in the past 24 hours. The performance is currently not impacted as it is below limits. But if the usage grows, further actions may be throttled or slowed down. Power Platform Requests used: 160817, Power Platform Requests transition limit: 200000."

Flow name: Admin | Sync Template v4 (Connection Identities)

Jenefer-Monroe commented 14 hours ago

Please post a separate issue for issues with that new flow, we have to keep issues to single topic in order to manage them all.

This flow, Admin | Audit Logs | Sync Audit Logs (V2), is also API intensive especially when you use the O365 Management API technique to get the Purview logs. This is because it cannot filter with the call to the API and you have to filter in the app, hence all those other recordtypes.

I suggest you change to use the Graph API to get Purview. It's a new API offered and our new recommendation. It allows us to filter with the call to the API to Power Platform only, and hence we dont have to spend API hits on all those other record types.