microsoft / coe-starter-kit

Other
743 stars 219 forks source link

[CoE Starter Kit - BUG] Impact Analysis Parse to CSV #8369

Closed AllyThorne closed 2 months ago

AllyThorne commented 4 months ago

Does this bug already exist in our backlog?

Describe the issue

Running the flow 'Parse Resources into CSV' fails in the March version of the CoE and reports the following error:

The value passed for ConditionOperator.In is empty. Attribute Name: admin_AppEnvironmentID, Attribute Id: 10cd8e70-cc82-4a63-8c2d-c7e578724394

Expected Behavior

it appears that the column is not being filled - are we able to identify which flow should be filling this column and triage the issue that way?

What solution are you experiencing the issue with?

Core

What solution version are you using?

4.24

What app or flow are you having the issue with?

Parse Resources into CSV

What method are you using to get inventory and telemetry?

Cloud flows

Steps To Reproduce

No response

Anything else?

This appears to be a similar issue to that raised here: https://github.com/microsoft/coe-starter-kit/issues/4828

AB#3391

AllyThorne commented 4 months ago

Impact Analysis Issue

Jenefer-Monroe commented 4 months ago

I dont repro this in the current build. Can you please do the following:

  1. Input parameters Do you have values for all 8 of the input parameters? image

  2. Point of failure Dive into that failed scope to show where specifically you are seeing the failure.

eeuo commented 4 months ago

I've also a problem with the flow "DLP Editor>Parse impacted resources into CSV" when trying to create a Task List about the conflicting apps and flows. When creating a csv file, the flow runs successfully.

Two different errors occur in the phase "Switch export option" -> "Export to Task list" -> "Add impacted apps to task list" -> "Add impacted app to table"

First error: "The supplied reference link -- admin_makers() -- is invalid. Expecting a reference link of the form /entityset(key)."

Second error: "')' or ',' expected at position 8 in '(Default-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx)'." Note that the default environment's GUID is anonymized.

dlp_error_1

dlp_error_2

Jenefer-Monroe commented 4 months ago

thank you, to area owner for investigation.

AllyThorne commented 4 months ago

Okay, I will try to get the input information :)

AllyThorne commented 4 months ago

@gulzarudheen I have looked into the parameters. On closer inspection what seems to have occurred is that when we selected to include all the environments in the policy the parameter that defines which environments are included is not being filled.

EnvironmentsInPolicy:[]

All of the other parameters seem fine. Upon testing again we are able to manually add all of the environments to the policy rather than selecting to exclude only specific environments.

However, we then receive the following error, the DLP analysis is still received via email for both flows and apps but we assume this is incomplete, as it fails to add an impacted app to the table.

DLP Issue 2

AllyThorne commented 4 months ago

Noting the following potentially related bug that references the selection of the 'exclude certain environments' functionality and it automatically switching to something else: https://github.com/microsoft/coe-starter-kit/issues/7928

PatrickP1776 commented 3 months ago

I think I figured this one out... looks to be a value mismatch for writing to the 'admin_dlpimpactanalysis' table, in the variable build steps for 'Add app to impacted apps', 'Add app to impacted apps 2', 'Add flow to impacted flows', and 'Add flow to impacted flows 2', the value referenced for the Environment ID in the varImpactedApps and varImpactedFlows is calling the 'admin_AppEnvironmentID' from the 'admin_App' table and the 'admin_FlowEnvironmentID' from the 'admin_Flow' table respectively. Since the value is a sting that includes the text 'Default-' for the default environment, the add items steps cannot write the value to 'admin_dlpimpactanalysis' table as the 'admin_ImpactedEnvironment' column is a lookup and requires a GUID for the environment record to be found.

As a solution I added a compose step that returns only the last 36 characters (the length of the GUID portion) of the 'admin_appenvironmentid' column value for the 'admin_App' table and 'admin_flowenvironmentid' column value for the 'admin_Flow' table respectively.

Screenshot 2024-07-01 152157

I then change the value references for the variable build steps for 'Add app to impacted apps', 'Add app to impacted apps 2', 'Add flow to impacted flows', and 'Add flow to impacted flows 2' to reference the respective compose out puts. As a result the value is a GUID which can be used by the 'admin_dlpimpactanalysis' table to lookup the environment.

Screenshot 2024-07-01 152314

There is likely a better way to get to this solution, but hopefully this helps.