search

Azure / logicapps

Azure Logic Apps labs, samples, and tools
MIT License
369 stars 303 forks source link

Bug in the "Microsoft Sentinel incident" trigger that makes the Sentinel query field returned in the result truncated #1093

Open ak-fusionone opened 5 months ago

ak-fusionone commented 5 months ago

Describe the Bug with repro steps

Hi team,

There is a bug that occurs in a specific strange condition that I noticed in a Logic App.

In order to reproduce this bug you need to create a new Analytic rule (following is a simplified example I have exported) in Sentinel:

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "workspace": {
            "type": "String"
        }
    },
    "resources": [
        {
            "id": "[concat(resourceId('Microsoft.OperationalInsights/workspaces/providers', parameters('workspace'), 'Microsoft.SecurityInsights'),'/alertRules/875277a6-73b2-4c8c-91ae-6224ea3912ca')]",
            "name": "[concat(parameters('workspace'),'/Microsoft.SecurityInsights/875277a6-73b2-4c8c-91ae-6224ea3912ca')]",
            "type": "Microsoft.OperationalInsights/workspaces/providers/alertRules",
            "kind": "Scheduled",
            "apiVersion": "2023-12-01-preview",
            "properties": {
                "displayName": "test bug",
                "description": "hello",
                "severity": "Informational",
                "enabled": true,
                "query": "let logonDiff = 10m;\r\nlet aadFunc = (tableName: string) {\r\n  let mainAadFunc=(tableName: string){\r\n    table(tableName)\r\n    | where ResultType == \"0\"\r\n    | where TimeGenerated >= ago(1d)\r\n    | where AppDisplayName !in (\"Office 365 Exchange Online\", \"Skype for Business Online\")\r\n    | project\r\n        SuccessLogonTime = TimeGenerated,\r\n        UserPrincipalName,\r\n        SuccessIPAddress = IPAddress, SuccessStatus = Status, SuccessAuthenticationDetails = AuthenticationDetails,\r\n        AppDisplayName,\r\n        SuccessIPBlock = strcat(split(IPAddress, \".\")[0], \".\", split(IPAddress, \".\")[1]),\r\n        Type,\r\n        SuccessDeviceDetail = DeviceDetail, SuccessLocationDetail = LocationDetails\r\n    | join kind= inner (\r\n        table(tableName)\r\n        // Result types:\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   50140=\"This occurred due to 'Keep me signed in' interrupt when the user was signing in.\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        | where ResultType !in (\"0\", \"50140\")\r\n        | where (ResultType == \"53003\" and ResultDescription == \"Access has been blocked due to conditional access policies.\")==false\r\n        // Excluded the failure of the conditional access policy since it's covered in another analytic rule\r\n        | where (ConditionalAccessStatus == 1 or ConditionalAccessStatus =~ \"failure\" or tostring(parse_json(Status).failureReason) == \"Access has been blocked due to conditional access policies.\") == false\r\n        | where ResultDescription !~ \"Other\"\r\n        | where (ResultType == \"50076\" and ResultDescription == \"Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access the resource.\") == false\r\n        | where (ResultType == \"500121\" and ResultDescription has_any (\"MFA denied; duplicate authentication attempt\", \"MFA successfully completed\")) == false\r\n        | where AppDisplayName !in (\"Office 365 Exchange Online\", \"Skype for Business Online\")\r\n        | project\r\n            FailedLogonTime = TimeGenerated,\r\n            UserPrincipalName,\r\n            FailedIPAddress = IPAddress, FailedStatus = Status, FailedAuthenticationDetails = AuthenticationDetails,\r\n            AppDisplayName,\r\n            ResultType,\r\n            ResultDescription,\r\n            Type,\r\n            FailedDeviceDetail = DeviceDetail, FailedLocationDetail = LocationDetails\r\n    ) on UserPrincipalName, AppDisplayName \r\n    | where SuccessLogonTime < FailedLogonTime and FailedLogonTime - SuccessLogonTime <= logonDiff and FailedIPAddress !startswith SuccessIPBlock\r\n    | summarize FailedLogonTime = max(FailedLogonTime), SuccessLogonTime = max(SuccessLogonTime)\r\n        by\r\n        UserPrincipalName,\r\n        SuccessIPAddress,\r\n        AppDisplayName,\r\n        FailedIPAddress,\r\n        tostring(SuccessStatus), tostring(FailedStatus),\r\n        tostring(SuccessAuthenticationDetails), tostring(FailedAuthenticationDetails),\r\n        ResultType,\r\n        ResultDescription,\r\n        Type,\r\n        tostring(SuccessDeviceDetail), tostring(SuccessLocationDetail),\r\n        tostring(FailedDeviceDetail), tostring(FailedLocationDetail)\r\n    | extend timestamp = SuccessLogonTime\r\n    | join kind= leftouter  (\r\n        DeviceInfo\r\n        | extend corporate = \"True\"\r\n        | extend FailedIPAddress = PublicIP\r\n        | project FailedIPAddress, corporate\r\n        | summarize by corporate, FailedIPAddress\r\n    ) on FailedIPAddress\r\n  };\r\n  let mainsearch=mainAadFunc(tableName);\r\n  mainsearch\r\n};\r\nlet aadSignin = aadFunc(\"SigninLogs\");\r\nlet aadNonInt = aadFunc(\"AADNonInteractiveUserSignInLogs\");\r\nunion isfuzzy=true aadSignin, aadNonInt",
                "queryFrequency": "P1D",
                "queryPeriod": "P14D",
                "triggerOperator": "GreaterThan",
                "triggerThreshold": 0,
                "suppressionDuration": "PT5H",
                "suppressionEnabled": false,
                "startTimeUtc": "2024-06-21T01:18:00.000Z",
                "tactics": [],
                "techniques": [],
                "subTechniques": [],
                "alertRuleTemplateName": null,
                "incidentConfiguration": {
                    "createIncident": true,
                    "groupingConfiguration": {
                        "enabled": false,
                        "reopenClosedIncident": false,
                        "lookbackDuration": "PT5H",
                        "matchingMethod": "AllEntities",
                        "groupByEntities": [],
                        "groupByAlertDetails": [],
                        "groupByCustomDetails": []
                    }
                },
                "eventGroupingSettings": {
                    "aggregationKind": "SingleAlert"
                },
                "alertDetailsOverride": null,
                "customDetails": null,
                "entityMappings": null,
                "sentinelEntitiesMappings": null,
                "templateVersion": null
            }
        }
    ]
}

And you need also to create a logic app that has "Microsoft Sentinel incident" as a trigger (first step). Add any random step after that since the bug remains in the first step "Microsoft Sentinel incident". Save the logic app.

You need to create a new Sentinel Automation with the "When incident is created" trigger. And you need to use the "Run Playbook" action. And you need to select the logic app you have created.

This means that when a Sentinel Incident is triggered, the logic app will retrieve the Sentinel Incident details.

The bug that I noticed is in the output provided in the first step.

If you didn't know, the Sentinel Analytic Rules are limited with around 15 000 characters in the query if I remember well. And the first step of that logic app should show all the details of the Sentinel Incident including the analytic rule query even if it reaches the 15 000 length (I have an example of output that shows the query include 15 000 characters).

But the example of analytic rule I have provided doesn't behave the same way. When a Sentinel Incident triggers, the first step of that logic app shows this:

{
  "eventUniqueId": "REDACTED",
  "objectSchemaType": "Incident",
  "objectEventType": "Create",
  "workspaceInfo": {
    "SubscriptionId": "REDACTED",
    "ResourceGroupName": "REDACTED",
    "WorkspaceName": "REDACTED"
  },
  "workspaceId": "REDACTED",
  "object": {
    "id": "REDACTED",
    "name": "REDACTED",
    "etag": "\"REDACTED\"",
    "type": "Microsoft.SecurityInsights/Incidents",
    "properties": {
      "title": "test bug",
      "severity": "Informational",
      "status": "New",
      "owner": {
        "objectId": null,
        "email": null,
        "assignedTo": null,
        "userPrincipalName": null
      },
      "labels": [],
      "firstActivityTimeUtc": "2024-06-07T01:01:16.9825705Z",
      "lastActivityTimeUtc": "2024-06-21T01:01:16.9825705Z",
      "lastModifiedTimeUtc": "2024-06-21T01:06:41.7833333Z",
      "createdTimeUtc": "2024-06-21T01:06:41.7433333Z",
      "incidentNumber": REDACTED,
      "additionalData": {
        "alertsCount": 1,
        "bookmarksCount": 0,
        "commentsCount": 0,
        "alertProductNames": [
          "Azure Sentinel"
        ],
        "tactics": [],
        "techniques": []
      },
      "relatedAnalyticRuleIds": [
        "REDACTED"
      ],
      "incidentUrl": "REDACTED",
      "providerName": "Microsoft XDR",
      "providerIncidentId": "REDACTED",
      "alerts": [
        {
          "id": "REDACTED",
          "type": "Microsoft.SecurityInsights/Entities",
          "kind": "SecurityAlert",
          "properties": {
            "systemAlertId": "REDACTED",
            "tactics": [],
            "alertDisplayName": "test bug",
            "description": "hello",
            "confidenceLevel": "Unknown",
            "severity": "Informational",
            "vendorName": "Microsoft",
            "productName": "Azure Sentinel",
            "productComponentName": "Scheduled Alerts",
            "alertType": "REDACTED",
            "processingEndTime": "2024-06-21T01:06:42.16Z",
            "status": "New",
            "endTimeUtc": "2024-06-21T01:01:16.9825705Z",
            "startTimeUtc": "2024-06-07T01:01:16.9825705Z",
            "timeGenerated": "2024-06-21T01:06:41.6666667Z",
            "providerAlertId": "REDACTED",
            "alertLink": "REDACTED",
            "resourceIdentifiers": [
              {
                "type": "LogAnalytics",
                "workspaceId": "REDACTED"
              }
            ],
            "additionalData": {
              "IncidentId": "3",
              "OriginSource": "Microsoft 365 defender",
              "Category": "SuspiciousActivity",
              "DetectionSource": "scheduledAlerts",
              "AssignedTo": null,
              "Determination": null,
              "Classification": null,
              "ThreatName": null,
              "ThreatFamilyName": null,
              "DetectorId": "REDACTED",
              "LastUpdated": "6/21/2024 1:06:42 AM",
              "Trigger Operator": "GreaterThan",
              "Trigger Threshold": "0",
              "Correlation Id": "REDACTED",
              "Search Query Results Overall Count": "119",
              "Data Sources": "[\"REDACTED\"]",
              "Analytic Rule Ids": "[\"REDACTED\"]",
              "Event Grouping": "SingleAlert",
              "Analytic Rule Name": "test bug",
              "ProcessedBySentinel": "True",
              "Alert generation status": "Full alert created",
              "Query": "// The query_now parameter represents the time (in UTC) at which the scheduled analytics rule ran to produce this alert.\nset query_now = datetime(2024-06-21T01:01:16.9825705Z);\nlet logonDiff = 10m;\r\nlet aadFunc = (tableName: string) {\r\n  let mainAadFunc=(tableName: string){\r\n    table(tableName)\r\n    | where ResultType == \"0\"\r\n    | where TimeGenerated >= ago(1d)\r\n    | where AppDisplayName !in (\"Office 365 Exchange Online\", \"Skype for Business Online\")\r\n    | project\r\n        SuccessLogonTime = TimeGenerated,\r\n        UserPrincipalName,\r\n        SuccessIPAddress = IPAddress, SuccessStatus = Status, SuccessAuthenticationDetails = AuthenticationDetails,\r\n        AppDisplayName,\r\n        SuccessIPBlock = strcat(split(IPAddress, \".\")[0], \".\", split(IPAddress, \".\")[1]),\r\n        Type,\r\n        SuccessDeviceDetail = DeviceDetail, SuccessLocationDetail = LocationDetails\r\n    | join kind= inner (\r\n        table(tableName)\r\n        // Result types:\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   50140=\"This occurred due to 'Keep me signed in' interrupt when the user was signing in.\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        //   aaaaa=\"description description description description description\"\r\n        | where ResultType !in (\"0\", \"50140\")\r\n        | where (ResultType == \"53003\" and ResultDescription == \"Access has been blocked due to conditional access policies.\")==false\r\n        // Excluded the failure of the conditional access policy since it's covered in another analytic rule\r\n        | where (ConditionalAccessStatus == 1 or ConditionalAccessStatus =~ \"failure\" or tostring(parse_json(Status).failureReason) == \"Access has been blocked due to conditional access policies.\") == false\r\n        | where ResultDescription !~ \"Other\"\r\n        | where (ResultType == \"50076\" and ResultDescription == \"Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access the resource.\") == false\r\n        | where (ResultType == \"500121\" and ResultDescription has_any (\"MFA denied; duplicate authentication attempt\", \"MFA successfully completed\")) == false\r\n        | where...",
              "Query Period": "14.00:00:00",
              "Query Start Time UTC": "2024-06-07 01:01:16Z",
              "Query End Time UTC": "2024-06-21 01:01:17Z"
            },
            "friendlyName": "test bug"
          }
        }
      ],
      "bookmarks": [],
      "relatedEntities": [],
      "comments": []
    }
  }
}

And as you can see the query was truncated to 2975 characters. The original query was longer and no matter what is the query length, the "Microsoft Sentinel incident" step always shows 2975 characters from the query by adding "…" at the end.

Among all the long analytic rules that generated Sentinel Incident, this is the only one that shows this strange behavior.

Do I need the query? Yes it's very important.

What made me think this is exactly a bug in the logic app is that in Sentinel when I access the Incident and I click on the Entity, I can see the real query not the trucated query.

Thank you for your efforts as usual.

What type of Logic App Is this happening in?

Consumption (Portal)

Are you using new designer or old designer

New Designer

Did you refer to the TSG before filing this issue? https://aka.ms/lauxtsg

No

Workflow JSON

No response

Screenshots or Videos

No response

Browser

Version 126.0.6478.62 (Official Build) (64-bit)

Additional context

No response

ak-fusionone commented 5 months ago

This shows the truncated query: image

ak-fusionone commented 5 months ago

The issue occured another time with another analytic rule query that ends with this part: image

ak-fusionone commented 4 months ago

Hi team,

Is there any update on this?

Otherwise who should I contact?

Thanks for any hint

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 45 days with no activity.

TinuW commented 2 months ago

Is there any update on this? Otherwise who should we contact? Thanks for any hint

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 45 days with no activity.

ak-fusionone commented 1 month ago

Is there any update on this?

Otherwise who should I contact?