search

Azure / Azure-Sentinel

Cloud-native SIEM for intelligent security analytics for your entire enterprise.
https://azure.microsoft.com/en-us/services/azure-sentinel/
MIT License
4.38k stars 2.89k forks source link

Automations rules not firing upon incident update of type comment added #9604

Closed FormindALH closed 6 months ago

FormindALH commented 6 months ago

Describe the bug Hi there !

It seems like sometimes automation rules of type incident update (specifically comment added or incident ownership change in our use-case) do not fire their affected logicapp. It also seems to happen randomly and with or without other automation rules having their child logicapp execution done or not. Although there was a specific case in which stopping a logicapp with a long execution time fired by an automation rule of type incident created did then permit correct execution of our automation rule, that does not seem to be the problem since it also happens without this.

We do use this on a cross-tenant sentinel configuration across multiple sentinels, however i do not think this could be the issue? (unless some sort of cap is in place for incident updates)

To Reproduce Steps to reproduce the behavior:

  1. Make automation rules, some of type incident created and others of type incident updated (comment added or owner changed), have them fire logicapps
  2. Some expected logicapp executions from incident update automation rules will not happen. Bear in mind the proportion in which it happens is not huge but definitely not unnoticeable even compared to our influx of incidents.

Expected behavior In our specific case every time a comment is added or the owner changed, the configured logicapp should be triggered.

Screenshots Here is an incident update supposed to trigger a logicapp : image

Here is the automation rule supposed to trigger : image

And here is the latest entry in the logicapp run and trigger history : image image

github-actions[bot] commented 6 months ago

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

v-muuppugund commented 6 months ago

Hi @FormindALH , Thanks for flagging this issue, we will investigate this issue and get back to you with some updates by 21Dec23. Thanks!

brynjare commented 6 months ago

I believe we might be seeing the same issue as @FormindALH in a multi-tenant setup, where certain automation rules will occasionally - and seemingly randomly - fail to trigger the associated playbook (which is shared between all tenants).

The same playbook is triggered both on incident creation and on incident updates, but so far it looks as if we're only experiencing the issue when incidents are updated.

When the "incident updated" automation rules fail to trigger the playbook, we can see related entries with the following details in SentinelHealth in the affected workspaces:

OperationName ExtendedProperties.TriggeredOn ExtendedProperties.TriggeredWhen Status Description Reason
Automation rule run Incident Updated Failure Could not trigger playbook: \<playbookName>. An internal server error occurred. Please contact support for assistance. Action run failed.
garis commented 6 months ago

The issue reported by @FormindALH is being investigated in case #2312210050000718. Will update if there is something noteworthy to share. @brynjare I would suggest you to do the same and raise a case with Sentinel if you can.

We should close this issue. A proper investigation involves sharing confidential data and this is not the correct place to do so.

v-muuppugund commented 6 months ago

Ok @garis , we are closing the issue (https://github.com/Azure/Azure-Sentinel/issues/9604) If you still need support for this issue, feel free to re-open at any time. Thank you for your co-operation!