To Reproduce
Run Existing query against data set that is TB a day in email traffic. Analysts will get excessive memory alerts and in cases for larger deployments
SEM00023 errors as materialize exceeds the limit and fails the query.
Expected behavior
Utilize improved query performance by dropping materialize and split data up more to improve performance.
Here is a fix which improves overall performance of query by over 90 percent on same data sets with same results.
KQL Query for DomainEntity_EmailUrlInfo is not optimized for larger data sets leading to memory issues in LAW.
Source- https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/Threat%20Intelligence/Analytic%20Rules/DomainEntity_EmailUrlInfo.yaml
To Reproduce Run Existing query against data set that is TB a day in email traffic. Analysts will get excessive memory alerts and in cases for larger deployments SEM00023 errors as materialize exceeds the limit and fails the query.
Expected behavior Utilize improved query performance by dropping materialize and split data up more to improve performance.
Here is a fix which improves overall performance of query by over 90 percent on same data sets with same results.
https://github.com/MSJosh/documentation/blob/main/Sentinel/Misc.%20KQL/TI/DomainEntity_EmailUrlInfo.yaml