n3tsurge
commented
10 months ago It would appear you have too many signatured events that are open and are reaching max agg or max terms. Let me find some diagnostic queries for you
Closed greycel closed 9 months ago
n3tsurge
commented
10 months ago It would appear you have too many signatured events that are open and are reaching max agg or max terms. Let me find some diagnostic queries for you
n3tsurge
commented
10 months ago You can run this query in Dev Tools in Kibana/Opensearch Dashboards. If you have more than 10000 unique signatures it means you are not effectively signaturing your events and are hitting Elastic/Opensearches max_buckets setting.
GET reflex-events/_search
{
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "status.name: New"
}
},
{
"range": {
"created_at": {
"gte": "now-7d"
}
}
}
]
}
},
"aggs": {
"unique_sigs": {
"cardinality": {
"field": "signature"
}
}
},
"size": 0
}
greycel
commented
10 months ago Yeah, ran the query and there are more than 95K unique signatures for the last 10 days.
Initially, while creating inputs for Windows sysmon, system, or security events, I wasn't sure which fields to include as part of the signature fields and left it blank as the system would generate default signatures for incoming events.
Can you help me understand how to handle this and effectively maintain signatures on all incoming events? Thank you...
n3tsurge
commented
10 months ago So it sounds like you are consuming all your Windows Event logs as Events into Reflex. This is not how Reflex is designed. Theres 3 operational modes for Reflex.
Alert Polling mode you define your Inputs and assign them to Agents with the poller role. When you do this you should be pointing at Alert indices not raw event indices.
Detections, you define your inputs to your raw event indices but DO NOT assign them to agents. Agents with detector roles will query these inputs and run rules against them.
In the hybrid mode you just mix up the 2 above and do both
greycel
commented
10 months ago I've configured three inputs (Windows sysmon, security, system) to poll from the common alert index, but there are way too many false-positive alerts due to which the volume of the incoming events is high.
If I update the existing inputs with the required signature fields like process.name, process.command_line, will that resolve this issue...?
Also any suggestion on effectively maintaining signatures?
n3tsurge
commented
10 months ago It may, it may not. By default the system will signature based on the event title and that is it. I typically recommend using the hostname, username, process name and other fields that may be distinct across events as signature fields. THis way you have the events rolled up by per host per user at least by default.
You can signature too little or signature too much...you partly need to understand your data and how you want things rolled up.
greycel
commented
10 months ago This is helpful, will try it out. Thank you...
greycel
commented
10 months ago Couldn't add signature fields to an existing input, appears like a UI bug while updating inputs. This is happening only for the signature field maybe is it because the field is empty from the beginning.
I was able to add them directly from the advanced editing, but after saving the input changes added fields aren't visible in the UI under Event Base Configuration:
"signature": [
"host.name",
"user.name",
"process.name"
],The setting is signature_fields instead of signature if you go to Advanced Edit again and change it. I will see about the UI error you shared
greycel
commented
10 months ago That worked... Thank you so much!
Signature fields added from advanced editing are now visible in Event Base Configuration and I'm able to add new tags directly from the Event Base Configuration section now.
Hi n3tsurge,
Noticed that the events are easily loaded in the queue page when the volume is below 65K, Added new log inputs, and now when the event volume is above 65K (even for a day or two) the data is not loading in the queue and receiving
search_phase_execution_exceptionin the API backend logs while querying OpenSearch.Using my own instance of OpenSearch as backend, with Reflex Version
2023.09.28-rc0, Is there any configuration to be made on the OpenSearch side, pls advise...