Security Analytics enables users for detecting security threats on their security event log data. It will also allow them to modify/tailor the pre-packaged solution.
Apache License 2.0
73
stars
74
forks
source link
Make usage of index-patterns available in correlation rules #1276
Is your feature request related to a problem?
When creating a new Detector, i can choose an index-pattern as the source. This comes in quite handy given that the rollover for my indices is done by data-prepper.
Example: For my aws-cloudtrail logs, the index naming pattern looks like this: "logs-aws-cloudtrail-%{yyyy.MM.dd}". This results into indexes like:
logs-aws-cloudtrail-2024.08.26
logs-aws-cloudtrail-2024.08.27
logs-aws-cloudtrail-2024.08.28
[...]
So working with detectors is cool. Now to the correlation rules.
When i want to create a new correlation rule, i need to specify a source index for each query. And here's the problem: index-patterns do not seem to work // to be accepted. I can either choose between a named index, or an index alias. But working with index-aliases would require to have an index template which sets this alias and an Index-State-Management policy which would declare the current write-index (if i got that right, never worked with aliases before).
Given that you can "only" search for the past 24h with correlation rules, would it make sense to have a dedicate "triage" index, which gets re-created daily? This would reduce me to a timeframe of 00:00 - 23:59 then, so i won't be able to detect correlations between 23:59 and 02:00 for example... 🥴
It would be more easy for me to have an index-pattern as the source for correlation rules.
What solution would you like?
When creating a new correlation rule, i would like to be able to choose an index-pattern as the source for such a rule. Similar to the process of creating a new Detector.
What alternatives have you considered?
An alternative might be to use an index-alias, which points to the two most recent indexes. That would require an index template and an ISM policy that takes care of managing the index-alias, if that is even possible. I have not tested this setup, but i think it might work.
Do you have any additional context?
Allowing index-patterns in Detectors but not in Correlation rules creates an inconsistency. Maybe there is a reason for this and i am just too stupid to see it. 😅
And maybe there is another best-practise approach for my use-case which i haven't seen. If so, i would highly appreciate it if someone could guard me into the right directions.
dblock
commented
2 months ago
Is your feature request related to a problem? When creating a new
Detector, i can choose an index-pattern as the source. This comes in quite handy given that the rollover for my indices is done bydata-prepper. Example: For my aws-cloudtrail logs, the index naming pattern looks like this:"logs-aws-cloudtrail-%{yyyy.MM.dd}". This results into indexes like:logs-aws-cloudtrail-2024.08.26logs-aws-cloudtrail-2024.08.27logs-aws-cloudtrail-2024.08.28[...]So working with detectors is cool. Now to the
correlation rules. When i want to create a new correlation rule, i need to specify a source index for each query. And here's the problem: index-patterns do not seem to work // to be accepted. I can either choose between a named index, or an index alias. But working with index-aliases would require to have an index template which sets this alias and an Index-State-Management policy which would declare the current write-index (if i got that right, never worked with aliases before).Given that you can "only" search for the past 24h with correlation rules, would it make sense to have a dedicate "triage" index, which gets re-created daily? This would reduce me to a timeframe of 00:00 - 23:59 then, so i won't be able to detect correlations between 23:59 and 02:00 for example... 🥴
It would be more easy for me to have an index-pattern as the source for correlation rules.
What solution would you like? When creating a new correlation rule, i would like to be able to choose an index-pattern as the source for such a rule. Similar to the process of creating a new Detector.
What alternatives have you considered? An alternative might be to use an index-alias, which points to the two most recent indexes. That would require an index template and an ISM policy that takes care of managing the index-alias, if that is even possible. I have not tested this setup, but i think it might work.
Do you have any additional context? Allowing index-patterns in Detectors but not in Correlation rules creates an inconsistency. Maybe there is a reason for this and i am just too stupid to see it. 😅 And maybe there is another best-practise approach for my use-case which i haven't seen. If so, i would highly appreciate it if someone could guard me into the right directions.
Thanks in advance for any answer! :)