Advanced Splunk SPL Support

[Status-418]

Hi, I'm looking to incorporate some of the more advanced Splunk SPL features in the sigmac.py script. After reading Issue #83 I have a feeling this may be somewhat related as it does mention falling back to SPL.

Background: Many of my Splunk searches utilize functions such as eval, rex, rename or even table. A very simple example could be:

index=winevent_s sourcetype=wineventlog:security CommandLine="*powershell*" | eval Severity=if(match(CommandLine, "http"), "Critical", "Low") | table _time HostName UserName ParentProcessName CommandLine Severity

While I understand that storing a search with this level of detail will make it likely not be translatable to another output target format it would be extremely powerful for everyone operating in a Splunk environment.

I've been doing some stepping through the code and from what I can tell at the moment the way the conditions are parsed out it won't allow more than one "|" in the condition. Looking at the function parseSearch in condition.py there can only be one line of conditions so having multiple lines in the condition field in the yml file will not work either.

If you have any suggestions on how to go about including other SPL features please feel free to make suggestions on where to start and how to go about it. I'm happy to try and help where I can.

Cheers

[thomaspatzke]

Hi!

There are some basic principles of Sigma that would be contradicted. We want to keep it generic, which means that it should be possible to map most Sigma features into query constructs of the target systems. Introduction of target-specific features would be a contradiction of this. Further, we want to keep Sigma simple instead of overloading it with features not required for >90% of log signatures.

I have plans to introduce a possibility for defining prefix and suffix query parts in the converter configuration. This would allow some preprocessing of log data before the generated query is applied to it.

[Status-418]

Hey, thank you for the quick response. I fully understand your point about keeping it generic. In my view, that's exactly what makes it so powerful for the community. I look forward to seeing what you come up with in terms of prefix and suffix enhancements.

Not to go on a tangent but I probably have to disagree with your statement related to 90%. In my experience, if I only can use one line of searches and filters plus do two differnet kind of aggregations per search I'd end up with a load of badly tuned alerts. But to be honest this may just be an artifact of Splunk and that it has functionality that other platforms don't appear to have. (In my admittedly very limited experience of other platforms:)

I absolutely love how the search and filters are dynamically generated. Maybe you would consider doing something similar with the conditions. For example, if I wanted to do the below search:

CommandLine="*powershell*" NOT CommandLine="*-w hidden*" 
| eval Severity=if(match(CommandLine, "http"), "Critical", "Low") 
| table _time HostName UserName ParentProcessName CommandLine Severity'

This could be reflected in the rule as follows:

...
detection:
    selection:
        CommandLine: 
            - '*powershell*'
    filter:
        CommandLine:
            - '*-w hidden*'
    func1:
        eval:
            Severity:
                - 'if(match(CommandLine, "http"), "Critical", "Low")'
    func2:
        table:
            - _time
            - HostName
            - Username
            - ParentProcessName
            - CommandLine
            - Severity
    condition: selection and not filter | func1 | func2
...

What to do with each of the functions (eval, table etc) would still have to be defined in the backends scritps. If a platform doesn't support a funtion it could simply stop processing aggregations or return a blank character. That way if I had a rule written for Splunk and I wanted to use it on ES it could retunrn the basic search with out all the other functions that ES might not support. This might be a good solution that lets us extend the functionality of sigma for speciffic platforms and at the same time maintains the generic nature we are aiming for..

Cheers