[sigmac] conversion to [elasticsearch] is incorrectly escaped for regex rule

canilc commented 1 year ago

Hello,

I think conversion of the following rule produces incorrectly escaped elasticsearch equivalent:

/rules/windows/process_creation/proc_creation_win_invoke_obfuscation_clip.yml

The rule was updated recently, but I was working on a previous version which is:

title: Invoke-Obfuscation CLIP+ Launcher
id: b222df08-0e07-11eb-adc1-0242ac120002
status: test
description: Detects Obfuscated use of Clip.exe to execute PowerShell
references:
    - https://github.com/Neo23x0/sigma/issues/1009  #(Task 26)
author: Jonathan Cheong, oscd.community
date: 2020/10/13
modified: 2022/11/17
tags:
    - attack.defense_evasion
    - attack.t1027
    - attack.execution
    - attack.t1059.001
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        CommandLine|re: '.*cmd.{0,5}(?:\/c|\/r).+clip(?:\.exe)?.{0,4}&&.+clipboard]::\(\s\\\"\{\d\}.+\-f.+\"'
        # Example 1: Cmd /c" echo/Invoke-Expression (New-Object Net.WebClient).DownloadString |cLiP&& POWerSheLl -Nolog -sT . (\"{1}{2}{0}\"-f'pe','Ad',(\"{1}{0}\" -f'Ty','d-' ) ) -Assemb ( \"{5}{1}{3}{0}{2}{4}\" -f'ows','y','.F',(\"{0}{1}{2}\" -f'stem.W','i','nd'),( \"{0}{1}\"-f 'o','rms' ),'S' ) ; ([SySTEM.wiNDows.FoRmS.CLiPbOArd]::( \"{1}{0}\" -f (\"{1}{0}\" -f'T','TTeX' ),'gE' ).\"invO`Ke\"( ) ) ^| ^&( \"{5}{1}{2}{4}{3}{0}\" -f 'n',( \"{1}{0}\"-f'KE-','o' ),(\"{2}{1}{0}\"-f 'pRESS','x','e' ),'o','i','iNV') ; [System.Windows.Forms.Clipboard]::(\"{0}{1}\" -f( \"{1}{0}\"-f'e','SetT' ),'xt').\"InV`oKe\"( ' ')"
        # Example 2: CMD/c " ECho Invoke-Expression (New-Object Net.WebClient).DownloadString|c:\WiNDowS\SySteM32\cLip && powershElL -noPRO -sTa ^& (\"{2}{0}{1}\" -f 'dd',(\"{1}{0}\"-f 'ype','-T' ),'A' ) -AssemblyN (\"{0}{3}{2}{1}{4}\"-f'Pr','nCo',(\"{0}{1}\"-f'e','ntatio'),'es','re' ) ; ^& ( ( [StRinG]${ve`RB`OSE`pr`e`FeReNCE} )[1,3] + 'x'-JoiN'') ( ( [sySTem.WInDOWs.ClipbOaRD]::( \"{1}{0}\" -f(\"{0}{1}\" -f'tTe','xt' ),'ge' ).\"IN`Vo`Ke\"( ) ) ) ; [System.Windows.Clipboard]::( \"{2}{1}{0}\" -f't',( \"{0}{1}\" -f 'tT','ex' ),'Se' ).\"In`V`oKe\"( ' ' )"
        # CommandLine|contains|all:
        #     - 'cmd'
        #     - '&&'
        #     - 'clipboard]::'
        #     - '-f'
        # CommandLine|contains:
        #     - '/c'
        #     - '/r'
    condition: selection
falsepositives:
    - Unknown
level: high

(Escaped the new lines to reproduce the bug)

So the conversion of

CommandLine|re: '.*cmd.{0,5}(?:\/c|\/r).+clip(?:\.exe)?.{0,4}&&.+clipboard]::\(\s\\\"\{\d\}.+\-f.+\"'

outputs:

EventID:1 AND CommandLine:/.*cmd.{0,5}(?:\\/c|\\/r).+clip(?:\\.exe)?.{0,4}&&.+clipboard]::\\(\\s\\\\\\"\\{\\d\\}.+\\-f.+\\"/

(?:\\/c|\\/r) part is incorrectly escaped which should've been (?:\/c|\/r) (because it already escapes another character in the regex there).

The command used to produce the output above is:

sigma convert -t elasticsearch -p sysmon ./rules/windows/process_creation/proc_creation_win_invoke_obfuscation_clip.yml

nasbench commented 1 year ago

cc @thomaspatzke

fukusuket commented 1 year ago

I think the following escapes are unnecessary. This is because the following characters are not regex metacharacters:

\/c　
\/r
\"
\-

I fixed a similar escaping issue yesterday. Can you solve it with the same fix as here? https://github.com/SigmaHQ/sigma/pull/3744

thomaspatzke commented 1 year ago

Tranferred to project where issue is contained.

gal-dd commented 1 year ago

Hi!

Can you explain to me the idea behind the escaping in |re block? e.g. if I want to match exact ip and some 4 digits port for example:

127.0.0.1:8000

the block should be:

ip|re: 127\.0\.0\.1:[1-9]\d{3}

The expecting result (with elasticsearch backend) should be:

ip:/127\.0\.0\.1:[1-9]\d{3}/

but the result is: ip:/127\\.0\\.0\\.1:[1-9]\\d{3}/.

clearly I don't want the \. and the \d to be escaped as they part of the pattern.

andurin commented 1 year ago

I guess there are 2 different issues.

Too many escaping within the rule from @canilc which should be resolvable with the hints of @fukusuket .
ES Backend has to many escapes which should be fixed by https://github.com/SigmaHQ/pySigma-backend-elasticsearch/commit/563c56526e1acfbf87db22fa331fb5428755b780

Thanks to you all - we have a new test-case and could do things better.

fukusuket commented 1 year ago

@andurin Thanks for the quick fix :)

FYI:

I recently fixed the following Sigma |re rules.

I checked how to escape in multiple programming languages for the fix as follows.

I would appreciate it if you could refer to it.

SigmaHQ / pySigma-backend-elasticsearch

[sigmac] conversion to [elasticsearch] is incorrectly escaped for regex rule #9