Equality matching of regexes in expressions is currently implemented
using re.match. This was done purely from the perspective of reflecting
both python matching primitives (re.match and re.search) in a useful
manner in our expression language and we were aware of their behaviour
at the time.
It turns out, however, that this behaviour is neither naturally expected
behaviour when looking at an equality operator nor very useful. It
immediately caught us out when writing our own example ruleset
expressions in that we wrote ignore matches for S/MIME signature
attachments that would look at only the beginning of filenames. This
allows evasion of analysis by simply letting the attachment name begin
with one of the configured patterns.
Change equality machting of regexes to require a match from beginning to
end of the operand by adding an explicit end-of-line anchor to the
regex. We still keep using re.match to keep the change small. Add test
coverage to both basic expression parser as well as expression rule
testing to catch regressuions on this.
It could be considered to later switch to re.search for both operators,
adding the beginning-of-line anchor for re.search explicitly as well.
There is at least one remaining difference in behaviour regarding
multiline mode which we do not currently use, though. We add a
cautionary comment to that effect.
Equality matching of regexes in expressions is currently implemented using re.match. This was done purely from the perspective of reflecting both python matching primitives (re.match and re.search) in a useful manner in our expression language and we were aware of their behaviour at the time.
It turns out, however, that this behaviour is neither naturally expected behaviour when looking at an equality operator nor very useful. It immediately caught us out when writing our own example ruleset expressions in that we wrote ignore matches for S/MIME signature attachments that would look at only the beginning of filenames. This allows evasion of analysis by simply letting the attachment name begin with one of the configured patterns.
Change equality machting of regexes to require a match from beginning to end of the operand by adding an explicit end-of-line anchor to the regex. We still keep using re.match to keep the change small. Add test coverage to both basic expression parser as well as expression rule testing to catch regressuions on this.
It could be considered to later switch to re.search for both operators, adding the beginning-of-line anchor for re.search explicitly as well. There is at least one remaining difference in behaviour regarding multiline mode which we do not currently use, though. We add a cautionary comment to that effect.