Ascii/Wide matching bug on some regexes

[vthib]

Describe the bug

With some regexes marked as "ascii wide", no matches are reported on inputs that should match.

To Reproduce

Here are some tests cases showing the issue:

// Works
assert_true_rule_blob(
    "rule test { strings: $a = /a.{1}c/ ascii wide condition: $a }",
    "a\0b\0c\0");

// Fails
assert_true_rule_blob(
    "rule test { strings: $a = /a.{1}1/ ascii wide condition: $a }",
    "a\0b\01\0");

Expected behavior

The second test case should match.

• YARA version: HEAD (1c309a8).

Additional context

The regex might seem a bit far-fetched, I have actually stumbled upon the issue with the regex a.{5}i, and just reduced for the reproducibility.

The bug does not exhibit when using "a.i". From what I can understand from some debugging:

• For the "a.{1}c" regex, the "a" literal is extracted. On a match, we do:
  • an ascii forward match: this fails
  • a wide forward match: this succeeds.
  • the backward validator does nothing
  • there is match
• For the "a.{1}1" regex, the "1" literal is extracted. On a match, we do:
  • an ascii forward match: this succeeds as it matches "1"
  • an ascii backward match: this fails
  • there is no match
• The "a.i" regex gets expanded into literals, hence no bug as it is related to ascii/wide detection on the AC literal matched.

So from what i can tell, the issue can happen when the ascii forward matcher is a prefix of the wide forward matcher. I should thus be possible to trigger with raw strings or hex strings, or even with some clever xor transformation, however I haven't tried to come up with more "realistic" strings that could trigger this bug.

[plusvic]

Fixed in 95dafbdb3c59497771fc6991890d53e962a16eaa

[vthib]

Thanks for the fix! Do you have a released planned in the near future? I would love to get one to get the fix for this issue and #1928

[vthib]

ping @plusvic, would it be possible to make a new release given the numerous fixes since the last one? That would be great :)

[plusvic]

Sorry for the delay @vthib, I'm starting to work in a 4.4.0 release that I hope to release soon.