Yara 4.2.0 regression when scanning pseudo processes on Linux

[mat-gas]

Hi,

there's a regression in Linux process scan for pseudo processes (kernel threads) in YARA 4.2.0 compared to 4.1.3

it works as expected on "real" processes (here 614840)

pip3 install yara-python==4.2.0

>>> import yara
>>> a=yara.compile('/tmp/rule.yar')
>>> a.match(pid=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.Error: internal error: 788577472
>>> a.match(pid=2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.Error: internal error: 745744400
>>> a.match(pid=614840)
[]

pip3 install yara-python==4.1.3

>>> import yara
>>> a=yara.compile('/tmp/rule.yar')
>>> a.match(pid=3)
[]
>>> a.match(pid=2)
[]
>>> a.match(pid=614840)
[]

ps faux|head
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           2  0.0  0.0      0     0 ?        S    mars11   0:00 [kthreadd]
root           3  0.0  0.0      0     0 ?        I<   mars11   0:00  \_ [rcu_gp]
root           4  0.0  0.0      0     0 ?        I<   mars11   0:00  \_ [rcu_par_gp]
root           6  0.0  0.0      0     0 ?        I<   mars11   0:00  \_ [kworker/0:0H-kblockd]
...
root      614840  0.0  0.0 241996 26760 ?        Sl   10:20   0:00 /usr/bin/kwalletd5

[mat-gas]

I also tried using yara 4.2.0 in commandline and it works as intended (no error):

cd /tmp/yara-4.2.0/
/tmp/yara-4.2.0$ ./build.sh 
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
....
/tmp/yara-4.2.0$  sudo yara  /tmp/rule.yar 3

==> no output, everything is "fine"

/tmp/yara-4.2.0$ yara  /tmp/rule.yar 3
/tmp/yara-4.2.0$ echo $?
0
/tmp/yara-4.2.0$ yara  /tmp/rule.yar 3333333
error scanning 3333333: can not attach to process (try running as root)  <== PID 3333333 doesn't exist here
/tmp/yara-4.2.0$ echo $?
1

but when using yara-python 4.2.0, I get an error (with an "internal error" different at each python startup)

/tmp/yara-4.2.0$ sudo python3
Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import yara
>>> a=yara.compile('/tmp/rule.yar')
>>> a.match(pid=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.Error: internal error: 284145600
>>> 
/tmp/yara-4.2.0$ sudo python3
Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import yara
>>> a=yara.compile('/tmp/rule.yar')
>>> a.match(pid=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.Error: internal error: 1257912256

Food for thoughts:

in Rules_match , yr_scanner_scan_proc seems to return a non standard error

iterator is initialized in yr_process_open_iterator

BUT field last_error seems to be never initialized

and in yr_scanner_scan_mem_blocks, result is taken from here directly

It could be the source of the bug where we "cannot" scan the process memory and take last_error from uninitialized memory

[mat-gas]

Confirmed that the error comes from uninitialized memory:

tried setting 2 values to last_error, and indeed got this error when using python

YR_API int yr_scanner_scan_proc(YR_SCANNER* scanner, int pid)
{
  YR_MEMORY_BLOCK_ITERATOR iterator;

  int result = yr_process_open_iterator(pid, &iterator);

  // XXX : added
  iterator.last_error = ERROR_COULD_NOT_MAP_FILE;
  iterator.last_error = ERROR_UNSUPPORTED_FILE_VERSION;
  // XXX : added

  if (result == ERROR_SUCCESS)
  {
    int prev_flags = scanner->flags;
    scanner->flags |= SCAN_FLAGS_PROCESS_MEMORY;
    result = yr_scanner_scan_mem_blocks(scanner, &iterator);
    scanner->flags = prev_flags;
    yr_process_close_iterator(&iterator);
  }

  return result;

sudo python3
Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import yara
>>> a=yara.compile('/tmp/rule.yar')
>>> a.match(pid=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.Error: could not map file "<proc>" into memory  <==== ERROR_COULD_NOT_MAP_FILE;
>>> 
/tmp/yara-python$ sudo python3
Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import yara
>>> a=yara.compile('/tmp/rule.yar')
>>> a.match(pid=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
yara.Error: rules file "<proc>" is incompatible with this version of YARA  <===  ERROR_UNSUPPORTED_FILE_VERSION;

but it doesn't change the fact that it's "failing" in 4.2.0 whereas it didn't fail on 4.1.3 (or failed gracefully)

[mat-gas]

seems somehow related to this : https://github.com/VirusTotal/yara/issues/1671

[plusvic]

The original issue https://github.com/VirusTotal/yara/issues/1671 was already solved.