Crash in yara rule matching seemingly due to compiled rule word-size mismatch

[michaelweiser]

Hi,

I'm observing crashes caused by Yara rule matching when analysing program call chains containing both 32 and 64 bit programs on Windows 10. My test case is Microsoft Excel 2013 with a small XLS file containing a macro that does a WMI query. This causes (amongst lots of other things) svchost.exe and wmiprvse.exe processes being spawned which are 64 bit. These I have seen fail in three ways:

1. Access violation when trying to find the SRW lock using InternalYaraScan:

2. Hang in RtlLookupFunctionEntry as if the SRW lock could not be found using InternalYaraScan (but didn't crash either):

3. Access violation in caller_dispatch():

On a hunch I had capemon.dll write/load the compiled rules to/from separate files like so:

diff --git a/CAPE/YaraHarness.c b/CAPE/YaraHarness.c
index 982bc99..c7d2e0b 100644
--- a/CAPE/YaraHarness.c
+++ b/CAPE/YaraHarness.c
@@ -345,7 +345,11 @@ BOOL YaraInit()
                PathRemoveFileSpec(analyzer_path);
        PathRemoveFileSpec(analyzer_path);
        sprintf(yara_dir, "%s\\data\\yara", analyzer_path);
+#ifdef _WIN64
+       sprintf(compiled_rules, "%s\\capemon64.yac", yara_dir);
+#else
        sprintf(compiled_rules, "%s\\capemon.yac", yara_dir);
+#endif

        yr_initialize();

That made these problems go away. It appears, the initial hooking of the 32 bit excel.exe writes out rules which confuse a 64 bit yara in programs spawned later. What's peculiar is that the yara docs promise exactly this not to be the case: https://yara.readthedocs.io/en/stable/capi.html#saving-and-retrieving-compiled-rules

What could be the cause for me to see this behaviour?

Thanks! Michael

[kevoreilly]

Funnily enough I am currently looking at either this exact issue or a very similar one...

Yesterday @RaduEmanuel92 posted a reply to our previous thread with a deadlock in RtlLookupFunctionEntry which looks to be the same as your (2) above: https://github.com/kevoreilly/capemon/issues/12#issuecomment-1309003579

I haven't yet replied to this as I thought I would see if I could solve it first. But this issue occurs in standalone mode with a single process and no external yara rules... It looks very much like the old hang but this time the location via yara and subsequent check of LdrpInvertedFunctionTableSRWLock in our_stackwalk appears to be working, so I'm currently not quite sure why it's happening.

This leads me to be all the more intrigued by the change in behaviour you are seeing with your change to compiled rules handling. Perhaps there are two issues here.

Would it not be straightforward to just delete all the yara rules for your tests? This should leave the internal yara scan in tact but prevent the external rules being loaded, which might in turn allow you to narrow it down a bit further.

Thanks for bringing this to my attention.

[michaelweiser]

I saw that comment as well and thought it may also be some form of what I reported as #49. That's why I rushed to get both reports out today.

I have disabled compilation of .yar files and only left the internal yara scan active. The size of the yac file goes down notably from 32 to 6 KiB. Execution still ends in an access violation though. In my one try in caller_dispatch() again. If I read the debug output correctly, in this execution the internal yara scan did not find the SRW lock but did not end up in the deadlock either. This is all rather strange.

Here's the diff for reference:

diff --git a/CAPE/YaraHarness.c b/CAPE/YaraHarness.c
index c7d2e0b..bcfe009 100644
--- a/CAPE/YaraHarness.c
+++ b/CAPE/YaraHarness.c
@@ -345,11 +345,11 @@ BOOL YaraInit()
                PathRemoveFileSpec(analyzer_path);
        PathRemoveFileSpec(analyzer_path);
        sprintf(yara_dir, "%s\\data\\yara", analyzer_path);
-#ifdef _WIN64
-       sprintf(compiled_rules, "%s\\capemon64.yac", yara_dir);
-#else
+//#ifdef _WIN64
+//     sprintf(compiled_rules, "%s\\capemon64.yac", yara_dir);
+//#else
        sprintf(compiled_rules, "%s\\capemon.yac", yara_dir);
-#endif
+//#endif

        yr_initialize();

@@ -376,6 +376,7 @@ BOOL YaraInit()
                        goto exit;
                }

+#if 0
                char FindString[MAX_PATH];
                WIN32_FIND_DATA FindFileData;
                sprintf(FindString, "%s\\*.yar", yara_dir);
@@ -422,6 +423,7 @@ BOOL YaraInit()
                }
                else
                        DebugOutput("YaraInit: Found no Yara rules in %s\n", yara_dir);
+#endif

                // Add 'internal' yara
                if (yr_compiler_add_string(Compiler, InternalYara, NULL) != 0)

[kevoreilly]

In order to hopefully be able to investigate further, I need to try and recreate it. I have never seen this exception myself, as far as I know. Would you be able to share the xls file? As these detonations are quite dependent on Office flavours, and I don't have 2013 on my Windows 10 x64 vms, is there any other more straightforward file which triggers this exception?

[michaelweiser]

The XLS is here: https://github.com/scVENUS/PeekabooAV/blob/master/tests/test-data/office/CheckVM.xls

I was also just today pondering how to make headway with this issue and will try to come up with a testcase, e.g. a 32 bit cmd.exe starting a 64 bit cmd.exe or somesuch. Bear with me.

[michaelweiser]

So I've been running below as a .bat script and it doesn't trigger the problem:

c:\windows\syswow64\cmd.exe /k c:\windows\sysnative\cmd.exe /k for /l %%i in (1,1,1000) do echo %%i

There must be something else going on.

(note2self: syswow64 contains the 32 bit cmd and sysnative the 64 bit cmd, system32 diverts into syswow64 and sysnative depending on the wordsize of the calling program. Simples. :roll_eyes:)

[michaelweiser]

So I grabbed capemon.yac out of a VM where wmiprvse.exe crashed and it turns out that even a linux yara cli crashes a the same point trying to use it:

(gdb) run -m -g -C ../capemon.yac ../capemon.yac                                                                                                                                                                                                                                  
Starting program: /home/m/yara/yara -m -g -C ../capemon.yac ../capemon.yac                                                                                                                                                                                                        
[Thread debugging using libthread_db enabled]                                                                                                                                                                                                                                     
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".                                                               

Program received signal SIGSEGV, Segmentation fault.                                                                                     
0x0000555555591ca8 in yr_scan_verify_match (context=context@entry=0x5555555c5100, ac_match=ac_match@entry=0x7ffff6d4d0d8, data=data@entry=0x7ffff7fb3000 "YARA\021\f\226", data_size=30616, data_base=0, offset=6481) at scan.c:898                                               
898       if (STRING_IS_DISABLED(string))                                                                                                                                                                                                                                         
(gdb) bt                                                                                                                                                                                                                                                                          
#0  0x0000555555591ca8 in yr_scan_verify_match (context=context@entry=0x5555555c5100, ac_match=ac_match@entry=0x7ffff6d4d0d8, data=data@entry=0x7ffff7fb3000 "YARA\021\f\226", data_size=30616, data_base=0, offset=6481) at scan.c:898                                           
#1  0x0000555555564c0a in _yr_scanner_scan_mem_block (scanner=scanner@entry=0x5555555c5100, block_data=block_data@entry=0x7ffff7fb3000 "YARA\021\f\226", block=block@entry=0x7fffffffdbc0) at scanner.c:83                                                                        
#2  0x0000555555565499 in yr_scanner_scan_mem_blocks (scanner=scanner@entry=0x5555555c5100, iterator=iterator@entry=0x7fffffffdba0) at scanner.c:451                                                                                                                              
#3  0x00005555555658b1 in yr_scanner_scan_mem (buffer_size=<optimized out>, buffer=<optimized out>, scanner=0x5555555c5100) at scanner.c:570                                                                                                                                      
#4  yr_scanner_scan_file (scanner=0x5555555c5100, filename=filename@entry=0x7fffffffe729 "../capemon.yac") at scanner.c:584                                                                                                                                                       
#5  0x00005555555599ae in main (argc=2, argv=0x7fffffffe458) at cli/yara.c:1467                                                                                                                                                                                                   
(gdb) p string
$1 = (YR_STRING *) 0x0

I'm also sometimes seeing a SIGSEGV happen just a bit earlier in which case the match itself points to inaccessible memory (what!?):

(gdb) run -s -C ../capemon.yac ../capemon.yac
Starting program: /home/m/yara/yara -s -C ../capemon.yac ../capemon.yac
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
_yr_scanner_scan_mem_block (scanner=scanner@entry=0x5555555c5100, block_data=block_data@entry=0x7ffff7fb3000 "YARA\021\f\226", block=block@entry=0x7fffffffdbd0) at scanner.c:81
81              if (match->backtrack <= i)
(gdb) bt
#0  _yr_scanner_scan_mem_block (scanner=scanner@entry=0x5555555c5100, block_data=block_data@entry=0x7ffff7fb3000 "YARA\021\f\226", block=block@entry=0x7fffffffdbd0) at scanner.c:81
#1  0x0000555555565499 in yr_scanner_scan_mem_blocks (scanner=scanner@entry=0x5555555c5100, iterator=iterator@entry=0x7fffffffdbb0) at scanner.c:451
#2  0x00005555555658b1 in yr_scanner_scan_mem (buffer_size=<optimized out>, buffer=<optimized out>, scanner=0x5555555c5100) at scanner.c:570
#3  yr_scanner_scan_file (scanner=0x5555555c5100, filename=filename@entry=0x7fffffffe729 "../capemon.yac") at scanner.c:584
#4  0x00005555555599ae in main (argc=2, argv=0x7fffffffe468) at cli/yara.c:1467
(gdb) p match
$1 = (YR_AC_MATCH *) 0xf704e59300007fff
(gdb) p match->backtrack
Cannot access memory at address 0xf704e5930000801fff

The crash happens with a 64 bit yara 4.0.2 cli compiled on Windows 10 as well as a 32 bit and 64 bit yara 4.0.2 cli compiled on Ubuntu 22.04. So the compiled rule file seems to be corrupt somehow.

I'm wondering what the best way is to find out in what way exactly it is corrupt. I have been trying to compare the files but since they're binary and contain the rules in highly pre-processed form nothing obvious jumped out right away. Other than the corrupt one being smaller than when I compile the rules myself using yarac:

m@cape:~/yara$ ./yarac /opt/CAPEv2/analyzer/windows/data/yara/*.yar foo.yac
/opt/CAPEv2/analyzer/windows/data/yara/Guloader.yar(12): warning in rule "Guloader": $antihook is slowing down scanning
/opt/CAPEv2/analyzer/windows/data/yara/Guloader.yar(26): warning in rule "GuloaderB": $antihook is slowing down scanning
m@cape:~/yara$ ls -la ../capemon.yac foo.yac 
-rw-rw-r-- 1 m m 30616 Nov 16  2022 ../capemon.yac
-rw-rw-r-- 1 m m 30734 Nov 16 14:30 foo.yac

I expect the same to happen if I try to walk the code flow using a debugger since I'd first need to understand how the compiled yara rules and matches actually work. Am I missing some simple and obvious approach?

When I divert the rules-saving into bitness-specific files it appears that the capemon.yac as written out by the hooked excel.exe is always corrupt as it is identical to the one I extracted before. capemon64.yac as written out by the first hooked 64bit process is larger and works with the yara cli.

m@cape:~/yara$ sha256sum ../capemon* foo.yac 
b5a42ca6e02b23445f2189c87d58745ff68b9fcbe726cd9374a96998dad2a5a0  ../capemon32.yac
163ad433dcdebce8897abe35fe35bb702882a58b6a2394b34b46ab5a5b79c4a0  ../capemon64.yac
b5a42ca6e02b23445f2189c87d58745ff68b9fcbe726cd9374a96998dad2a5a0  ../capemon.yac
c2a9dbe280467c7c73902daf949f21955f2451ed27e3d6b90f93ef030f2f4d31  foo.yac
m@cape:~/yara$ ls -la ../capemon* foo.yac 
-rw-rw-r-- 1 m m 30616 Nov 16  2022 ../capemon32.yac
-rw-rw-r-- 1 m m 31872 Nov 16  2022 ../capemon64.yac
-rw-rw-r-- 1 m m 30616 Nov 16  2022 ../capemon.yac
-rw-rw-r-- 1 m m 30734 Nov 16 14:30 foo.yac

• capemon.yac extracted from VM where wmiprvse.exe crashed with capemon.dll without bitness-divert
• capemon32.yac and capemon64.yac extracted from VM with capemon.dll including betness-divert (no crashes)
• foo.yac as compiled using yarac on Linux above

[michaelweiser]

It appears, the 32 bit Windows version of yara generates different output than all the other versions:

m@cape:~/yara$ ls -la iys*
-rw-rw-r-- 1 m m 5501 Nov 17 08:15 iys32-nl.yac
-rw-rw-r-- 1 m m 5485 Nov 17 08:35 iys32-windows.yac
-rw-rw-r-- 1 m m 5501 Nov 17 08:36 iys32.yac
-rw-rw-r-- 1 m m 5501 Nov 17 08:35 iys64-windows.yac
-rw-rw-r-- 1 m m 5501 Nov 17 08:37 iys64.yac
-rw-rw-r-- 1 m m  413 Nov 17 08:35 iys.yar
m@cape:~/yara$ sha256sum iys*
fb10b85cb016d27773042a7de3238095529f7cc147255bc51bee6d827d8e730c  iys32-nl.yac
d9afe817be52918be8ef1b662e90e970e6674ab76ee475041ad5f47b9132ddf7  iys32-windows.yac
fb10b85cb016d27773042a7de3238095529f7cc147255bc51bee6d827d8e730c  iys32.yac
fb10b85cb016d27773042a7de3238095529f7cc147255bc51bee6d827d8e730c  iys64-windows.yac
fb10b85cb016d27773042a7de3238095529f7cc147255bc51bee6d827d8e730c  iys64.yac
05b39cab9c22be0a7df08f3f7499e979d744d6e63a14780ee9dbee5c850d8703  iys.yar

• iys.yar: the InternalYaraScan rule taken directly from the source without any formatting

  rule RtlInsertInvertedFunctionTable{strings:$10_0_19041_662 = {48 8D 0D [4] E8 [4] [7] 8B 44 24 ?? 44 8B CB 4C 8B 44 24 ?? 48 8B D7 89 44 24 ?? E8}$10_0_18362_1350 = {48 8D 0D [4] 33 D2 85 C0 48 0F 48 DA E8 [4] 33 C9 E8 [4] 8B 44 24 ?? 44 8B CF 4C 8B C3 89 44 24 ?? 48 8B D6 E8}$10_0_10240_16384 = {48 8D 0D [4] 48 8B E8 E8 [4] 33 C9 E8 [4] 8B 15 [4] 3B 15 [4] 0F 84}condition:uint16(0) == 0x5a4d and any of them}

• iys32.yac and iys64.yac: iys.yar compiled using 32 bit or 64 bit yarac 4.0.2 compiled on ubuntu 22.04, respecitively
• iys32-windows.yac and iys64-windows.yac: iys.yar compiled using 32 bit or 64 bit yarac 4.0.2 compiled on windows 10 using vs2019, respecitively
• iys32-nl.yac: iys.yar with newline formatting added for comparison compiled using 32 bit yarac 4.0.2 compiled on ubuntu 22.04

For validation I compiled all the yara rules including the internal yara scan rule using 32 and 64 bit yarac on Windows like so:

C:\Users\u\source\repos\yara\windows\vs2017\Release>yarac32.exe Al-khaser.yar Blister.yar BuerLoader.yar BumbleBee.yar DridexLoader.yar EmotetPacker.yar Formbook.yar GetTickCountAntiVM.yar Guloader.yar IcedID.yar MysterySnail.yar Pafish.yar QakBot.yar RdtscpAntiVM.yar SingleStepAntiHook.yar UrsnifV3.yar VBCrypter.yar Zloader.yar capemon32-windows.yac
Guloader.yar(12): warning in rule "Guloader": $antihook is slowing down scanning
Guloader.yar(26): warning in rule "GuloaderB": $antihook is slowing down scanning

C:\Users\u\source\repos\yara\windows\vs2017\Release>yarac64.exe Al-khaser.yar Blister.yar BuerLoader.yar BumbleBee.yar DridexLoader.yar EmotetPacker.yar Formbook.yar GetTickCountAntiVM.yar Guloader.yar IcedID.yar MysterySnail.yar Pafish.yar QakBot.yar RdtscpAntiVM.yar SingleStepAntiHook.yar UrsnifV3.yar VBCrypter.yar Zloader.yar capemon64-windows.yac
Guloader.yar(12): warning in rule "Guloader": $antihook is slowing down scanning
Guloader.yar(26): warning in rule "GuloaderB": $antihook is slowing down scanning

This produced files identical to what capemon.dll produces:

m@cape:~/yara$ ls -la capemon32-windows.yac capemon64-windows.yac ../capemon32.yac ../capemon64.yac 
-rw-rw-r-- 1 m m 30616 Nov 17 09:11 capemon32-windows.yac
-rw-rw-r-- 1 m m 30616 Nov 16 22:41 ../capemon32.yac
-rw-rw-r-- 1 m m 31872 Nov 17 09:11 capemon64-windows.yac
-rw-rw-r-- 1 m m 31872 Nov 16 22:56 ../capemon64.yac
m@cape:~/yara$ sha256sum capemon32-windows.yac capemon64-windows.yac ../capemon32.yac ../capemon64.yac 
b5a42ca6e02b23445f2189c87d58745ff68b9fcbe726cd9374a96998dad2a5a0  capemon32-windows.yac
163ad433dcdebce8897abe35fe35bb702882a58b6a2394b34b46ab5a5b79c4a0  capemon64-windows.yac
b5a42ca6e02b23445f2189c87d58745ff68b9fcbe726cd9374a96998dad2a5a0  ../capemon32.yac
163ad433dcdebce8897abe35fe35bb702882a58b6a2394b34b46ab5a5b79c4a0  ../capemon64.yac

• capemon32.yac and capemon64.yac extracted from VM with capemon.dll including bitness-divert (no crashes)
• capemon32-windows.yac and capemon64-windows.yac created using yarac32 and yarac64 4.0.2 on Windows, respectively

This discrepancy also happens with my small cmd.exe reproducer, it just doesn't seem to trigger the crash. I do see the internal yara scan being started in the analyzer output. But it never reports a match nor crashes. As observed before, the crash happens when a match is evaluated. So it would make sense for cmd.exe not to crash if the internal yara scan never produces a match. (There is also sometimes a timeout being reported for the scan after about two seconds. I've not looked into the cause.)

I'll validate with the latest version of yara and if it persists I'll look into the cause.

BTW: If I try to replace the libyara{32,64}.lib in the capemon source with ones I compiled myself I get errors like this even if it is the same version (checked out from git tag 4.0.2):

3>LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library
3>LINK : warning LNK4286: symbol '__wassert' defined in 'libucrtd.lib(assert.obj)' is imported by 'libyara32.lib(re.obj)'
[...]
3>LINK : warning LNK4286: symbol '__read' defined in 'libucrtd.lib(read.obj)' is imported by 'OLDNAMES.lib(read.obi)'
3>libyara32.lib(compiler.obj) : error LNK2019: unresolved external symbol __imp___filelength referenced in function __yr_compiler_default_include_callback
3>libyara32.lib(compiler.obj) : error LNK2019: unresolved external symbol __imp__read referenced in function __yr_compiler_default_include_callback
3>OLDNAMES.lib(read.obi) : error LNK2001: unresolved external symbol __imp__read
3>libyara32.lib(lexer.obj) : error LNK2019: unresolved external symbol __imp__clearerr referenced in function _yy_get_next_buffer
3>libyara32.lib(re_lexer.obj) : error LNK2001: unresolved external symbol __imp__clearerr
3>libyara32.lib(hex_lexer.obj) : error LNK2001: unresolved external symbol __imp__clearerr
3>libyara32.lib(lexer.obj) : error LNK2019: unresolved external symbol __imp__atof referenced in function _yara_yylex
3>libyara32.lib(math.obj) : error LNK2019: unresolved external symbol __imp__log2 referenced in function _data_entropy
3>libyara32.lib(value.obj) : error LNK2019: unresolved external symbol __imp___dclass referenced in function _json_real
3>libyara32.lib(strconv.obj) : error LNK2019: unresolved external symbol __imp__strtod referenced in function _jsonp_strtod
3>MSVCRT.lib(chandler4gs.obj) : error LNK2019: unresolved external symbol __except_handler4_common referenced in function __except_handler4
3>C:\Users\u\source\repos\capemon\Debug\capemon_dbg.dll : fatal error LNK1120: 8 unresolved externals

Do you have a special build procedure or local changes for the yara builds you include in capemon?

[michaelweiser]

Current yara master now consistently produces different files depending on word size of the compiler independent of the operating system but a 64 bit yara no longer crashes trying to use a file created by a 32 bit yarac.

So I'd like to update yara in capemon next but run into above error messages. Can you help?

m@cape:~/yara$ ls -la ../capemon32.yac ../capemon64.yac capemon32-windows.yac capemon64-windows.yac capemon32-*master.yac capemon64-*master.yac 
-rw-rw-r-- 1 m m 29543 Nov 17 09:31 capemon32-master.yac
-rw-rw-r-- 1 m m 29543 Nov 17 09:25 capemon32-windows-master.yac
-rw-rw-r-- 1 m m 30616 Nov 17 09:11 capemon32-windows.yac
-rw-rw-r-- 1 m m 30616 Nov 16 22:41 ../capemon32.yac
-rw-rw-r-- 1 m m 30571 Nov 17 09:30 capemon64-master.yac
-rw-rw-r-- 1 m m 30571 Nov 17 09:25 capemon64-windows-master.yac
-rw-rw-r-- 1 m m 31872 Nov 17 09:11 capemon64-windows.yac
-rw-rw-r-- 1 m m 31872 Nov 16 22:56 ../capemon64.yac
m@cape:~/yara$ sha256sum ../capemon32.yac ../capemon64.yac capemon32-windows.yac capemon64-windows.yac capemon32-*master.yac capemon64-*master.yac 
b5a42ca6e02b23445f2189c87d58745ff68b9fcbe726cd9374a96998dad2a5a0  ../capemon32.yac
163ad433dcdebce8897abe35fe35bb702882a58b6a2394b34b46ab5a5b79c4a0  ../capemon64.yac
b5a42ca6e02b23445f2189c87d58745ff68b9fcbe726cd9374a96998dad2a5a0  capemon32-windows.yac
163ad433dcdebce8897abe35fe35bb702882a58b6a2394b34b46ab5a5b79c4a0  capemon64-windows.yac
c006a745feda0bab77b9b37144c383e9c15dbea81a9cbf56b0e5f4aa9278c7e6  capemon32-master.yac
c006a745feda0bab77b9b37144c383e9c15dbea81a9cbf56b0e5f4aa9278c7e6  capemon32-windows-master.yac
123301d86b23ba1225c0660eb95ec091110a4f0f0ff8a066dbcaecc4406e8bc7  capemon64-master.yac
123301d86b23ba1225c0660eb95ec091110a4f0f0ff8a066dbcaecc4406e8bc7  capemon64-windows-master.yac

• capemon32.yac and capemon64.yac extracted from VM with capemon.dll including bitness-divert (no crashes)
• capemon32-windows.yac and capemon64-windows.yac created using yarac32 and yarac64 4.0.2 on Windows, respectively
• capemon32-master.yac and capemon64-master.yac created using 32 and 64 bit yarac git HEAD compiled on Ubuntu 22.04, respectively
• capemon32-windows-master.yac and capemon64-windows-master.yac created using yarac32 and yarac64 git HEAD on Windows, respectively

m@cape:~/yara$ ./yara --version
4.2.1
m@cape:~/yara$ file yara .libs/yara
yara:       Bourne-Again shell script, ASCII text executable
.libs/yara: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=3fb6fffc3f3c855ddcea6f946b60ccbc94db3037, for GNU/Linux 3.2.0, with debug_info, not stripped
m@cape:~/yara$ ./yara -m -C capemon64-windows-master.yac capemon32-windows.yac 
Guloader [author="kevoreilly",description="Guloader bypass",cape_options="bp0=$trap0,bp0=$trap1+4,action0=skip,bp1=$trap2+11,bp1=$trap3+19,action1=skip,bp2=$antihook,action2=goto:ntdll::NtAllocateVirtualMemory,count=0"] capemon32-windows.yac
UrsnifV3 [author="kevoreilly",description="Ursnif Config Extraction",cape_options="br0=$crypto32-73,dumpsize=eax,action1=dump:ebx,typestring1=UrsnifV3 Config,bp2=$timing_trap-6,action2=setesi:9,count=1"] capemon32-windows.yac
m@cape:~/yara$ ./yara -m -C capemon32-windows-master.yac capemon32-windows.yac 
Guloader [author="kevoreilly",description="Guloader bypass",cape_options="bp0=$trap0,bp0=$trap1+4,action0=skip,bp1=$trap2+11,bp1=$trap3+19,action1=skip,bp2=$antihook,action2=goto:ntdll::NtAllocateVirtualMemory,count=0"] capemon32-windows.yac
UrsnifV3 [author="kevoreilly",description="Ursnif Config Extraction",cape_options="br0=$crypto32-73,dumpsize=eax,action1=dump:ebx,typestring1=UrsnifV3 Config,bp2=$timing_trap-6,action2=setesi:9,count=1"] capemon32-windows.yac
m@cape:~/yara$ git checkout -f v4.0.2
[...]
m@cape:~/yara$ ./configure --enable-debug && make clean && make -j
[...]
m@cape:~/yara$ ./yara -m -C capemon64-windows-master.yac capemon32-windows.yac 
rules were compiled with a different version of YARA
m@cape:~/yara$ ./yara -m -C capemon64-windows.yac capemon32-windows.yac 
Guloader [author="kevoreilly",description="Guloader bypass",cape_options="bp0=$trap0,bp0=$trap1+4,action0=skip,bp1=$trap2+11,bp1=$trap3+19,action1=skip,bp2=$antihook,action2=goto:ntdll::NtAllocateVirtualMemory,count=0"] capemon32-windows.yac
UrsnifV3 [author="kevoreilly",description="Ursnif Config Extraction",cape_options="br0=$crypto32-73,dumpsize=eax,action1=dump:ebx,typestring1=UrsnifV3 Config,bp2=$timing_trap-6,action2=setesi:9,count=1"] capemon32-windows.yac
m@cape:~/yara$ ./yara -m -C capemon32-windows.yac capemon32-windows.yac 
error scanning capemon32-windows.yac: error: 4
m@cape:~/yara$ strace ./yara -m -C capemon32-windows.yac capemon32-windows.yac 2>&1 | grep SEGV
rt_sigaction(SIGSEGV, {sa_handler=0x55c7088ccc70, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7fa9236d8520}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGSEGV, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7fa9236d8520}, NULL, 8) = 0
rt_sigaction(SIGSEGV, {sa_handler=0x55c7088ccc70, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7fa9236d8520}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7fa9236d8520}, 8) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=NULL} ---
rt_sigaction(SIGSEGV, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7fa9236d8520}, NULL, 8) = 0

[kevoreilly]

Hi Michael, thank you for this thorough investigation. I will have a look at how I compiled yara into capemon previously - I recall it was a bit of a pain, alas I didn't write detailed notes, but hopefully we can get it updated.

[michaelweiser]

Hey Kev, thanks! Just to be clear: I'm by no means certain a newer yara will solve the issue. It's just the next logical thing to try so we don't waste time debugging a problem that has maybe been solved by yara upstream already. And being able to update and debug yara seems a worthwhile goal in itself to me.

My test of scanning the yac file with itself is by no means conclusive. In fact I was surprised I was able to reproduce a crash and even the exact same crash by doing so.

I read the link errors as libyara using msvcrt functions like read() and strtod() and msvcrt not being linked into capemon.dll. I'm just unsure what the correct solution is with regards to capemon: Avoid/replace the function calls somehow to avoid a dependency on msvcrt, just link it dynamically (maybe causing a need to install the visual c++ runtime redistributable) or link a static version of it (maybe causing symbol conflicts with other modules doing the same). I'm not enough of a Windows programmer to know the best practice here and capemon likely has a very special set of requirements beyond that.

[kevoreilly]

No worries even if the upgrade doesn't solve this, it will be one less chore and excluded from the possibilities. I spent some time on Friday trying to 'quickly' compile updated lib but this failed. I have some vague memories of having to compile the dependencies (OpenSSL and Janssen) separately the first time then manually combining three lib files... I wish I had documented it better but I'll keep plugging away until I get it working and make sure to document it this time!

[michaelweiser]

Looking at https://stackoverflow.com/questions/68330445/linker-can-not-find-imp-clock-and-imp-time64-when-building-a-sample-of-gtsam and recompiling libyara with /MT in line with capemon, I'm now down to these errors:

2>------ Build started: Project: capemon, Configuration: Release Win32 ------
2>LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/LTCG' specification
2>   Creating library C:\Users\u\source\repos\capemon\Release\capemon.lib and object C:\Users\u\source\repos\capemon\Release\capemon.exp
2>LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library
2>libyara32.lib(value.obj) : error LNK2001: unresolved external symbol __imp___dclass
2>libyara32.lib(strconv.obj) : error LNK2001: unresolved external symbol __imp__strtod
2>MSVCRT.lib(chandler4gs.obj) : error LNK2001: unresolved external symbol __except_handler4_common
2>C:\Users\u\source\repos\capemon\Release\capemon.dll : fatal error LNK1120: 3 unresolved externals
2>Done building project "capemon.vcxproj" -- FAILED.
========== Build: 1 succeeded, 1 failed, 1 up-to-date, 0 skipped ==========

These look they stem from libcrypto or libjansson included in libyara because there's no strconv.obj or value.obj anywhere in the yara build. So recompiling those libs with /MT might do the trick now.

[michaelweiser]

I managed to build jannson libs with private runtime which I was able to link into yara HEAD and that again into capemon without errors. Here are my build commands for jansson v2.9 (which is what yara uses regardless of the confusing nuget package version 1.0.1 which might refer to an sover):

md build32
cd build32
cmake -G "Visual Studio 17" -A win32 -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded ..
cd ..
md build64
cd build64
cmake -G "Visual Studio 17" -A x64 -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded ..
cd ..
cmake --build build32 --config Release
cmake --build build64 --config Release

• CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded enables passing flag /MT to the compiler which causes the runtime to be compiled in
• CMAKE_POLICY_DEFAULT_CMP0091=NEW enables use of CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded - otherwise it's ignored
• -G is the generator and needs adjusting for different Visual Studio versions

In a first few quick tests, 64bit wmiprvse.exe activated by 32bit excel.exe no longer seems to crash but it doesn't seem to be working fully either. I'll keep digging.

[michaelweiser]

It appears the remaining analysis problems seem to have been due to #52. With this resolved my tests run smoothly now.

[kevoreilly]

But the yara issue persists?

[michaelweiser]

Not at all. That seems to have been cured by the upgrade to yara devel HEAD with libjansson 2.9.

[kevoreilly]

Is there any chance you could share the lib files, or outline the steps you took to get there? I've found compiling this really difficult so any help you can offer would be appreciated.

[michaelweiser]

Hello Kev,

happy new year!

First off: I have just verified that #52 is not enough to fix the yara rule miscompilation issue. So the yara update is needed as well to make everything work.

I have also verified that updating to yara release 4.2.3 is enough to make the issue go away - devel HEAD is not required.

I'll try to list all the steps I took to upgrade the libs:

0. install Visual Studio 2022 Community Edition (and potentially git for windows if like me you find git support in VS counterintuitive), clone capemon (duh :)

1. clone jansson from https://github.com/akheron/jansson

2. check out tag v2.9

3. install cmake

4. build jansson (details are in comment https://github.com/kevoreilly/capemon/issues/48#issuecomment-1329360072):

   md build32
   cd build32
   cmake -G "Visual Studio 17" -A win32 -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded ..
   cd ..
   md build64
   cd build64
   cmake -G "Visual Studio 17" -A x64 -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded ..
   cd ..
   cmake --build build32 --config Release
   cmake --build build64 --config Release

5. clone yara from https://github.com/VirusTotal/yara

6. check out tag 4.2.3

7. open yara\windows\vs2017\yara.sln in Visual Studio

8. retarget solution as necessary to make it compile with the toolsets and SDKs available

9. in properties of project libyara for configuration Release and platforms x86 and x64 under C/C++ / Code Generation change Runtime Library from Multi-threaded DLL (/MD) to Multi-threaded (/MT)

10. build any configuration of libyara once: this will download and unpack openssl and jansson nuget packages and place them into windows\vs2017\packages

11. copy jansson\build32\lib\Release\jansson.lib to yara\windows\vs2017\packages\YARA.Jansson.x86.1.1.0\lib

12. copy jansson\build32\include\* to yara\windows\vs2017\packages\YARA.Jansson.x86.1.1.0\include

13. (re-)build project libyara configuration Release for platform x86

14. copy yara\windows\vs2017\libyara\Release\libyara32.lib (and optionally libyara32.pdb if you want to do debugging) to capemon\libyara\lib

15. copy jansson\build64\lib\Release\jansson.lib to yara\windows\vs2017\packages\YARA.Jansson.x64.1.1.0\lib

16. copy jansson\build64\include\* to yara\windows\vs2017\packages\YARA.Jansson.x64.1.1.0\include

17. (re-)build configuration Release for platform x64 - note that building libyara64.lib will delete libyara32.lib, so building and saving the output will have to be done in this order before moving on to the next target

18. copy yara\windows\vs2017\libyara\Release\libyara64.lib (and again optionally libyara64.pdb) to capemon\libyara\lib

19. replace the contents of capemon\libyara\include with those of yara\libyara\include to keep them in sync with the new libs

20. open capemon\capemon.sln and (re-)build with the new libs which should succeed now without any unresolved symbols

[kevoreilly]

EDIT - seems I forgot to factor in the break and sleep required to obtain a successful result. Turns out my Cmake was slightly too old for those options. I updated it and it now works.

Hopefully I will publish a new monitor with Yara 4.3.2 soon and we can close this issue.

Thanks for your help

[kevoreilly]

Not sure if this is the best place for this but I've noticed that if there are any exceptions in capemon arising from within the yara code, there are assertion dialogs produced on the desktop. The setting we need for capemon is for it to die in silence - I've tried googling for the cmake instructions to achieve this but have so far drawn a blank. Any idea?