search

intel / hyperscan

High-performance regular expression matching library
https://www.hyperscan.io
Other
4.78k stars 710 forks source link

Hyperscan crashes when built for 32-bit Windows #249

Open eliaslevy opened 4 years ago

eliaslevy commented 4 years ago

We've found that Hyperscan crashes reliably when built to target 32-bit Windows.

To reproduce:

cl simplegrep.c /I hyperscan-win-32\include /link hs.lib /libpath:hyperscan-win-32\lib
simplegrep "(crypto::(capi|certificates|scauth|cng|keys|stores|providers|sc)|dpapi::(capi|wwan|blob|cng|wifi|cache|chrome|cred|vault|credhist|masterkey|protect)|kerberos::(silver|ptt|tgt|golden|purge|list)|lsadump::(trust|dcsync|dcshadow|sam|rpdata|cache|lsa|netsync|secrets|backupkeys)|sekurlsa::(logonpasswords|process|credman|wdigest|ssp|kerberos|livessp|tickets|dpapi|msv|searchpasswords|tspkg|pth|ekeys|minidump)|token::(list|whoami|revert|elevate)|vault::(list|cred)|privilege::debug|ts::multirdp)" hyperscan-win-32\include\hs_common.h

The return code is

echo %errorlevel%
-1073741819

In the the Event View, Windows Logs, Application, you'll see an event like:

Exception thrown at 0x51A8EDCB (hs.dll) in simplegrep.exe: 0xC0000005: Access violation reading location 0xA4DC6A68.

The call stack when the crash occurs is:

In fd_engine_exec case 4 is taken, which has code generated by the macro call FDR_MAIN_LOOP(z, state, get_conf_stride_4). This macro generates code that calls a number of functions that are inlined. Some of those make use of intrinsic instructions

The access violation appear to occur in load_m128_from_u64a, in hs.dll, and defined in simd_utils.h, when it calls the _mm_set_epi64x intrinsic, which tries to access an invalid address . load_m128_from_u64a is called from get_conf_stride_4, the second instance of a call (load_m128_from_u64a(ft + reach4))

The same occurs if we use the 32-bit toolchain to target 32-bit (i.e. using vcvars32.bat to set up the environment).

eliaslevy commented 4 years ago

Has someone from the Hyperscan team reproduced this issue?

eliaslevy commented 4 years ago

I've tried reproducing the error in 32-bit Linux using a 32-bit Debian Docker image in a x64 host without success. The issue seems specific to 32-bit Windows.

gliwka commented 3 years ago

Can confirm, same issue here. 32-bit Linux (Centos 6, Devtoolset 7) works fine.

variar commented 3 years ago

Looks like I've stumbled upon same issue in my project. Unfortunately, I can't reproduce it myself, but got several crash dumps from users with similar callstack:

OS Version: Windows 10.0.19041 (928)
Report Version: 104

Crashed Thread: 15840

Application Specific Information:
Fatal Error: EXCEPTION_ACCESS_VIOLATION_READ

Thread 15840 Crashed:
0   klogg.exe                       0x917c03            [inlined] load_m128_from_u64a (simd_utils.h:186)
1   klogg.exe                       0x917c03            [inlined] nfaExecLimEx64_Stream (limex_runtime_impl.h:253)
2   klogg.exe                       0x917c03            nfaExecLimEx64_Stream_Silent (limex_runtime_impl.h:570)
3   klogg.exe                       0x91198f            nfaExecLimEx64_QR (limex_runtime_impl.h:890)
4   klogg.exe                       0x9a50dc            roseRunProgram (program_runtime.c:2486)
5   klogg.exe                       0x758a51            [inlined] roseProcessMatchInline (match.c:244)
6   klogg.exe                       0x758a51            [inlined] roseCallback_i (match.c:512)
7   klogg.exe                       0x758a51            roseFloatingCallback (match.c:533)
8   klogg.exe                       0xaf11b5            [inlined] confWithBit (fdr_confirm_runtime.h:96)
9   klogg.exe                       0xaf11b5            [inlined] do_confWithBit_teddy (teddy_runtime_common.h:438)
10  klogg.exe                       0xaf11b5            fdr_exec_teddy_msks3 (teddy.c:1095)
11  klogg.exe                       0x8c8811            fdrExec (fdr.c:849)
12  klogg.exe                       0x714baa            hwlmExec (hwlm.c:198)
13  klogg.exe                       0x735182            [inlined] roseBlockFloating (block.c:259)
14  klogg.exe                       0x735182            roseBlockExec (block.c:398)
15  klogg.exe                       0x6f2ea1            [inlined] rawBlockExec (runtime.c:188)
16  klogg.exe                       0x6f2ea1            hs_scan (runtime.c:419)

This is on Windows 10.0.19041 x86. My programs links statically with hyperscan library.

bo3b commented 2 years ago

Entirely possible I have no idea what I'm talking about, but why is it being built for Debian?

-DCMAKE_BUILD_TYPE=RelWithDebInfo

I just set up a Github build system for this, and I'm specifically using MinSizeRel for Windows builds. We haven't seen any crashes so far.

liquidaty commented 2 years ago

@bo3b the "Deb" in RelWithDebInfo means "debug", not Debian.

I'm seeing a similar issue as the OP, built with mingw-w64. simplegrep.exe with no args prints usage just fine, but the actual search immediately crashes no matter how trivial the pattern or input file

bo3b commented 2 years ago

@bo3b the "Deb" in RelWithDebInfo means "debug", not Debian.

I'm seeing a similar issue as the OP, built with mingw-w64. simplegrep.exe with no args prints usage just fine, but the actual search immediately crashes no matter how trivial the pattern or input file

Ah, right, duh. I'd say too much Linux for me recently.

Not sure if it's worth experimenting, but I created an alternate build using Github Actions for Nuget, and there is a 32 bit release build available there. https://github.com/bo3b/hyperscan-nuget

liquidaty commented 2 years ago

@bo3b thx, will check it out! BTW resolved my issue by swapping __mingw_aligned_malloc for posix_memalign and using __mingw_aligned_free in aligned_free_internal in src/util/alloc.cpp. Probably different from whatever is causing OP's issue but wanted to close the loop on prior comment