Open trythreetimes opened 5 years ago
We don't enable pcre JIT as it conflicts with database serialization and deserialization feature shared by Chimera and Hyperscan. I think this is a useful feature by our users and we try to avoid running pcre by making Hyperscan as a fast filter in Chimera.
Thank you for your response. I tried to set it, but it led to memory leak.
Did you make any code modifications?
Oh, yes, i modified a lot to add character iterator and \G for chimera, and i removed the stream mode because it is useless for chimera, but i don't think my modifications leads to memory leak, everything runs well except #139 by designed.
Actually there is no stream mode but only block mode in Chimera.
I know. I mean i removed stream mode and vector mode from hyperscan because chimera only support block mode. Totally about 7500 lines including test case, this make the library 10% smaller for static linking. And ll_buf, the buffer for caseful long literal support , is very hard to modify when adding character iterator. So the best way is to let it go. Everything went smoothly until #139, if chimera team can solve this problem, it would be great.
@xiangwang1 I had the same problem with hyperscan 5.2.0 & pcre 8.43 (build from source and enable jit ) 。 My program behaves fine when using chimera, but when it exits valgrind reports some memory leaks。 Details: ==99601== 528 bytes in 2 blocks are indirectly lost in loss record 3 of 3 ==99601== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==99601== by 0x2A3EB1: fast_forward_first_n_chars (in ...................) ==99601== by 0x175D189: _pcre_jit_compile (in ...................) ==99601== by 0x175EFA1: pcre_study (in ...................) ==99601== by 0xE5B606: buildPcre (ch_compile.cpp:349) ==99601== by 0xE5B606: PatternData (ch_compile.cpp:285) ==99601== by 0xE5B606: make_unique<(anonymous namespace)::PatternData, char const&, unsigned int&, unsigned int&, unsigned int&, unsigned int&, long unsigned int&, long unsigned int&, const hs_platform_info&> (make_unique.hpp:74) ==99601== by 0xE5B606: ch::ch_compile_multi_int(char const const, unsigned int const, unsigned int const, unsigned int, unsigned int, unsigned long, unsigned long, hs_platform_info const*, ch_database**) (ch_compile.cpp:499) ==99601== by 0xE5C93C: ch_compile (ch_compile.cpp:707)
@xiangwang1 I had the same problem with hyperscan 5.2.0 & pcre 8.43 (build from source and enable jit ) 。 My program behaves fine when using chimera, but when it exits valgrind reports some memory leaks。 Details: ==99601== 528 bytes in 2 blocks are indirectly lost in loss record 3 of 3 ==99601== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==99601== by 0x2A3EB1: fast_forward_first_n_chars (in ...................) ==99601== by 0x175D189: _pcre_jit_compile (in ...................) ==99601== by 0x175EFA1: pcre_study (in ...................) ==99601== by 0xE5B606: buildPcre (ch_compile.cpp:349) ==99601== by 0xE5B606: PatternData (ch_compile.cpp:285) ==99601== by 0xE5B606: make_unique<(anonymous namespace)::PatternData, char const&, unsigned int&, unsigned int&, unsigned int&, unsigned int&, long unsigned int&, long unsigned int&, const hs_platform_info&> (make_unique.hpp:74) ==99601== by 0xE5B606: ch::ch_compile_multi_int(char const const, unsigned int const, unsigned int const, unsigned int, unsigned int, unsigned long, unsigned long, hs_platform_info const*, ch_database**) (ch_compile.cpp:499) ==99601== by 0xE5C93C: ch_compile (ch_compile.cpp:707)
I found a similar issue in another project, and its method solved my memory leak problem. FYI: https://github.com/SpiderLabs/ModSecurity/issues/610
Dose chimera support pcre's JIT, looks like it is closed by default in cmake. Thanks in advance.