Closed IvanNardi closed 2 weeks ago
@IvanNardi
Is this issue still valid?
Yes, according to oss-fuzz. If you are interested I can provide an up-to-date artifact to reproduce the error
@IvanNardi
Thanks for the feedback, lets workout this issue as well together as we done same way of FPC feature.This way we can achieve more good results.
@IvanNardi Can we work on this?
I don't have any expertise on ahocorasick code, so I can't guide/suggest you how to fix it; I can gladly review your changes, tough. To reproduce the error, with latest code:
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ ./autogen.sh --enable-debug-build --enable-fuzztargets --with-sanitizer && make -s -j
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ ./fuzz/fuzz_filecfg_protocols ~/Downloads/clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3967398786
INFO: Loaded 1 modules (17 inline 8-bit counters): 17 [0x55849c114e7c, 0x55849c114e8d),
INFO: Loaded 1 PC tables (17 PCs): 17 [0x55849c114e90,0x55849c114fa0),
./fuzz/fuzz_filecfg_protocols: Running 1 inputs 1 time(s) each.
Running: /home/ivan/Downloads/clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424
=================================================================
==43734==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 40 byte(s) in 1 object(s) allocated from:
#0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
#2 0x55849b88a9dd in ndpi_calloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:67:13
#3 0x55849bb7181a in node_create /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:802:25
#4 0x55849bb7307a in node_create_next /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1007:10
#5 0x55849bb72223 in ac_automata_add /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:255:19
#6 0x55849b7d5f79 in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4573:8
#7 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
#8 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
#9 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
#10 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#11 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#12 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#13 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#14 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16
Indirect leak of 328 byte(s) in 1 object(s) allocated from:
#0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
#2 0x55849b88a9dd in ndpi_calloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:67:13
#3 0x55849bb84148 in node_resize_mp /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1027:13
#4 0x55849bb733db in node_register_matchstr /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1055:30
#5 0x55849bb72a2d in ac_automata_add /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:280:6
#6 0x55849b7d5f79 in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4573:8
#7 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
#8 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
#9 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
#10 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#11 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#12 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#13 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#14 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16
Indirect leak of 257 byte(s) in 1 object(s) allocated from:
#0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
#2 0x55849b88ac14 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
#3 0x55849b7d5d5c in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4558:14
#4 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
#5 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
#6 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
#7 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#8 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#9 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#10 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
#11 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16
SUMMARY: AddressSanitizer: 625 byte(s) leaked in 3 allocation(s).
INFO: a leak has been found in the initial corpus.
INFO: to ignore leaks on libFuzzer side use -detect_leaks=0
clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424.zip
@IvanNardi
Thanks for the details and sorry for the late reply.I have some experience with same type of memory issue fixing.Let me analysis the code and update my finding within early next week.So then we can plan together the fixing approach.
Oss-fuzz keeps reporting a memory leak in ahocorasick code, via
fuzz_filecfg_protocols
fuzzer. Some examples (these reports should be public): https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64150&q=ndpi&can=1&sort=-id https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62269&q=ndpi&can=1&sort=-id https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61934&q=ndpi&can=1&sort=-idIt seems that the leak is about inserting duplicated patterns.
The stack reported is something like: