search

ntop / nDPI

Open Source Deep Packet Inspection Software Toolkit
http://www.ntop.org
GNU Lesser General Public License v3.0
3.76k stars 888 forks source link

Memory leak in ahocorasick #2258

Closed IvanNardi closed 2 weeks ago

IvanNardi commented 7 months ago

Oss-fuzz keeps reporting a memory leak in ahocorasick code, via fuzz_filecfg_protocols fuzzer. Some examples (these reports should be public): https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64150&q=ndpi&can=1&sort=-id https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62269&q=ndpi&can=1&sort=-id https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61934&q=ndpi&can=1&sort=-id

It seems that the leak is about inserting duplicated patterns.

The stack reported is something like:

            #6 0x67f7c9 in ac_automata_add [ndpi/src/lib/third_party/src/ahocorasick.c:255](https://github.com/ntop/nDPI/blob/7b2bbb2309264766697507365231fd6ee5717e31/src/lib/third_party/src/ahocorasick.c#L255):19
        #7 0x58df28 in ndpi_add_host_risk_mask [ndpi/src/lib/ndpi_main.c:4262](https://github.com/ntop/nDPI/blob/7b2bbb2309264766697507365231fd6ee5717e31/src/lib/ndpi_main.c#L4262):8
        #8 0x592605 in ndpi_handle_rule [ndpi/src/lib/ndpi_main.c:4344](https://github.com/ntop/nDPI/blob/7b2bbb2309264766697507365231fd6ee5717e31/src/lib/ndpi_main.c#L4344):11
        #9 0x592605 in load_protocols_file_fd [ndpi/src/lib/ndpi_main.c:5036](https://github.com/ntop/nDPI/blob/7b2bbb2309264766697507365231fd6ee5717e31/src/lib/ndpi_main.c#L5036):8
mmanoj commented 1 month ago

@IvanNardi

Is this issue still valid?

IvanNardi commented 1 month ago

Yes, according to oss-fuzz. If you are interested I can provide an up-to-date artifact to reproduce the error

mmanoj commented 1 month ago

@IvanNardi

Thanks for the feedback, lets workout this issue as well together as we done same way of FPC feature.This way we can achieve more good results.

mmanoj commented 1 month ago

@IvanNardi Can we work on this?

IvanNardi commented 1 month ago

I don't have any expertise on ahocorasick code, so I can't guide/suggest you how to fix it; I can gladly review your changes, tough. To reproduce the error, with latest code:

ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ ./autogen.sh --enable-debug-build --enable-fuzztargets --with-sanitizer && make -s -j
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ ./fuzz/fuzz_filecfg_protocols ~/Downloads/clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424 
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3967398786
INFO: Loaded 1 modules   (17 inline 8-bit counters): 17 [0x55849c114e7c, 0x55849c114e8d), 
INFO: Loaded 1 PC tables (17 PCs): 17 [0x55849c114e90,0x55849c114fa0), 
./fuzz/fuzz_filecfg_protocols: Running 1 inputs 1 time(s) each.
Running: /home/ivan/Downloads/clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424

=================================================================
==43734==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 40 byte(s) in 1 object(s) allocated from:
    #0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55849b88a9dd in ndpi_calloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:67:13
    #3 0x55849bb7181a in node_create /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:802:25
    #4 0x55849bb7307a in node_create_next /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1007:10
    #5 0x55849bb72223 in ac_automata_add /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:255:19
    #6 0x55849b7d5f79 in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4573:8
    #7 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
    #8 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
    #9 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
    #10 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #11 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #12 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #13 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #14 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16

Indirect leak of 328 byte(s) in 1 object(s) allocated from:
    #0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55849b88a9dd in ndpi_calloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:67:13
    #3 0x55849bb84148 in node_resize_mp /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1027:13
    #4 0x55849bb733db in node_register_matchstr /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:1055:30
    #5 0x55849bb72a2d in ac_automata_add /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ahocorasick.c:280:6
    #6 0x55849b7d5f79 in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4573:8
    #7 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
    #8 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
    #9 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
    #10 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #11 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #12 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #13 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #14 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16

Indirect leak of 257 byte(s) in 1 object(s) allocated from:
    #0 0x55849b7651bf in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x6f91bf) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #1 0x55849b88a986 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55849b88ac14 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #3 0x55849b7d5d5c in ndpi_add_host_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4558:14
    #4 0x55849b7dfb08 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4655:11
    #5 0x55849b7df078 in load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5446:8
    #6 0x55849b7a7c1e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:21:3
    #7 0x55849b6add16 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x641d16) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #8 0x55849b697e98 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x62be98) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #9 0x55849b69d96a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x63196a) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #10 0x55849b6c72d2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x65b2d2) (BuildId: 89b530cdc1074590301d775e4c28576b2bea37d2)
    #11 0x7fcae91b9082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16

SUMMARY: AddressSanitizer: 625 byte(s) leaked in 3 allocation(s).

INFO: a leak has been found in the initial corpus.

INFO: to ignore leaks on libFuzzer side use -detect_leaks=0

clusterfuzz-testcase-minimized-fuzz_filecfg_protocols-4513089035239424.zip

mmanoj commented 1 month ago

@IvanNardi

Thanks for the details and sorry for the late reply.I have some experience with same type of memory issue fixing.Let me analysis the code and update my finding within early next week.So then we can plan together the fixing approach.