CESNET / ipfixcol

IPFIXcol is an implementation of an IPFIX (RFC 7011) collector
Other
64 stars 37 forks source link

Memory leak in UDP input plugin #67

Closed ghost closed 9 years ago

ghost commented 9 years ago

From time to time, I get the following report from Valgrind:

==26923== 10 bytes in 2 blocks are definitely lost in loss record 12 of 153
==26923==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26923==    by 0x69E04B0: input_init (udp_input.c:159)
==26923==    by 0x408F14: config_add_input (configurator.c:355)
==26923==    by 0x409F4A: config_process_new_startup (configurator.c:681)
==26923==    by 0x40A2A7: config_reconf (configurator.c:1133)
==26923==    by 0x406EDF: main (ipfixcol.c:340)

There's no clear way to reproduce this issue (it just happens from time to time), although I cannot pinpoint the source of the problem, yet. What I can say though is that the issue is not solved by checking whether the variables in udp_input.c:181,183,185,187 are already set before (over)writing the pointer.

thorgrin commented 9 years ago

I do not see how the leak is possible at the moment. The only case when this leak occurs is when I'm terminating the collector with second signal - in case it won't die the first time. Can you provide more information about when this happens?

ghost commented 9 years ago

Same for me. I spent quite some time trying to debug this problem, but without any luck so far. I'm certainly not running terminating the collector with a second signal. Some observations:

<collectingProcess>
    <name>UDP collector</name>
    <udpCollector>
        <name>Listening port 3055</name>
        <localPort>3055</localPort>
        <templateLifeTime>1800</templateLifeTime>
        <optionsTemplateLifeTime>1800</optionsTemplateLifeTime>
        <!--## Local address to listen on. If empty, bind to all interfaces -->
        <localIPAddress></localIPAddress>
    </udpCollector>
    <exportingProcess>Integrated export</exportingProcess>
    <statisticsFile>ipfixcol_stat.log</statisticsFile>
</collectingProcess>
/* Input is pointer to configurator structure, don't free it */
// free(plugin->input);

By the way, as a precaution, I added some extra checks to udp_input.c as part of #72.

thorgrin commented 9 years ago

I've tried with ipfix storage plugin and data is send by ipfixsend tool. There are no leaks at all. Maybe the valgrind does not handle dynamically loading plugins correctly?

ghost commented 9 years ago

Just checking: which version of Valgrind are you using? I'm using valgrind-3.10.0.SVN.

thorgrin commented 9 years ago

I'm using valgrind-3.8.1 which comes with OpenSuse 12.3. Can you try the same test on other versions?

ghost commented 9 years ago

Since valgrind-3.8.1 is rather old, I tested it again on Valgrind's latest-and-greatest release, valgrind-3.10.1.SVN, and the leak is still reported.

thorgrin commented 9 years ago

I've just tested it with valgrind-3.11.0.SVN and still I'm not able to reproduce the issue. I'm gonna close it until we can reproduce it more reliably.