valgrind leak

[karlp]

I can't work it out, but simply opening and closing the mosquitto libs causes valgrind errors.

C code, no leaks in valgrind.

int main(int argc, char** argv) {
        mosquitto_lib_init();
        printf("ok, done, now exiting\n");
        mosquitto_lib_cleanup();
        return 0;
}

Lua code, leaks 1 32byte block in valgrind

#!/usr/bin/lua
local mosq = require("mosquitto")
mosq.init()
print("Exiting thanks")
mosq.cleanup()

Valgrind report...

==5387== 64 (32 direct, 32 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 8
==5387==    at 0x4A06409: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==5387==    by 0x32EAC68E62: ???
==5387==    by 0x32EACDF71E: ???
==5387==    by 0x3F072477F9: ???
==5387==    by 0x3F07249768: ???
==5387==    by 0x3F0724F028: ???
==5387==    by 0x3F01A06414: ???
==5387==    by 0x3F01A037D2: ???
==5387==    by 0x5017C4A: ???
==5387==    by 0x32FB40C027: ??? (in /usr/lib64/liblua-5.1.so)
==5387==    by 0x32FB41674E: ??? (in /usr/lib64/liblua-5.1.so)
==5387==    by 0x32FB40C43C: ??? (in /usr/lib64/liblua-5.1.so)
==5387== 
==5387== LEAK SUMMARY:
==5387==    definitely lost: 32 bytes in 1 blocks
==5387==    indirectly lost: 32 bytes in 1 blocks
==5387==      possibly lost: 0 bytes in 0 blocks
==5387==    still reachable: 3,541 bytes in 11 blocks
==5387==         suppressed: 0 bytes in 0 blocks

Any ideas? I know it's not going to ever really matter, but it's nice to make them all disappear.

[ncopa]

What does valgind say if you comment out the lua-mosquitto stuff?

#!/usr/bin/lua
--local mosq = require("mosquitto")
--mosq.init()
print("Exiting thanks")
--mosq.cleanup()

I just want make sure that the leak is in in lua-mosquitto.

[ncopa]

Please check also that it is not libmosquitto that causes the leak. Try this tiny C app in valgrind: (build with: gcc -lmosquitto test.c or similar)

#include <mosquitto.h>
int main(void)
{
        mosquitto_lib_init();
        mosquitto_lib_cleanup();
}

[karlp]

.... did you read my first mail? That's the same code I tested before filing the bug :smile:

It's also not the require line by itself, that's also clean. Note that I'm using the flukso code, not the ncopa code with the tweaked lua init stuff.

Also, clean lua code is valgrind clean, and lua with uloop (libubox) is also valgrind clean.

[ncopa]

sorry for not reading the issue clear enough.

So you are saying that valgrind does not report anything with a single mosq = require("mosquitto")?

the only thing mosq.init() does is calling mosquitto_lib_init() and the only thing mosq.cleanup() does is calling mosquitto_lib_cleanup().

There are no mallocs in the code either.

[karlp]

Yeah, I know, that's why I filed it here, to see if anyone had any ideas. I couldn't work it out. :person_frowning:

I've put a few examples of plain lua, lua-mosquitto, libubox+lua, plain libmosquitto here: https://github.com/karlp/valgrind-testing-lua-mosquitto

It must be something in how the registration is done by the lua runtime, but I can't see anything obvious in the difference. For reference, libubox's lua binding is here: http://git.openwrt.org/?p=project/libubox.git;a=blob;f=lua/uloop.c;h=2a0a516b5e2be149d6713d5a304ae5a14709a73c;hb=HEAD

[ncopa]

I think I figured it out. It is a problem in either libdl/dlclose() or libmosquitto or a combination of those two. It can be reproduced without Lua.

libtestmos.c:

/* gcc -fPIC -shared -o libtestmos.so libtestmos.c -lmosquitto */
#include <stdio.h>
int testinit(void)
{
        mosquitto_lib_init();
        printf("initialized libmosquitto\n");
        mosquitto_lib_cleanup();
        printf("cleaned up libmosquitto\n");
        return 0;
}

testmos.c:

/* gcc -o testmos testmos.c -ldl */
#include <dlfcn.h>
#include <stdio.h>

int dlerr(const char *s)
{
    fprintf("%s: %s\n", s, dlerror());
    return 1;
}

int main(void)
{
    int (*testinit)(void);
    void *h = dlopen("./libtestmos.so", RTLD_NOW);
    if (!h)
        return dlerr("dlopen");

    testinit = dlsym(h, "testinit");
    if (!testinit)
        return dlerr("dlsym");
    testinit();
    dlclose(h);
    return 0;
}

What is interesting is that if you don't do the dlclose(), it will not leak.

[karlp]

It's from a malloc in liblua, I thought it might be from the lua_replace(L, LUA_ENVIRONINDEX) call, not having anything to unwind there later? :shrugs:

I've got lots of libmosquitto code that has no valgrind problems now or earlier, so I'm not convinced it's libmosquitto at least. And still, libubox is also opened as a C library, and isn't leaking.

I've just added a luasocket example, which also has no leaks in valgrind, and also opens a C library behind the scenes. https://github.com/karlp/valgrind-testing-lua-mosquitto/blob/master/open-close-socket.lua

[ncopa]

It's from a malloc in liblua,

Not necessarily. it is triggered via liblua but there are many ??? lines in the stacktrace

I thought it might be from the lua_replace(L, LUA_ENVIRONINDEX) call, not having anything to unwind there later? :shrugs:

no, I tested comment out those. Infact, I tested comment out many things. The only thing that made the difference was commenting out mosquitto_lib_init. That is why I bothered write the testmos testcase.

I don't think problem is in mosquitto or lua or lua-mosquitto. I think problem is in openssl and dlclose from libc. I bet you will see this symtom with any lua module that is linked to libssl.

And I suspect that you will never lose more than those 32 bytes, regarless how many times you load and unload lua-mosquitto. You still have reachable blocks so it is not necessarily a leak.

[karlp]

Works for me, it's definitely a one off, so yes, in general the runtime environment will reclaim memory, it's just nice to get clean valgrind runs, so you can only focus on what you are working on :)