-server tests sometimes fail due to a race-cond memory leak with Malamute / *ZMQ

[aquette]

This one is still to hunt, and not bold enough to be fixed quickly.

1) step to reproduce

while make memcheck ; do if $? -ne 0; then break; else sleep 1 ; fi ; done

2) memcheck trace

==5776== ==5776== HEAP SUMMARY: ==5776== in use at exit: 306 bytes in 5 blocks ==5776== total heap usage: 4,263 allocs, 4,258 frees, 1,382,428 bytes allocated ==5776== ==5776== 24 bytes in 1 blocks are indirectly lost in loss record 1 of 5 ==5776== at 0x4C2AD10: calloc (vg_replace_malloc.c:623) ==5776== by 0x54EFD91: zlist_append (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54F4B60: zmsg_append (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54F4C15: zmsg_recv (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52881E0: mlm_proto_recv (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x529021E: ??? (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54F3DBC: zloop_start (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52909AE: mlm_client (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54DCAB2: ??? (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x65C8063: start_thread (pthread_create.c:309) ==5776== by 0x62FD62C: clone (clone.S:111) ==5776== ==5776== 48 bytes in 1 blocks are indirectly lost in loss record 2 of 5 ==5776== at 0x4C2AD10: calloc (vg_replace_malloc.c:623) ==5776== by 0x54EFB92: zlist_new (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54F4482: zmsg_new (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54F4BE6: zmsg_recv (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52881E0: mlm_proto_recv (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x529021E: ??? (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54F3DBC: zloop_start (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52909AE: mlm_client (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54DCAB2: ??? (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x65C8063: start_thread (pthread_create.c:309) ==5776== by 0x62FD62C: clone (clone.S:111) ==5776== ==5776== 96 bytes in 1 blocks are indirectly lost in loss record 3 of 5 ==5776== at 0x4C2AD10: calloc (vg_replace_malloc.c:623) ==5776== by 0x54E992A: zframe_new (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54E99EE: zframe_recv (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54F4BFF: zmsg_recv (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52881E0: mlm_proto_recv (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x529021E: ??? (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54F3DBC: zloop_start (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52909AE: mlm_client (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54DCAB2: ??? (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x65C8063: start_thread (pthread_create.c:309) ==5776== by 0x62FD62C: clone (clone.S:111) ==5776== ==5776== 106 bytes in 1 blocks are indirectly lost in loss record 4 of 5 ==5776== at 0x4C28C20: malloc (vg_replace_malloc.c:296) ==5776== by 0x57970C9: ??? (in /usr/lib/libzmq.so.5.1.2) ==5776== by 0x54E994B: zframe_new (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54F5055: zmsg_addmem (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54F5D30: zmsg_dup (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52869FB: mlm_msg_set_proto (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x528B8F0: ??? (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x528CF39: ??? (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x528DD0F: ??? (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54F3DBC: zloop_start (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x528C59B: mlm_server (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54DCAB2: ??? (in /usr/lib/libczmq.so.4.0.2) ==5776== ==5776== 306 (32 direct, 274 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5 ==5776== at 0x4C2AD10: calloc (vg_replace_malloc.c:623) ==5776== by 0x54F446F: zmsg_new (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x54F4BE6: zmsg_recv (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52881E0: mlm_proto_recv (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x529021E: ??? (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54F3DBC: zloop_start (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x52909AE: mlm_client (in /usr/lib/libmlm.so.1.0.1) ==5776== by 0x54DCAB2: ??? (in /usr/lib/libczmq.so.4.0.2) ==5776== by 0x65C8063: start_thread (pthread_create.c:309) ==5776== by 0x62FD62C: clone (clone.S:111) ==5776== ==5776== LEAK SUMMARY: ==5776== definitely lost: 32 bytes in 1 blocks ==5776== indirectly lost: 274 bytes in 4 blocks ==5776== possibly lost: 0 bytes in 0 blocks ==5776== still reachable: 0 bytes in 0 blocks ==5776== suppressed: 0 bytes in 0 blocks ==5776== ==5776== For counts of detected and suppressed errors, rerun with: -v ==5776== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) Makefile:1829: recipe for target 'memcheck' failed make: *** [memcheck] Error 1

[jana-rapava]

Fixed by commit ID b88816e511b0a001abd03ec80914a43c0feb91af