Leaks- Threads

[jorgeriesco]

Hola, me sale este leak summary cuando uso pthreads. Revisé harto y no veo algo evidente que me generé leaks. No se si quizás los threads piden algún tipo de memoria que hay que liberar específicamente, aparte de liberar los threads mismos. Sería de gran ayuda si alguien sabe, gracias!

LEAK SUMMARY: ==10412== definitely lost: 0 bytes in 0 blocks ==10412== indirectly lost: 0 bytes in 0 blocks ==10412== possibly lost: 0 bytes in 0 blocks ==10412== still reachable: 1,638 bytes in 4 blocks ==10412== suppressed: 0 bytes in 0 blocks

[IchottMano]

Hola!

Estas haciendole join a todos tus threads? Puedes ver este link donde explican un poco que la información de los threads se mantiene en memoria por si en algun momento quieres usarla, lo cual se marca como memoria no liberada still reachable. Debes liberar esta memoria para obtener el puntaje.

Si no era esto, siempre puedes usar la opción --leak-check=full de valgrind para ver dónde pesite la memoria que aún no has liberado.

Saludos!

[jorgeriesco]

Buena gracias, pero no logré solucionarlo.

Por lo que vi, al parecer estoy haciendo join a todos. Vi el link y corrí un comando que sale ahí, además e usar --leak-check=full, pero no me dió algo muy concluyente.

Además pasa algo raro. Algunas veces, cuando corro el programa, no hay leaks y otras si (la mayoría si). No se si tendrá que ver con el manejo de la concurrencia o la sincronización o algo así.

Puse: valgrind --leak-check=full [atributos] y tambien puse: valgrind -v --leak-check=full --show-reachable=yes [atributos]

Pero no vi algo como que me de una linea exacta donde mirar.

[IchottMano]

Podrias postear un pantallazo completo de tu output de valgrind? Saludos!

[jorgeriesco]

Este es corriendo: valgrind --leak-check=full ./mapreduce text.txt output.csv threads 1

==14906== Memcheck, a memory error detector ==14906== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==14906== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info ==14906== Command: ./mapreduce text.txt dede threads 1 ==14906== ==14906== ==14906== HEAP SUMMARY: ==14906== in use at exit: 1,638 bytes in 4 blocks ==14906== total heap usage: 2,315 allocs, 2,311 frees, 88,833,174 bytes allocated ==14906== ==14906== LEAK SUMMARY: ==14906== definitely lost: 0 bytes in 0 blocks ==14906== indirectly lost: 0 bytes in 0 blocks ==14906== possibly lost: 0 bytes in 0 blocks ==14906== still reachable: 1,638 bytes in 4 blocks ==14906== suppressed: 0 bytes in 0 blocks ==14906== Reachable blocks (those to which a pointer was found) are not shown. ==14906== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==14906== ==14906== For counts of detected and suppressed errors, rerun with: -v ==14906== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Que opinas? Gracias!

[IchottMano]

Intenta usando también la opción --show-reachable=yes a ver si te tira donde está la memoria no liberada.

[jorgeriesco]

Me tira esta gigantesca cuestion al hacer esto: valgrind -v --leak-check=full --show-reachable=yes ./mapreduce text.txt dede threads 2

Leí todo y no vi algo q me diga una linea exacta de mi código o que pueda deducir algo. Igual se me pudo pasar algo o no se interpretar todas las cosas quizás.

==13430== Memcheck, a memory error detector ==13430== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==13430== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info ==13430== Command: ./mapreduce text.txt dede threads 2 ==13430== --13430-- Valgrind options: --13430-- -v --13430-- --leak-check=full --13430-- --show-reachable=yes --13430-- Contents of /proc/version: --13430-- Linux version 4.10.0-42-generic (buildd@lgw01-amd64-012) (gcc version 6.3.0 20170406 (Ubuntu 6.3.0-12ubuntu2) ) #46-Ubuntu SMP Mon Dec 4 14:38:01 UTC 2017 --13430-- --13430-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-lzcnt-rdtscp-sse3-avx-avx2-bmi --13430-- Page sizes: currently 4096, max supported 4096 --13430-- Valgrind library directory: /usr/lib/valgrind --13430-- Reading syms from /home/jorge/Escritorio/T2/mapreduce --13430-- Reading syms from /lib/x86_64-linux-gnu/ld-2.26.so --13430-- Considering /lib/x86_64-linux-gnu/ld-2.26.so .. --13430-- .. CRC mismatch (computed 89c8df08 wanted 47b839f9) --13430-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/ld-2.26.so .. --13430-- .. CRC is valid --13430-- Reading syms from /usr/lib/valgrind/memcheck-amd64-linux --13430-- Considering /usr/lib/valgrind/memcheck-amd64-linux .. --13430-- .. CRC mismatch (computed c2dc3c92 wanted ad3f4971) --13430-- object doesn't have a symbol table --13430-- object doesn't have a dynamic symbol table --13430-- Scheduler: using generic scheduler lock implementation. --13430-- Reading suppressions file: /usr/lib/valgrind/default.supp ==13430== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-13430-by-jorge-on-??? ==13430== embedded gdbserver: writing to /tmp/vgdb-pipe-to-vgdb-from-13430-by-jorge-on-??? ==13430== embedded gdbserver: shared mem /tmp/vgdb-pipe-shared-mem-vgdb-13430-by-jorge-on-??? ==13430== ==13430== TO CONTROL THIS PROCESS USING vgdb (which you probably ==13430== don't want to do, unless you know exactly what you're doing, ==13430== or are doing some strange experiment): ==13430== /usr/lib/valgrind/../../bin/vgdb --pid=13430 ...command... ==13430== ==13430== TO DEBUG THIS PROCESS USING GDB: start GDB like this ==13430== /path/to/gdb ./mapreduce ==13430== and then give GDB the following command ==13430== target remote | /usr/lib/valgrind/../../bin/vgdb --pid=13430 ==13430== --pid is optional if only one valgrind process is running ==13430== --13430-- REDIR: 0x401f860 (ld-linux-x86-64.so.2:strlen) redirected to 0x380a6091 (???) --13430-- REDIR: 0x401f640 (ld-linux-x86-64.so.2:index) redirected to 0x380a60ab (???) --13430-- Reading syms from /usr/lib/valgrind/vgpreload_core-amd64-linux.so --13430-- Considering /usr/lib/valgrind/vgpreload_core-amd64-linux.so .. --13430-- .. CRC mismatch (computed da1984fa wanted 7644421c) --13430-- object doesn't have a symbol table --13430-- Reading syms from /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so --13430-- Considering /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so .. --13430-- .. CRC mismatch (computed bbbdc1f5 wanted 3c7fb4f9) --13430-- object doesn't have a symbol table ==13430== WARNING: new redirection conflicts with existing -- ignoring it --13430-- old: 0x0401f860 (strlen ) R-> (0000.0) 0x380a6091 ??? --13430-- new: 0x0401f860 (strlen ) R-> (2007.0) 0x04c32de0 strlen --13430-- REDIR: 0x401d8d0 (ld-linux-x86-64.so.2:strcmp) redirected to 0x4c33ee0 (strcmp) --13430-- REDIR: 0x401fda0 (ld-linux-x86-64.so.2:mempcpy) redirected to 0x4c372b0 (mempcpy) --13430-- Reading syms from /lib/x86_64-linux-gnu/libpthread-2.26.so --13430-- Considering /usr/lib/debug/.build-id/b9/5ecb6ece407081730183e686ff6cf2e0bce4f6.debug .. --13430-- .. build-id is valid --13430-- Reading syms from /lib/x86_64-linux-gnu/libc-2.26.so --13430-- Considering /lib/x86_64-linux-gnu/libc-2.26.so .. --13430-- .. CRC mismatch (computed b6c34c34 wanted f79829d1) --13430-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.26.so .. --13430-- .. CRC is valid --13430-- REDIR: 0x50eff50 (libc.so.6:memmove) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50ef010 (libc.so.6:strncpy) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f0230 (libc.so.6:strcasecmp) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50eea60 (libc.so.6:strcat) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50ef040 (libc.so.6:rindex) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f1bb0 (libc.so.6:rawmemchr) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f00c0 (libc.so.6:mempcpy) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50efee0 (libc.so.6:bcmp) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50eefd0 (libc.so.6:strncmp) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50eead0 (libc.so.6:strcmp) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f0020 (libc.so.6:memset) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x510df10 (libc.so.6:wcschr) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50eef70 (libc.so.6:strnlen) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50eeb40 (libc.so.6:strcspn) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f0280 (libc.so.6:strncasecmp) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50eeb10 (libc.so.6:strcpy) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f03c0 (libc.so.6:memcpy@@GLIBC_2.14) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50ef070 (libc.so.6:strpbrk) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50eea90 (libc.so.6:index) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50eef40 (libc.so.6:strlen) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50fa510 (libc.so.6:memrchr) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f02d0 (libc.so.6:strcasecmp_l) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50efeb0 (libc.so.6:memchr) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x510ecc0 (libc.so.6:wcslen) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50ef350 (libc.so.6:strspn) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f0200 (libc.so.6:stpncpy) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f01d0 (libc.so.6:stpcpy) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f1be0 (libc.so.6:strchrnul) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x50f0320 (libc.so.6:strncasecmp_l) redirected to 0x4a2a740 (_vgnU_ifunc_wrapper) --13430-- REDIR: 0x51d96f0 (libc.so.6:strrchr_avx2) redirected to 0x4c32770 (rindex) --13430-- REDIR: 0x50e9eb0 (libc.so.6:malloc) redirected to 0x4c2fac0 (malloc) --13430-- REDIR: 0x50ef850 (libc.so.6:__GI_strstr) redirected to 0x4c37510 (strstr_sse2) --13430-- REDIR: 0x50ea3e0 (libc.so.6:free) redirected to 0x4c30cf0 (free) --13430-- REDIR: 0x51b50b0 (libc.so.6:strcmp_ssse3) redirected to 0x4c33da0 (strcmp) --13430-- REDIR: 0x51c6ca0 (libc.so.6:__strcpy_ssse3) redirected to 0x4c32e00 (strcpy) --13430-- REDIR: 0x50eb540 (libc.so.6:calloc) redirected to 0x4c31a90 (calloc) --13430-- REDIR: 0x51d9e00 (libc.so.6:memcpy_avx_unaligned_erms) redirected to 0x4c36540 (memmove) --13430-- Reading syms from /lib/x86_64-linux-gnu/libgcc_s.so.1 --13430-- object doesn't have a symbol table --13430-- REDIR: 0x51d98c0 (libc.so.6:strlen_avx2) redirected to 0x4c32d20 (strlen) --13430-- REDIR: 0x51d9500 (libc.so.6:__strchrnul_avx2) redirected to 0x4c36de0 (strchrnul) --13430-- REDIR: 0x51d9de0 (libc.so.6:mempcpy_avx_unaligned_erms) redirected to 0x4c36ef0 (mempcpy) ==13430== ==13430== HEAP SUMMARY: ==13430== in use at exit: 1,638 bytes in 4 blocks ==13430== total heap usage: 2,315 allocs, 2,311 frees, 88,833,174 bytes allocated ==13430== ==13430== Searching for pointers to 4 not-freed blocks ==13430== Checked 97,032 bytes ==13430== ==13430== 36 bytes in 1 blocks are still reachable in loss record 1 of 4 ==13430== at 0x4C2FB2F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==13430== by 0x401D899: strdup (strdup.c:42) ==13430== by 0x4018C7E: _dl_load_cache_lookup (dl-cache.c:314) ==13430== by 0x4009330: _dl_map_object (dl-load.c:2324) ==13430== by 0x401593A: dl_open_worker (dl-open.c:237) ==13430== by 0x51B3DB3: _dl_catch_error (dl-error-skeleton.c:198) ==13430== by 0x4015148: _dl_open (dl-open.c:660) ==13430== by 0x51B331C: do_dlopen (dl-libc.c:87) ==13430== by 0x51B3DB3: _dl_catch_error (dl-error-skeleton.c:198) ==13430== by 0x51B33DB: dlerror_run (dl-libc.c:46) ==13430== by 0x51B33DB: libc_dlopen_mode (dl-libc.c:163) ==13430== by 0x4E4F73A: pthread_cancel_init (unwind-forcedunwind.c:52) ==13430== by 0x4E4F923: _Unwind_ForcedUnwind (unwind-forcedunwind.c:126) ==13430== ==13430== 36 bytes in 1 blocks are still reachable in loss record 2 of 4 ==13430== at 0x4C2FB2F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==13430== by 0x400C383: _dl_new_object (dl-object.c:165) ==13430== by 0x40068C4: _dl_map_object_from_fd (dl-load.c:1028) ==13430== by 0x4008F84: _dl_map_object (dl-load.c:2458) ==13430== by 0x401593A: dl_open_worker (dl-open.c:237) ==13430== by 0x51B3DB3: _dl_catch_error (dl-error-skeleton.c:198) ==13430== by 0x4015148: _dl_open (dl-open.c:660) ==13430== by 0x51B331C: do_dlopen (dl-libc.c:87) ==13430== by 0x51B3DB3: _dl_catch_error (dl-error-skeleton.c:198) ==13430== by 0x51B33DB: dlerror_run (dl-libc.c:46) ==13430== by 0x51B33DB: libc_dlopen_mode (dl-libc.c:163) ==13430== by 0x4E4F73A: pthread_cancel_init (unwind-forcedunwind.c:52) ==13430== by 0x4E4F923: _Unwind_ForcedUnwind (unwind-forcedunwind.c:126) ==13430== ==13430== 384 bytes in 1 blocks are still reachable in loss record 3 of 4 ==13430== at 0x4C31B45: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==13430== by 0x401265D: _dl_check_map_versions (dl-version.c:293) ==13430== by 0x4015C2E: dl_open_worker (dl-open.c:286) ==13430== by 0x51B3DB3: _dl_catch_error (dl-error-skeleton.c:198) ==13430== by 0x4015148: _dl_open (dl-open.c:660) ==13430== by 0x51B331C: do_dlopen (dl-libc.c:87) ==13430== by 0x51B3DB3: _dl_catch_error (dl-error-skeleton.c:198) ==13430== by 0x51B33DB: dlerror_run (dl-libc.c:46) ==13430== by 0x51B33DB: libc_dlopen_mode (dl-libc.c:163) ==13430== by 0x4E4F73A: pthread_cancel_init (unwind-forcedunwind.c:52) ==13430== by 0x4E4F923: _Unwind_ForcedUnwind (unwind-forcedunwind.c:126) ==13430== by 0x4E4D8FF: pthread_unwind (unwind.c:121) ==13430== by 0x4E44A24: __do_cancel (pthreadP.h:297) ==13430== by 0x4E44A24: pthread_exit (pthread_exit.c:28) ==13430== ==13430== 1,182 bytes in 1 blocks are still reachable in loss record 4 of 4 ==13430== at 0x4C31B45: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==13430== by 0x400C0C5: _dl_new_object (dl-object.c:75) ==13430== by 0x40068C4: _dl_map_object_from_fd (dl-load.c:1028) ==13430== by 0x4008F84: _dl_map_object (dl-load.c:2458) ==13430== by 0x401593A: dl_open_worker (dl-open.c:237) ==13430== by 0x51B3DB3: _dl_catch_error (dl-error-skeleton.c:198) ==13430== by 0x4015148: _dl_open (dl-open.c:660) ==13430== by 0x51B331C: do_dlopen (dl-libc.c:87) ==13430== by 0x51B3DB3: _dl_catch_error (dl-error-skeleton.c:198) ==13430== by 0x51B33DB: dlerror_run (dl-libc.c:46) ==13430== by 0x51B33DB: __libc_dlopen_mode (dl-libc.c:163) ==13430== by 0x4E4F73A: pthread_cancel_init (unwind-forcedunwind.c:52) ==13430== by 0x4E4F923: _Unwind_ForcedUnwind (unwind-forcedunwind.c:126) ==13430== ==13430== LEAK SUMMARY: ==13430== definitely lost: 0 bytes in 0 blocks ==13430== indirectly lost: 0 bytes in 0 blocks ==13430== possibly lost: 0 bytes in 0 blocks ==13430== still reachable: 1,638 bytes in 4 blocks ==13430== suppressed: 0 bytes in 0 blocks ==13430== ==13430== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) ==13430== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

[IchottMano]

Bueno, por lo que veo aparentemente es un bug causado por la función pthread_exit, donde por debajo la librería de threads no libera toda su memoria al hacer exit del thread. Para arreglar esto, puedes terminar los threads usando return, o, si lo deseas, puedes poner en un README que los leaks tienen esta causa. Dime si esto resuelve efectivamente tu problema. Saludos!

[gregschuit]

A mi me pasa lo mismo! Lo que encontré fue esto en stack overflow. Basicamente reafirma lo que ya han dicho. Espero que no lo consideren para el puntaje de manejo de memoria.