search

vlang / v

Simple, fast, safe, compiled language for developing maintainable software. Compiles itself in <1s with zero library dependencies. Supports automatic C => V translation. https://vlang.io
MIT License
35.88k stars 2.17k forks source link

Solving the memory leaks in V #514

Closed LouCle closed 5 years ago

LouCle commented 5 years ago

@medvednikov Valgrind on c artifact (compiler.c) informs us of following memory leaks when running REPL and then immediately killing it after start.

==14786== 28 bytes in 1 blocks are still reachable in loss record 8 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10B792: string_add (compiler.c:1717)
==14786==    by 0x11ACBA: vtmp_path (compiler.c:7727)
==14786==    by 0x1384E0: init_consts (compiler.c:17663)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 32 bytes in 1 blocks are still reachable in loss record 9 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10ABAF: array__push (compiler.c:1255)
==14786==    by 0x10FBAB: os__init_os_args (compiler.c:4475)
==14786==    by 0x11AD03: main (compiler.c:7733)
==14786== 
==14786== 36 bytes in 1 blocks are still reachable in loss record 10 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10B792: string_add (compiler.c:1717)
==14786==    by 0x11ECFC: run_repl (compiler.c:8875)
==14786==    by 0x11AFD3: main (compiler.c:7803)
==14786== 
==14786== 44 bytes in 1 blocks are still reachable in loss record 11 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10A4F9: new_array_from_c_array (compiler.c:1029)
==14786==    by 0x138B36: init_consts (compiler.c:17670)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 48 bytes in 1 blocks are still reachable in loss record 12 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10A4F9: new_array_from_c_array (compiler.c:1029)
==14786==    by 0x1384AC: init_consts (compiler.c:17662)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 64 bytes in 1 blocks are still reachable in loss record 13 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10A4F9: new_array_from_c_array (compiler.c:1029)
==14786==    by 0x1389DE: init_consts (compiler.c:17667)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 144 bytes in 36 blocks are still reachable in loss record 14 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10EBF0: map_new_entry (compiler.c:4000)
==14786==    by 0x10EC92: map__set (compiler.c:4011)
==14786==    by 0x136E55: build_keys (compiler.c:17291)
==14786==    by 0x138A5A: init_consts (compiler.c:17669)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 160 bytes in 1 blocks are still reachable in loss record 15 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10A49B: new_array (compiler.c:1020)
==14786==    by 0x13852B: init_consts (compiler.c:17664)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 160 bytes in 1 blocks are still reachable in loss record 16 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10A49B: new_array (compiler.c:1020)
==14786==    by 0x138553: init_consts (compiler.c:17664)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 160 bytes in 1 blocks are still reachable in loss record 17 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10A4F9: new_array_from_c_array (compiler.c:1029)
==14786==    by 0x13875C: init_consts (compiler.c:17665)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 168 bytes in 1 blocks are still reachable in loss record 18 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10D70C: memdup (compiler.c:3088)
==14786==    by 0x138629: init_consts (compiler.c:17664)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 256 bytes in 1 blocks are still reachable in loss record 19 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10A4F9: new_array_from_c_array (compiler.c:1029)
==14786==    by 0x138924: init_consts (compiler.c:17666)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 256 bytes in 1 blocks are still reachable in loss record 20 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x110CC4: os__get_line (compiler.c:5021)
==14786==    by 0x11ED59: run_repl (compiler.c:8882)
==14786==    by 0x11AFD3: main (compiler.c:7803)
==14786== 
==14786== 1,000 bytes in 1 blocks are still reachable in loss record 21 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x13838F: init_consts (compiler.c:17658)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 1,536 bytes in 1 blocks are still reachable in loss record 22 of 23
==14786==    at 0x483AD7B: realloc (vg_replace_malloc.c:826)
==14786==    by 0x10ABD8: array__push (compiler.c:1261)
==14786==    by 0x10ED72: map__set (compiler.c:4033)
==14786==    by 0x136E55: build_keys (compiler.c:17291)
==14786==    by 0x138A5A: init_consts (compiler.c:17669)
==14786==    by 0x11ACEE: main (compiler.c:7732)
==14786== 
==14786== 2,240 bytes in 1 blocks are still reachable in loss record 23 of 23
==14786==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==14786==    by 0x10D5FC: v_malloc (compiler.c:3042)
==14786==    by 0x10A5C1: array_repeat (compiler.c:1049)
==14786==    by 0x136F00: build_token_str (compiler.c:17304)
==14786==    by 0x138A1C: init_consts (compiler.c:17668)
==14786==    by 0x11ACEE: main (compiler.c:7732)

A good chunk of them are within init_consts

Loss record 21 of 23 (1000 bytes) is due to g_str_buf being allocated 1000 bytes. This was also pointed out by the person who wrote the blogpost about V.

Loss record 20 of 23 (256 bytes) is inside run_repl() in main.v, but appears to be stem from the os.get_line function.

Loss record 23 of 23 (2240 bytes) stems from the malloc inside array_repeat in builtin array.v (and is of course variable to nr_repeats and elm_size).

Loss record 22 of 23 (1536 bytes) is the _push function also inside array.v. (and is also variable to the arguments)

The following is from running println('hello') inside REPL and then immediatelly killing the V process.

==18371== 55,680 bytes in 94 blocks are still reachable in loss record 1,326 of 1,331
==18371==    at 0x483AD7B: realloc (vg_replace_malloc.c:826)
==18371==    by 0x10ABD8: array__push (compiler.c:1261)
==18371==    by 0x11804A: Parser_fn_args (compiler.c:7089)
==18371==    by 0x115506: Parser_fn_decl (compiler.c:6409)
==18371==    by 0x11F84B: Parser_parse (compiler.c:9091)
==18371==    by 0x11B20D: V_compile (compiler.c:7860)
==18371==    by 0x11F07A: run_repl (compiler.c:8908)
==18371==    by 0x11AFE2: main (compiler.c:7805)
==18371== 
==18371== 72,000 bytes in 9 blocks are indirectly lost in loss record 1,327 of 1,331
==18371==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==18371==    by 0x10D5FC: v_malloc (compiler.c:3042)
==18371==    by 0x10A5C1: array_repeat (compiler.c:1049)
==18371==    by 0x114823: new_fn (compiler.c:6194)
==18371==    by 0x1149CA: Parser_fn_decl (compiler.c:6219)
==18371==    by 0x11F84B: Parser_parse (compiler.c:9091)
==18371==    by 0x11B762: V_compile (compiler.c:7976)
==18371==    by 0x11F07A: run_repl (compiler.c:8908)
==18371==    by 0x11AFE2: main (compiler.c:7805)
==18371== 
==18371== 80,000 bytes in 10 blocks are still reachable in loss record 1,328 of 1,331
==18371==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==18371==    by 0x10D5FC: v_malloc (compiler.c:3042)
==18371==    by 0x10A5C1: array_repeat (compiler.c:1049)
==18371==    by 0x114823: new_fn (compiler.c:6194)
==18371==    by 0x1149CA: Parser_fn_decl (compiler.c:6219)
==18371==    by 0x11F82E: Parser_parse (compiler.c:9083)
==18371==    by 0x11B762: V_compile (compiler.c:7976)
==18371==    by 0x11F07A: run_repl (compiler.c:8908)
==18371==    by 0x11AFE2: main (compiler.c:7805)
==18371== 
==18371== 89,673 (6,216 direct, 83,457 indirect) bytes in 37 blocks are definitely lost in loss record 1,329 of 1,331
==18371==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==18371==    by 0x10D5FC: v_malloc (compiler.c:3042)
==18371==    by 0x10D70C: memdup (compiler.c:3088)
==18371==    by 0x1148F1: new_fn (compiler.c:6194)
==18371==    by 0x1149CA: Parser_fn_decl (compiler.c:6219)
==18371==    by 0x11F84B: Parser_parse (compiler.c:9091)
==18371==    by 0x11B762: V_compile (compiler.c:7976)
==18371==    by 0x11F07A: run_repl (compiler.c:8908)
==18371==    by 0x11AFE2: main (compiler.c:7805)
==18371== 
==18371== 224,000 bytes in 28 blocks are still reachable in loss record 1,330 of 1,331
==18371==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==18371==    by 0x10D5FC: v_malloc (compiler.c:3042)
==18371==    by 0x10A5C1: array_repeat (compiler.c:1049)
==18371==    by 0x114823: new_fn (compiler.c:6194)
==18371==    by 0x1149CA: Parser_fn_decl (compiler.c:6219)
==18371==    by 0x11F84B: Parser_parse (compiler.c:9091)
==18371==    by 0x11B762: V_compile (compiler.c:7976)
==18371==    by 0x11F07A: run_repl (compiler.c:8908)
==18371==    by 0x11AFE2: main (compiler.c:7805)
==18371== 
==18371== 1,176,000 bytes in 147 blocks are still reachable in loss record 1,331 of 1,331
==18371==    at 0x483877F: malloc (vg_replace_malloc.c:299)
==18371==    by 0x10D5FC: v_malloc (compiler.c:3042)
==18371==    by 0x10A5C1: array_repeat (compiler.c:1049)
==18371==    by 0x114823: new_fn (compiler.c:6194)
==18371==    by 0x1149CA: Parser_fn_decl (compiler.c:6219)
==18371==    by 0x11F84B: Parser_parse (compiler.c:9091)
==18371==    by 0x11B20D: V_compile (compiler.c:7860)
==18371==    by 0x11F07A: run_repl (compiler.c:8908)
==18371==    by 0x11AFE2: main (compiler.c:7805)

I'm gonna spend some time going through these leaks. I'll link all relevant PRs to this issue. I recommend that all other contributors help out patching these up.

medvednikov commented 5 years ago

If you plan to add a bunch of free's, I wouldn't recommend doing it.

Soon V will automatically handle most of these cases.

But the V compiler will be compiled with a special flag, so that there are 0 frees in order to achieve the best performance.

RuneBlaze commented 5 years ago

Could you perhaps elaborate on how V plans to automatically handle these cases? From my limited understanding init_consts seems to be generated by the compiler so I assume it needs to be handled on an ad-hoc basis.