search

westes / flex

The Fast Lexical Analyzer - scanner generator for lexing in C and C++
Other
3.56k stars 534 forks source link

Memory leaks in xstrdup #340

Closed xcainiao closed 4 months ago

xcainiao commented 6 years ago

./flex ./flex_memory_leaks_in_xstrdup

git log

commit 60b578fdaab4bf8d95203caaa947b476eeb03c10
Author: Simon Sobisch <simonsobisch@web.de>
Date:   Fri Mar 16 21:53:48 2018 +0100

    doc: describe mailing list usage in more detail

compile

./configure CC="gcc" CXX="g++" CFLAGS="-g -fsanitize=address"
make

error

==110424==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dd2 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dc3 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 3 byte(s) leaked in 2 allocation(s).

=================================================================
==110422==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dd2 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dc3 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 3 byte(s) leaked in 2 allocation(s).

=================================================================
==110421==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dd2 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dc3 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 3 byte(s) leaked in 2 allocation(s).

testcase https://github.com/xcainiao/poc/blob/master/flex_memory_leaks_in_xstrdup

westes commented 6 years ago

The leaks come from libasan.so.2, according to the trace you provided.

On Monday, 16 April 2018, 8:51 am +0000, xcainiao notifications@github.com wrote:

./flex ../../poc/flex_memory_leaks_in_xstrdup

git log

commit 60b578fdaab4bf8d95203caaa947b476eeb03c10
Author: Simon Sobisch <simonsobisch@web.de>
Date:   Fri Mar 16 21:53:48 2018 +0100

    doc: describe mailing list usage in more detail

compile

./configure CC="gcc" CXX="g++" CFLAGS="-g -fsanitize=address"
make

error

==110424==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dd2 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dc3 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 3 byte(s) leaked in 2 allocation(s).

=================================================================
==110422==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dd2 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dc3 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 3 byte(s) leaked in 2 allocation(s).

=================================================================
==110421==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dd2 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x7f320858230f in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x6230f)
    #1 0x420712 in xstrdup /home/fuzz/github/flex_o/src/misc.c:231
    #2 0x435dc3 in ndinstal /home/fuzz/github/flex_o/src/sym.c:180
    #3 0x441ee7 in flexscan scan.l:342
    #4 0x43d4c0 in yylex /home/fuzz/github/flex_o/src/yylex.c:51
    #5 0x427ca7 in yyparse /home/fuzz/github/flex_o/src/parse.c:1405
    #6 0x41edea in readin /home/fuzz/github/flex_o/src/main.c:1490
    #7 0x41af93 in flex_main /home/fuzz/github/flex_o/src/main.c:170
    #8 0x41b282 in main /home/fuzz/github/flex_o/src/main.c:209
    #9 0x7f3207e6d82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 3 byte(s) leaked in 2 allocation(s).

testcase https://github.com/xcainiao/poc/blob/master/flex_memory_leaks_in_xstrdup

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/westes/flex/issues/340

-- Will Estes westes575@gmail.com

xcainiao commented 6 years ago

No, this because dont free(s2) before program exit. (s2 = strdup(s))

westes commented 6 years ago

Then please submit a pull request.

On Monday, 16 April 2018, 12:52 pm +0000, xcainiao notifications@github.com wrote:

No, this because dont free(s2) before program exit.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/westes/flex/issues/340#issuecomment-381588550

-- Will Estes westes575@gmail.com

Explorer09 commented 6 years ago

@xcainiao The report only says leaking 1 or 2 bytes, which makes me suspicious. How did you test for such a small leak? BTW, I did find a leak could happen in ndinstal() if the addsym() function it calls returns non-zero.

xcainiao commented 6 years ago

I found this by fuzzer. Maybe same leaks as below code.

gcc -o strdupxx strdupxx.c -g -fsanitize=address -lm -ldl -lpthread

#include <stdio.h>
void main()
{
    char *p1 = "m";
    char *p2;
    p2 = strdup(p1);
    //free(p2);
}

output

==20348==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x45daaf in __interceptor_strdup (/home/c/strdupxx+0x45daaf)
    #1 0x4ac089 in main /home/c/strdupxx.c:6
    #2 0x7fb8013fb82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
SUMMARY: AddressSanitizer: 2 byte(s) leaked in 1 allocation(s).
Explorer09 commented 6 years ago

@xcainiao I don't need your sample demonstration about strdup() leaking -- I knew what it is. What I was asking is how did you make the leak in Flex specifically? Not everyone will compile flex with AddressSanitizer, and I would be happy if can constantly test for it, in Travis CI or other build servers, so we can avoid similar problems in the future (a large portion of flex code has not been written with neat style or robustness, and I bet you can find more leaks in other places in flex).

xcainiao commented 6 years ago

@Explorer09 Thank you for your explanation. I found this use LibFuzzer. You can get it from http://llvm.org/docs/LibFuzzer.html.

My simple LibFuzzer test code:

#define filename  "./test.poc"
//int main ()
int LLVMFuzzerTestOneInput(char *data, int size)
{
    int argc = 2;
    char *argv[2];
    FILE* temfile = fopen(filename,"w");
    fwrite(data, 1, size, temfile);
    fclose(temfile);
    argv[0] = "./flex";
    argv[1] = "./test.poc";
#if defined(ENABLE_NLS) && ENABLE_NLS
#if HAVE_LOCALE_H
    setlocale (LC_MESSAGES, "");
        setlocale (LC_CTYPE, "");
    textdomain (PACKAGE);
    bindtextdomain (PACKAGE, LOCALEDIR);
#endif
#endif
    return flex_main (argc, argv);
}