search

ihrwein / actiondb

A safe and efficient unstructured text (log) parsing library.
https://github.com/ihrwein/actiondb
20 stars 5 forks source link

Crash, when we using pattern_file with wrong syntax #100

Open mitzkia opened 8 years ago

mitzkia commented 8 years ago

relevant part of syslog-ng.conf:

pattern_file("/home/testbot/syslog-ng-project/install/pattern.json")

backtrace:

Error parsing native-module, syntax error, unexpected LL_IDENTIFIER, expecting ')' in syslog-ng.conf at line 10, column 5:

pattern_file("/home/testbot/syslog-ng-project/install/pattern.json")
^^^^^^^^^^^^

syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng * Error in `sbin/syslog-ng': corrupted double-linked list: 0x0000000000abb650 * ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x77725)[0x7f170080c725] /lib/x86_64-linux-gnu/libc.so.6(+0x80679)[0x7f1700815679] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f1700818abc] /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so(+0xe5828)[0x7f16fe673828] /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so(+0xe7275)[0x7f16fe675275] /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so(+0xed949)[0x7f16fe67b949] /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so(+0xed908)[0x7f16fe67b908] /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so(+0xed8e5)[0x7f16fe67b8e5] /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so(+0xfe28e)[0x7f16fe68c28e] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(log_pipe_unref+0x7c)[0x7f170108c0f6] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(+0x4b373)[0x7f170109c373] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(plugin_parse_config+0x19d)[0x7f170109ca8e] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(parser_expr_parse+0xc29)[0x7f17010c5b59] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(+0x5f05b)[0x7f17010b005b] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(main_parse+0x11ab)[0x7f17010b1c40] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(+0x2ab23)[0x7f170107bb23] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(cfg_run_parser+0x74)[0x7f170107c90c] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(cfg_read_config+0x93)[0x7f170107cb4f] /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0(main_loop_read_and_init_config+0x3e)[0x7f17010971c0] sbin/syslog-ng(main+0x207)[0x401b6f] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f17007b5830] sbin/syslog-ng(_start+0x29)[0x401669] ======= Memory map: ======== 00400000-00403000 r-xp 00000000 00:2b 25303755 /home/testbot/syslog-ng-project/install/sbin/syslog-ng 00602000-00603000 r--p 00002000 00:2b 25303755 /home/testbot/syslog-ng-project/install/sbin/syslog-ng 00603000-00604000 rw-p 00003000 00:2b 25303755 /home/testbot/syslog-ng-project/install/sbin/syslog-ng 00a61000-00acf000 rw-p 00000000 00:00 0 [heap] 7f16f8000000-7f16f8021000 rw-p 00000000 00:00 0 7f16f8021000-7f16fc000000 ---p 00000000 00:00 0 7f16fe06f000-7f16fe177000 r-xp 00000000 00:28 153 /lib/x86_64-linux-gnu/libm-2.23.so 7f16fe177000-7f16fe376000 ---p 00108000 00:28 153 /lib/x86_64-linux-gnu/libm-2.23.so 7f16fe376000-7f16fe377000 r--p 00107000 00:28 153 /lib/x86_64-linux-gnu/libm-2.23.so 7f16fe377000-7f16fe378000 rw-p 00108000 00:28 153 /lib/x86_64-linux-gnu/libm-2.23.so 7f16fe378000-7f16fe38e000 r-xp 00000000 00:28 141 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f16fe38e000-7f16fe58d000 ---p 00016000 00:28 141 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f16fe58d000-7f16fe58e000 rw-p 00015000 00:28 141 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f16fe58e000-7f16fea34000 r-xp 00000000 00:2b 24778846 /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so 7f16fea34000-7f16fec33000 ---p 004a6000 00:2b 24778846 /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so 7f16fec33000-7f16fec4b000 r--p 004a5000 00:2b 24778846 /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so 7f16fec4b000-7f16fec53000 rw-p 004bd000 00:2b 24778846 /home/testbot/syslog-ng-project/install/lib/syslog-ng/libactiondb_parser.so 7f16fec53000-7f16fec68000 r-xp 00000000 00:2b 24778710 /home/testbot/syslog-ng-project/install/lib/syslog-ng/libaffile.so 7f16fec68000-7f16fee68000 ---p 00015000 00:2b 24778710 /home/testbot/syslog-ng-project/install/lib/syslog-ng/libaffile.so 7f16fee68000-7f16fee69000 r--p 00015000 00:2b 24778710 /home/testbot/syslog-ng-project/install/lib/syslog-ng/libaffile.so 7f16fee69000-7f16fee6a000 rw-p 00016000 00:2b 24778710 /home/testbot/syslog-ng-project/install/lib/syslog-ng/libaffile.so 7f16fee6a000-7f16fee6b000 rw-p 00000000 00:00 0 7f16fee6b000-7f16fee76000 r-xp 00000000 00:28 56 /lib/x86_64-linux-gnu/libnss_files-2.23.so 7f16fee76000-7f16ff075000 ---p 0000b000 00:28 56 /lib/x86_64-linux-gnu/libnss_files-2.23.so 7f16ff075000-7f16ff076000 r--p 0000a000 00:28 56 /lib/x86_64-linux-gnu/libnss_files-2.23.so 7f16ff076000-7f16ff077000 rw-p 0000b000 00:28 56 /lib/x86_64-linux-gnu/libnss_files-2.23.so 7f16ff077000-7f16ff07d000 rw-p 00000000 00:00 0 7f16ff07d000-7f16ff096000 r-xp 00000000 00:2b 24777433 /home/testbot/syslog-ng-project/install/lib/libz.so.1.2.3.3 7f16ff096000-7f16ff295000 ---p 00019000 00:2b 24777433 /home/testbot/syslog-ng-project/install/lib/libz.so.1.2.3.3 7f16ff295000-7f16ff296000 r--p 00018000 00:2b 24777433 /home/testbot/syslog-ng-project/install/lib/libz.so.1.2.3.3 7f16ff296000-7f16ff297000 rw-p 00019000 00:2b 24777433 /home/testbot/syslog-ng-project/install/lib/libz.so.1.2.3.3 7f16ff297000-7f16ff29a000 r-xp 00000000 00:28 42 /lib/x86_64-linux-gnu/libdl-2.23.so 7f16ff29a000-7f16ff499000 ---p 00003000 00:28 42 /lib/x86_64-linux-gnu/libdl-2.23.so 7f16ff499000-7f16ff49a000 r--p 00002000 00:28 42 /lib/x86_64-linux-gnu/libdl-2.23.so 7f16ff49a000-7f16ff49b000 rw-p 00003000 00:28 42 /lib/x86_64-linux-gnu/libdl-2.23.so 7f16ff49b000-7f16ff697000 r-xp 00000000 00:2b 24777889 /home/testbot/syslog-ng-project/install/lib/libcrypto.so.1.0.0 7f16ff697000-7f16ff896000 ---p 001fc000 00:2b 24777889 /home/testbot/syslog-ng-project/install/lib/libcrypto.so.1.0.0 7f16ff896000-7f16ff8b0000 r--p 001fb000 00:2b 24777889 /home/testbot/syslog-ng-project/install/lib/libcrypto.so.1.0.0 7f16ff8b0000-7f16ff8bc000 rw-p 00215000 00:2b 24777889 /home/testbot/syslog-ng-project/install/lib/libcrypto.so.1.0.0 7f16ff8bc000-7f16ff8bf000 rw-p 00000000 00:00 0 7f16ff8bf000-7f16ff92f000 r-xp 00000000 00:2b 24777892 /home/testbot/syslog-ng-project/install/lib/libssl.so.1.0.0 7f16ff92f000-7f16ffb2f000 ---p 00070000 00:2b 24777892 /home/testbot/syslog-ng-project/install/lib/libssl.so.1.0.0 7f16ffb2f000-7f16ffb32000 r--p 00070000 00:2b 24777892 /home/testbot/syslog-ng-project/install/lib/libssl.so.1.0.0 7f16ffb32000-7f16ffb38000 rw-p 00073000 00:2b 24777892 /home/testbot/syslog-ng-project/install/lib/libssl.so.1.0.0 7f16ffb38000-7f16ffb43000 r-xp 00000000 00:2b 24777701 /home/testbot/syslog-ng-project/install/lib/libivykis.so.0.3.6 7f16ffb43000-7f16ffd43000 ---p 0000b000 00:2b 24777701 /home/testbot/syslog-ng-project/install/lib/libivykis.so.0.3.6 7f16ffd43000-7f16ffd44000 r--p 0000b000 00:2b 24777701 /home/testbot/syslog-ng-project/install/lib/libivykis.so.0.3.6 7f16ffd44000-7f16ffd45000 rw-p 0000c000 00:2b 24777701 /home/testbot/syslog-ng-project/install/lib/libivykis.so.0.3.6 7f16ffd45000-7f16ffd7e000 r-xp 00000000 00:2b 24777519 /home/testbot/syslog-ng-project/install/lib/libpcre.so.0.0.1 7f16ffd7e000-7f16fff7d000 ---p 00039000 00:2b 24777519 /home/testbot/syslog-ng-project/install/lib/libpcre.so.0.0.1 7f16fff7d000-7f16fff7e000 r--p 00038000 00:2b 24777519 /home/testbot/syslog-ng-project/install/lib/libpcre.so.0.0.1 7f16fff7e000-7f16fff7f000 rw-p 00039000 00:2b 24777519 /home/testbot/syslog-ng-project/install/lib/libpcre.so.0.0.1 7f16fff7f000-7f16fff83000 r-xp 00000000 00:2b 24778487 /home/testbot/syslog-ng-project/install/lib/libevtlog.so.0.0.0 7f16fff83000-7f1700182000 ---p 00004000 00:2b 24778487 /home/testbot/syslog-ng-project/install/lib/libevtlog.so.0.0.0 7f1700182000-7f1700183000 r--p 00003000 00:2b 24778487 /home/testbot/syslog-ng-project/install/lib/libevtlog.so.0.0.0 7f1700183000-7f1700184000 rw-p 00004000 00:2b 24778487 /home/testbot/syslog-ng-project/install/lib/libevtlog.so.0.0.0 7f1700184000-7f1700188000 r-xp 00000000 00:2b 24777956 /home/testbot/syslog-ng-project/install/lib/libgthread-2.0.so.0.1400.6 7f1700188000-7f1700387000 ---p 00004000 00:2b 24777956 /home/testbot/syslog-ng-project/install/lib/libgthread-2.0.so.0.1400.6 7f1700387000-7f1700388000 r--p 00003000 00:2b 24777956 /home/testbot/syslog-ng-project/install/lib/libgthread-2.0.so.0.1400.6 7f1700388000-7f1700389000 rw-p 00004000 00:2b 24777956 /home/testbot/syslog-ng-project/install/lib/libgthread-2.0.so.0.1400.6 7f1700389000-7f170038c000 r-xp 00000000 00:2b 24777952 /home/testbot/syslog-ng-project/install/lib/libgmodule-2.0.so.0.1400.6 7f170038c000-7f170058b000 ---p 00003000 00:2b 24777952 /home/testbot/syslog-ng-project/install/lib/libgmodule-2.0.so.0.1400.6 7f170058b000-7f170058c000 r--p 00002000 00:2b 24777952 /home/testbot/syslog-ng-project/install/lib/libgmodule-2.0.so.0.1400.6 7f170058c000-7f170058d000 rw-p 00003000 00:2b 24777952 /home/testbot/syslog-ng-project/install/lib/libgmodule-2.0.so.0.1400.6 7f170058d000-7f1700594000 r-xp 00000000 00:28 3699 /lib/x86_64-linux-gnu/librt-2.23.so 7f1700594000-7f1700793000 ---p 00007000 00:28 3699 /lib/x86_64-linux-gnu/librt-2.23.so 7f1700793000-7f1700794000 r--p 00006000 00:28 3699 /lib/x86_64-linux-gnu/librt-2.23.so 7f1700794000-7f1700795000 rw-p 00007000 00:28 3699 /lib/x86_64-linux-gnu/librt-2.23.so 7f1700795000-7f1700955000 r-xp 00000000 00:28 44 /lib/x86_64-linux-gnu/libc-2.23.so 7f1700955000-7f1700b54000 ---p 001c0000 00:28 44 /lib/x86_64-linux-gnu/libc-2.23.so 7f1700b54000-7f1700b58000 r--p 001bf000 00:28 44 /lib/x86_64-linux-gnu/libc-2.23.so 7f1700b58000-7f1700b5a000 rw-p 001c3000 00:28 44 /lib/x86_64-linux-gnu/libc-2.23.so 7f1700b5a000-7f1700b5e000 rw-p 00000000 00:00 0 7f1700b5e000-7f1700b76000 r-xp 00000000 00:28 83 /lib/x86_64-linux-gnu/libpthread-2.23.so 7f1700b76000-7f1700d75000 ---p 00018000 00:28 83 /lib/x86_64-linux-gnu/libpthread-2.23.so 7f1700d75000-7f1700d76000 r--p 00017000 00:28 83 /lib/x86_64-linux-gnu/libpthread-2.23.so 7f1700d76000-7f1700d77000 rw-p 00018000 00:28 83 /lib/x86_64-linux-gnu/libpthread-2.23.so 7f1700d77000-7f1700d7b000 rw-p 00000000 00:00 0 7f1700d7b000-7f1700e50000 r-xp 00000000 00:2b 24777941 /home/testbot/syslog-ng-project/install/lib/libglib-2.0.so.0.1400.6 7f1700e50000-7f170104f000 ---p 000d5000 00:2b 24777941 /home/testbot/syslog-ng-project/install/lib/libglib-2.0.so.0.1400.6 7f170104f000-7f1701050000 r--p 000d4000 00:2b 24777941 /home/testbot/syslog-ng-project/install/lib/libglib-2.0.so.0.1400.6 7f1701050000-7f1701051000 rw-p 000d5000 00:2b 24777941 /home/testbot/syslog-ng-project/install/lib/libglib-2.0.so.0.1400.6 7f1701051000-7f1701118000 r-xp 00000000 00:2b 24778638 /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0.0.0 7f1701118000-7f1701317000 ---p 000c7000 00:2b 24778638 /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0.0.0 7f1701317000-7f170131a000 r--p 000c6000 00:2b 24778638 /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0.0.0 7f170131a000-7f1701320000 rw-p 000c9000 00:2b 24778638 /home/testbot/syslog-ng-project/install/lib/libsyslog-ng-7.0.so.0.0.0 7f1701320000-7f1701321000 rw-p 00000000 00:00 0 7f1701321000-7f1701347000 r-xp 00000000 00:28 29 /lib/x86_64-linux-gnu/ld-2.23.so 7f170152c000-7f1701533000 r--s 00000000 00:28 4226 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 7f1701533000-7f170153d000 rw-p 00000000 00:00 0 7f1701540000-7f1701541000 rw-p 00000000 00:00 0 7f1701541000-7f1701543000 rw-s 00000000 00:2c 3 /dev/shm/mongoc-3489 (deleted) 7f1701543000-7f1701546000 rw-p 00000000 00:00 0 7f1701546000-7f1701547000 r--p 00025000 00:28 29 /lib/x86_64-linux-gnu/ld-2.23.so 7f1701547000-7f1701548000 rw-p 00026000 00:28 29 /lib/x86_64-linux-gnu/ld-2.23.so 7f1701548000-7f1701549000 rw-p 00000000 00:00 0 7ffd3422b000-7ffd3424c000 rw-p 00000000 00:00 0 [stack] 7ffd343c4000-7ffd343c6000 r--p 00000000 00:00 0 [vvar] 7ffd343c6000-7ffd343c8000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Aborted (core dumped)

ihrwein commented 8 years ago

It's easily reproducible, thanks for reporting!

bkil-syslogng commented 8 years ago

It's only reproducible if you ship it via a single click container, otherwise you're hours away from even starting...

ihrwein commented 8 years ago

Backtrace (trough valgrind):

mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng
==3625== Invalid free() / delete / delete[] / realloc()
==3625==    at 0x4C2BDEC: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3625==    by 0x75AD6BE: yydestruct (native-grammar.y:297)
==3625==    by 0x75AE9AE: native_parse (native-grammar.c:2984)
==3625==    by 0x4E81EDC: cfg_parser_parse (cfg-parser.h:83)
==3625==    by 0x4E82650: plugin_parse_config (plugin.c:216)
==3625==    by 0x4EAA362: parser_expr_parse (parser-expr-grammar.y:375)
==3625==    by 0x4E94E06: cfg_parser_parse (cfg-parser.h:83)
==3625==    by 0x4E96998: main_parse (cfg-grammar.y:630)
==3625==    by 0x4E6398C: cfg_parser_parse (cfg-parser.h:83)
==3625==    by 0x4E6474D: cfg_run_parser (cfg.c:367)
==3625==    by 0x4E6498E: cfg_read_config (cfg.c:439)
==3625==    by 0x4E7D86F: main_loop_read_and_init_config (mainloop.c:444)
==3625==  Address 0x6da96f0 is 0 bytes inside a block of size 12 free'd
==3625==    at 0x4C2BDEC: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3625==    by 0x4E66BCF: cfg_lexer_free_token (cfg-lexer.c:740)
==3625==    by 0x4E679D3: cfg_token_block_free (cfg-lexer.c:1110)
==3625==    by 0x4E66D64: cfg_lexer_lex (cfg-lexer.c:789)
==3625==    by 0x75AECEA: native_lex (native-parser.c:51)
==3625==    by 0x75ADC55: native_parse (native-grammar.c:2698)
==3625==    by 0x4E81EDC: cfg_parser_parse (cfg-parser.h:83)
==3625==    by 0x4E82650: plugin_parse_config (plugin.c:216)
==3625==    by 0x4EAA362: parser_expr_parse (parser-expr-grammar.y:375)
==3625==    by 0x4E94E06: cfg_parser_parse (cfg-parser.h:83)
==3625==    by 0x4E96998: main_parse (cfg-grammar.y:630)
==3625==    by 0x4E6398C: cfg_parser_parse (cfg-parser.h:83)
ihrwein commented 8 years ago

I think the problem is with this LL_IDENTIFIER: https://github.com/balabit/syslog-ng/blob/master/modules/native/native-grammar.ym#L82

The tricky free comes from this location: https://github.com/balabit/syslog-ng/blob/3ca4f76be44d754229433ee760bd903f121fd1a6/lib/cfg-lexer.c#L740

Removing the free() call eliminates the double free (valgrind is also happy).