Assert fail w/o other symptoms - regcomp.c:13901 S_handle_regex_sets with a regex that seems sort of valid

p5pRT commented 9 years ago

Migrated from rt.perl.org#126481 (status was 'resolved')

Searchable as RT126481$

p5pRT commented 9 years ago

From @dcollinsn

Greetings Porters\,

I have compiled bleadperl with the afl-gcc compiler using:

./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des AFL_HARDEN=1 make && make test

And then fuzzed the resulting binary using:

AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@

After reducing testcases using `afl-tmin` and performing additional minimization by hand\, I have located the following testcase that triggers an assert fail in DEBUGGING perls without any other symptoms in the normal perl interpreter. The testcase is the file:

0=/(?[!!(\w])/

dcollins@nightshade64:/usr/local/perl-afl$ ./bin/perl -e '0=/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed. Aborted

The output with a normal perl is the expected error:

dcollins@nightshade64:/usr/local/perl-afl$ ~/perl/perl -e '0=/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. Unmatched ( in regex; marked by \<-- HERE in m/(?[!!(\w \<-- HERE ])/ at -e line 1.

**GDB**

(gdb) run Starting program: /usr/local/perl-afl/bin/perl -e 0=/$\?\[\!\!\(\\w\]$/ [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed.

Program received signal SIGABRT\, Aborted. 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007ffff6cf54e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007ffff6ced226 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007ffff6ced2d2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x000000000070ace9 in S_handle_regex_sets ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, return_invlist=return_invlist@entry=0x0\, flagp=flagp@entry=0x7fffffffd98c\, depth=depth@entry=5\, oregcomp_parse=oregcomp_parse@entry=0x11c3b41 "?[!!(\\w])") at regcomp.c:13901 #5 0x00000000006e23ea in S_reg (pRExC_state=0x7fffffffddd0\, paren=91\, flagp=0x7fffffffd98c\, depth=5) at regcomp.c:10492 #6 0x000000000070c37d in S_regatom ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, flagp=flagp@entry=0x7fffffffdb0c\, depth=depth@entry=4) at regcomp.c:11800 #7 0x0000000000718d39 in S_regpiece (depth=3\, flagp=\\, pRExC_state=0x7fffffffddd0) at regcomp.c:10878 #8 S_regbranch (pRExC_state=pRExC_state@entry=0x7fffffffddd0\, flagp=flagp@entry=0x7fffffffdbbc\, first=first@entry=1\, depth=depth@entry=2) at regcomp.c:10803 #9 0x0000000000738b03 in S_reg ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, ---Type \ to continue\, or q \ to quit--- flagp=flagp@entry=0x7fffffffdc9c\, depth=1\, paren=0) at regcomp.c:10548 #10 0x00000000007772eb in Perl_re_op_compile (patternp=patternp@entry=0x0\, pat_count=\\, pat_count@entry=0\, expr=\\, expr@entry=0x11c3918\, eng=0xef8d40 \<PL_core_reg_engine>\, old_re=old_re@entry=0x0\, is_bare_re=is_bare_re@entry=0x0\, orig_rx_flags=0\, pm_flags=0) at regcomp.c:6953 #11 0x00000000004d3fca in Perl_pmruntime (o=0x11c3958\, expr=0x11c3918\, repl=\\, isreg=isreg@entry=true\, floor=\) at op.c:5580 #12 0x00000000006550f5 in Perl_yyparse (gramtype=gramtype@entry=258) at perly.y:1032 #13 0x000000000051e645 in S_parse_body (env=env@entry=0x0\, xsinit=xsinit@entry=0x428c60 \<xs_init>) at perl.c:2307 #14 0x0000000000525437 in perl_parse (my_perl=\\, xsinit=xsinit@entry=0x428c60 \<xs_init>\, argc=\\, argv=\\, env=env@entry=0x0) at perl.c:1634 #15 0x0000000000428888 in main (argc=3\, argv=0x7fffffffe658\, env=0x7fffffffe678) at perlmain.c:114 (gdb) f 4 #4 0x000000000070ace9 in S_handle_regex_sets ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, return_invlist=return_invlist@entry=0x0\, flagp=flagp@entry=0x7fffffffd98c\, depth=depth@entry=5\, oregcomp_parse=oregcomp_parse@entry=0x11c3b41 "?[!!(\\w])") at regcomp.c:13901 13901 assert(IS_OPERAND(lhs)); (gdb) info locals stacked_operator = 33 '!' lhs = \ rhs = 0x11ba9d8 stacked_ptr = \ fence_ptr = \ top_index = \ top_ptr = \ current = 0x0 only_to_avoid_leaks = \ curchar = 93 ']' start = 18590168 end = 0 final = 0x0 result_string = \ fence_stack = 0x11ba948 fence = 2 len = 0 node = \ save_fold = false save_end = \ save_parse = \ in_locale = false re_debug_flags = 0 ---Type \ to continue\, or q \ to quit--- __PRETTY_FUNCTION__ = "S_handle_regex_sets" (gdb) q

**VALGRIND**

dcollins@nightshade64:/usr/local/perl-afl$ valgrind ./bin/perl -e '0=/(?[!!(\w])/' ==44884== Memcheck\, a memory error detector ==44884== Copyright (C) 2002-2015\, and GNU GPL'd\, by Julian Seward et al. ==44884== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==44884== Command: ./bin/perl -e 0=/(?[!!(\\w])/ ==44884== The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed. ==44884== ==44884== Process terminating with default action of signal 6 (SIGABRT) ==44884== at 0x5BDC107: raise (raise.c:56) ==44884== by 0x5BDD4E7: abort (abort.c:89) ==44884== by 0x5BD5225: __assert_fail_base (assert.c:92) ==44884== by 0x5BD52D1: __assert_fail (assert.c:101) ==44884== by 0x70ACE8: S_handle_regex_sets (regcomp.c:13901) ==44884== by 0x6E23E9: S_reg (regcomp.c:10492) ==44884== by 0x70C37C: S_regatom (regcomp.c:11800) ==44884== by 0x718D38: S_regpiece (regcomp.c:10878) ==44884== by 0x718D38: S_regbranch (regcomp.c:10803) ==44884== by 0x738B02: S_reg.constprop.46 (regcomp.c:10548) ==44884== by 0x7772EA: Perl_re_op_compile (regcomp.c:6953) ==44884== by 0x4D3FC9: Perl_pmruntime (op.c:5580) ==44884== by 0x6550F4: Perl_yyparse (perly.y:1032) ==44884== ==44884== HEAP SUMMARY: ==44884== in use at exit: 139\,830 bytes in 561 blocks ==44884== total heap usage: 688 allocs\, 127 frees\, 165\,712 bytes allocated ==44884== ==44884== LEAK SUMMARY: ==44884== definitely lost: 176 bytes in 1 blocks ==44884== indirectly lost: 1\,974 bytes in 20 blocks ==44884== possibly lost: 16 bytes in 1 blocks ==44884== still reachable: 137\,664 bytes in 539 blocks ==44884== suppressed: 0 bytes in 0 blocks ==44884== Rerun with --leak-check=full to see details of leaked memory ==44884== ==44884== For counts of detected and suppressed errors\, rerun with: -v ==44884== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Aborted

**PERL -V**

dcollins@nightshade64:/usr/local/perl-afl$ ./bin/perl -V Summary of my perl5 (revision 5 version 23 subversion 5) configuration: Commit id: 7195e5da55a40d15e29ad80562668bdd6895441f Platform: osname=linux\, osvers=3.16.0-4-amd64\, archname=x86_64-linux-ld uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux ' config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=define usemymalloc=n\, bincompat5005=undef Compiler: cc='ccache afl-gcc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-g'\, cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion=''\, gccversion='5.2.0'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='long double'\, nvsize=16\, Off_t='off_t'\, lseeksize=8 alignbytes=16\, prototype=define Linker and Libraries: ld='ccache afl-gcc'\, ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/local/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Oct 22 2015 15:44:40 @INC: /usr/local/perl-afl/lib/site_perl/5.23.5/x86_64-linux-ld /usr/local/perl-afl/lib/site_perl/5.23.5 /usr/local/perl-afl/lib/5.23.5/x86_64-linux-ld /usr/local/perl-afl/lib/5.23.5 /usr/local/perl-afl/lib/site_perl/5.23.4 /usr/local/perl-afl/lib/site_perl .

p5pRT commented 9 years ago

From @khwilliamson

On 10/28/2015 08:59 PM\, Dan Collins (via RT) wrote:

# New Ticket Created by Dan Collins # Please include the string: [perl #126481] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=126481 >

Greetings Porters\,

I have compiled bleadperl with the afl-gcc compiler using:

./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des AFL_HARDEN=1 make && make test

And then fuzzed the resulting binary using:

AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@

After reducing testcases using `afl-tmin` and performing additional minimization by hand\, I have located the following testcase that triggers an assert fail in DEBUGGING perls without any other symptoms in the normal perl interpreter. The testcase is the file:

0=/(?[!!(\w])/

dcollins@nightshade64:/usr/local/perl-afl$ ./bin/perl -e '0=/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed. Aborted

I have an easy fix about ready. I'm writing now only to say this is not a regression\, as

§ perl.5.22.0 -le 'qr/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13798: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed.

The output with a normal perl is the expected error:

dcollins@nightshade64:/usr/local/perl-afl$ ~/perl/perl -e '0=/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. Unmatched ( in regex; marked by \<-- HERE in m/(?[!!(\w \<-- HERE ])/ at -e line 1.

**GDB**

(gdb) run Starting program: /usr/local/perl-afl/bin/perl -e 0=/$\?\[\!\!\(\\w\]$/ [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed.

Program received signal SIGABRT\, Aborted. 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007ffff6cf54e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007ffff6ced226 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007ffff6ced2d2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x000000000070ace9 in S_handle_regex_sets ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, return_invlist=return_invlist@entry=0x0\, flagp=flagp@entry=0x7fffffffd98c\, depth=depth@entry=5\, oregcomp_parse=oregcomp_parse@entry=0x11c3b41 "?[!!(\\w])") at regcomp.c:13901 #5 0x00000000006e23ea in S_reg (pRExC_state=0x7fffffffddd0\, paren=91\, flagp=0x7fffffffd98c\, depth=5) at regcomp.c:10492 #6 0x000000000070c37d in S_regatom ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, flagp=flagp@entry=0x7fffffffdb0c\, depth=depth@entry=4) at regcomp.c:11800 #7 0x0000000000718d39 in S_regpiece (depth=3\, flagp=\\, pRExC_state=0x7fffffffddd0) at regcomp.c:10878 #8 S_regbranch (pRExC_state=pRExC_state@entry=0x7fffffffddd0\, flagp=flagp@entry=0x7fffffffdbbc\, first=first@entry=1\, depth=depth@entry=2) at regcomp.c:10803 #9 0x0000000000738b03 in S_reg ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, ---Type \ to continue\, or q \ to quit--- flagp=flagp@entry=0x7fffffffdc9c\, depth=1\, paren=0) at regcomp.c:10548 #10 0x00000000007772eb in Perl_re_op_compile (patternp=patternp@entry=0x0\, pat_count=\\, pat_count@entry=0\, expr=\\, expr@entry=0x11c3918\, eng=0xef8d40 \<PL_core_reg_engine>\, old_re=old_re@entry=0x0\, is_bare_re=is_bare_re@entry=0x0\, orig_rx_flags=0\, pm_flags=0) at regcomp.c:6953 #11 0x00000000004d3fca in Perl_pmruntime (o=0x11c3958\, expr=0x11c3918\, repl=\\, isreg=isreg@entry=true\, floor=\) at op.c:5580 #12 0x00000000006550f5 in Perl_yyparse (gramtype=gramtype@entry=258) at perly.y:1032 #13 0x000000000051e645 in S_parse_body (env=env@entry=0x0\, xsinit=xsinit@entry=0x428c60 \<xs_init>) at perl.c:2307 #14 0x0000000000525437 in perl_parse (my_perl=\\, xsinit=xsinit@entry=0x428c60 \<xs_init>\, argc=\\, argv=\\, env=env@entry=0x0) at perl.c:1634 #15 0x0000000000428888 in main (argc=3\, argv=0x7fffffffe658\, env=0x7fffffffe678) at perlmain.c:114 (gdb) f 4 #4 0x000000000070ace9 in S_handle_regex_sets ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, return_invlist=return_invlist@entry=0x0\, flagp=flagp@entry=0x7fffffffd98c\, depth=depth@entry=5\, oregcomp_parse=oregcomp_parse@entry=0x11c3b41 "?[!!(\\w])") at regcomp.c:13901 13901 assert(IS_OPERAND(lhs)); (gdb) info locals stacked_operator = 33 '!' lhs = \ rhs = 0x11ba9d8 stacked_ptr = \ fence_ptr = \ top_index = \ top_ptr = \ current = 0x0 only_to_avoid_leaks = \ curchar = 93 ']' start = 18590168 end = 0 final = 0x0 result_string = \ fence_stack = 0x11ba948 fence = 2 len = 0 node = \ save_fold = false save_end = \ save_parse = \ in_locale = false re_debug_flags = 0 ---Type \ to continue\, or q \ to quit--- __PRETTY_FUNCTION__ = "S_handle_regex_sets" (gdb) q

**VALGRIND**

dcollins@nightshade64:/usr/local/perl-afl$ valgrind ./bin/perl -e '0=/(?[!!(\w])/' ==44884== Memcheck\, a memory error detector ==44884== Copyright (C) 2002-2015\, and GNU GPL'd\, by Julian Seward et al. ==44884== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==44884== Command: ./bin/perl -e 0=/(?[!!(\\w])/ ==44884== The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed. ==44884== ==44884== Process terminating with default action of signal 6 (SIGABRT) ==44884== at 0x5BDC107: raise (raise.c:56) ==44884== by 0x5BDD4E7: abort (abort.c:89) ==44884== by 0x5BD5225: __assert_fail_base (assert.c:92) ==44884== by 0x5BD52D1: __assert_fail (assert.c:101) ==44884== by 0x70ACE8: S_handle_regex_sets (regcomp.c:13901) ==44884== by 0x6E23E9: S_reg (regcomp.c:10492) ==44884== by 0x70C37C: S_regatom (regcomp.c:11800) ==44884== by 0x718D38: S_regpiece (regcomp.c:10878) ==44884== by 0x718D38: S_regbranch (regcomp.c:10803) ==44884== by 0x738B02: S_reg.constprop.46 (regcomp.c:10548) ==44884== by 0x7772EA: Perl_re_op_compile (regcomp.c:6953) ==44884== by 0x4D3FC9: Perl_pmruntime (op.c:5580) ==44884== by 0x6550F4: Perl_yyparse (perly.y:1032) ==44884== ==44884== HEAP SUMMARY: ==44884== in use at exit: 139\,830 bytes in 561 blocks ==44884== total heap usage: 688 allocs\, 127 frees\, 165\,712 bytes allocated ==44884== ==44884== LEAK SUMMARY: ==44884== definitely lost: 176 bytes in 1 blocks ==44884== indirectly lost: 1\,974 bytes in 20 blocks ==44884== possibly lost: 16 bytes in 1 blocks ==44884== still reachable: 137\,664 bytes in 539 blocks ==44884== suppressed: 0 bytes in 0 blocks ==44884== Rerun with --leak-check=full to see details of leaked memory ==44884== ==44884== For counts of detected and suppressed errors\, rerun with: -v ==44884== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Aborted

**PERL -V**

dcollins@nightshade64:/usr/local/perl-afl$ ./bin/perl -V Summary of my perl5 (revision 5 version 23 subversion 5) configuration: Commit id: 7195e5da55a40d15e29ad80562668bdd6895441f Platform: osname=linux\, osvers=3.16.0-4-amd64\, archname=x86_64-linux-ld uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux ' config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=define usemymalloc=n\, bincompat5005=undef Compiler: cc='ccache afl-gcc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-g'\, cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion=''\, gccversion='5.2.0'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='long double'\, nvsize=16\, Off_t='off_t'\, lseeksize=8 alignbytes=16\, prototype=define Linker and Libraries: ld='ccache afl-gcc'\, ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/local/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Oct 22 2015 15:44:40 @INC: /usr/local/perl-afl/lib/site_perl/5.23.5/x86_64-linux-ld /usr/local/perl-afl/lib/site_perl/5.23.5 /usr/local/perl-afl/lib/5.23.5/x86_64-linux-ld /usr/local/perl-afl/lib/5.23.5 /usr/local/perl-afl/lib/site_perl/5.23.4 /usr/local/perl-afl/lib/site_perl .

p5pRT commented 9 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 9 years ago

From @khwilliamson

On 10/29/2015 01:18 PM\, Karl Williamson wrote:

On 10/28/2015 08:59 PM\, Dan Collins (via RT) wrote:

# New Ticket Created by Dan Collins # Please include the string: [perl #126481] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=126481 >

Greetings Porters\,

I have compiled bleadperl with the afl-gcc compiler using:

./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des AFL_HARDEN=1 make && make test

And then fuzzed the resulting binary using:

AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@

After reducing testcases using `afl-tmin` and performing additional minimization by hand\, I have located the following testcase that triggers an assert fail in DEBUGGING perls without any other symptoms in the normal perl interpreter. The testcase is the file:

0=/(?[!!(\w])/

dcollins@nightshade64:/usr/local/perl-afl$ ./bin/perl -e '0=/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed. Aborted

I have an easy fix about ready. I'm writing now only to say this is not a regression\, as

§ perl.5.22.0 -le 'qr/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13798: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed.

Actually\, I had a thinko. It IS a regression in 5.22 from 5.20\,

§ perl.5.20.2 -le 'qr/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. Incomplete expression within '(?[ ])' in regex; marked by \<-- HERE in m/(?[!!(\w \<-- HERE ])/ at -e line 1.

So it does qualify for a maintenance release. I'll submit a patch later today.

The output with a normal perl is the expected error:

dcollins@nightshade64:/usr/local/perl-afl$ ~/perl/perl -e '0=/(?[!!(\w])/' The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. Unmatched ( in regex; marked by \<-- HERE in m/(?[!!(\w \<-- HERE ])/ at -e line 1.

**GDB**

(gdb) run Starting program: /usr/local/perl-afl/bin/perl -e 0=/$\?\[\!\!\(\\w\]$/ [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed.

Program received signal SIGABRT\, Aborted. 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007ffff6cf54e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007ffff6ced226 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007ffff6ced2d2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x000000000070ace9 in S_handle_regex_sets ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, return_invlist=return_invlist@entry=0x0\, flagp=flagp@entry=0x7fffffffd98c\, depth=depth@entry=5\, oregcomp_parse=oregcomp_parse@entry=0x11c3b41 "?[!!(\\w])") at regcomp.c:13901 #5 0x00000000006e23ea in S_reg (pRExC_state=0x7fffffffddd0\, paren=91\, flagp=0x7fffffffd98c\, depth=5) at regcomp.c:10492 #6 0x000000000070c37d in S_regatom ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, flagp=flagp@entry=0x7fffffffdb0c\, depth=depth@entry=4) at regcomp.c:11800 #7 0x0000000000718d39 in S_regpiece (depth=3\, flagp=\\, pRExC_state=0x7fffffffddd0) at regcomp.c:10878 #8 S_regbranch (pRExC_state=pRExC_state@entry=0x7fffffffddd0\, flagp=flagp@entry=0x7fffffffdbbc\, first=first@entry=1\, depth=depth@entry=2) at regcomp.c:10803 #9 0x0000000000738b03 in S_reg ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, ---Type \ to continue\, or q \ to quit--- flagp=flagp@entry=0x7fffffffdc9c\, depth=1\, paren=0) at regcomp.c:10548 #10 0x00000000007772eb in Perl_re_op_compile (patternp=patternp@entry=0x0\, pat_count=\\, pat_count@entry=0\, expr=\\, expr@entry=0x11c3918\, eng=0xef8d40 \<PL_core_reg_engine>\, old_re=old_re@entry=0x0\, is_bare_re=is_bare_re@entry=0x0\, orig_rx_flags=0\, pm_flags=0) at regcomp.c:6953 #11 0x00000000004d3fca in Perl_pmruntime (o=0x11c3958\, expr=0x11c3918\, repl=\\, isreg=isreg@entry=true\, floor=\) at op.c:5580 #12 0x00000000006550f5 in Perl_yyparse (gramtype=gramtype@entry=258) at perly.y:1032 #13 0x000000000051e645 in S_parse_body (env=env@entry=0x0\, xsinit=xsinit@entry=0x428c60 \<xs_init>) at perl.c:2307 #14 0x0000000000525437 in perl_parse (my_perl=\\, xsinit=xsinit@entry=0x428c60 \<xs_init>\, argc=\\, argv=\\, env=env@entry=0x0) at perl.c:1634 #15 0x0000000000428888 in main (argc=3\, argv=0x7fffffffe658\, env=0x7fffffffe678) at perlmain.c:114 (gdb) f 4 #4 0x000000000070ace9 in S_handle_regex_sets ( pRExC_state=pRExC_state@entry=0x7fffffffddd0\, return_invlist=return_invlist@entry=0x0\, flagp=flagp@entry=0x7fffffffd98c\, depth=depth@entry=5\, oregcomp_parse=oregcomp_parse@entry=0x11c3b41 "?[!!(\\w])") at regcomp.c:13901 13901 assert(IS_OPERAND(lhs)); (gdb) info locals stacked_operator = 33 '!' lhs = \ rhs = 0x11ba9d8 stacked_ptr = \ fence_ptr = \ top_index = \ top_ptr = \ current = 0x0 only_to_avoid_leaks = \ curchar = 93 ']' start = 18590168 end = 0 final = 0x0 result_string = \ fence_stack = 0x11ba948 fence = 2 len = 0 node = \ save_fold = false save_end = \ save_parse = \ in_locale = false re_debug_flags = 0 ---Type \ to continue\, or q \ to quit--- __PRETTY_FUNCTION__ = "S_handle_regex_sets" (gdb) q

**VALGRIND**

dcollins@nightshade64:/usr/local/perl-afl$ valgrind ./bin/perl -e '0=/(?[!!(\w])/' ==44884== Memcheck\, a memory error detector ==44884== Copyright (C) 2002-2015\, and GNU GPL'd\, by Julian Seward et al. ==44884== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==44884== Command: ./bin/perl -e 0=/(?[!!(\\w])/ ==44884== The regex_sets feature is experimental in regex; marked by \<-- HERE in m/(?[ \<-- HERE !!(\w])/ at -e line 1. perl: regcomp.c:13901: S_handle_regex_sets: Assertion `(! ((lhs)->sv_flags & 0x00000100))' failed. ==44884== ==44884== Process terminating with default action of signal 6 (SIGABRT) ==44884== at 0x5BDC107: raise (raise.c:56) ==44884== by 0x5BDD4E7: abort (abort.c:89) ==44884== by 0x5BD5225: __assert_fail_base (assert.c:92) ==44884== by 0x5BD52D1: __assert_fail (assert.c:101) ==44884== by 0x70ACE8: S_handle_regex_sets (regcomp.c:13901) ==44884== by 0x6E23E9: S_reg (regcomp.c:10492) ==44884== by 0x70C37C: S_regatom (regcomp.c:11800) ==44884== by 0x718D38: S_regpiece (regcomp.c:10878) ==44884== by 0x718D38: S_regbranch (regcomp.c:10803) ==44884== by 0x738B02: S_reg.constprop.46 (regcomp.c:10548) ==44884== by 0x7772EA: Perl_re_op_compile (regcomp.c:6953) ==44884== by 0x4D3FC9: Perl_pmruntime (op.c:5580) ==44884== by 0x6550F4: Perl_yyparse (perly.y:1032) ==44884== ==44884== HEAP SUMMARY: ==44884== in use at exit: 139\,830 bytes in 561 blocks ==44884== total heap usage: 688 allocs\, 127 frees\, 165\,712 bytes allocated ==44884== ==44884== LEAK SUMMARY: ==44884== definitely lost: 176 bytes in 1 blocks ==44884== indirectly lost: 1\,974 bytes in 20 blocks ==44884== possibly lost: 16 bytes in 1 blocks ==44884== still reachable: 137\,664 bytes in 539 blocks ==44884== suppressed: 0 bytes in 0 blocks ==44884== Rerun with --leak-check=full to see details of leaked memory ==44884== ==44884== For counts of detected and suppressed errors\, rerun with: -v ==44884== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Aborted

**PERL -V**

dcollins@nightshade64:/usr/local/perl-afl$ ./bin/perl -V Summary of my perl5 (revision 5 version 23 subversion 5) configuration: Commit id: 7195e5da55a40d15e29ad80562668bdd6895441f Platform: osname=linux\, osvers=3.16.0-4-amd64\, archname=x86_64-linux-ld uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux ' config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=define usemymalloc=n\, bincompat5005=undef Compiler: cc='ccache afl-gcc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-g'\, cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion=''\, gccversion='5.2.0'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='long double'\, nvsize=16\, Off_t='off_t'\, lseeksize=8 alignbytes=16\, prototype=define Linker and Libraries: ld='ccache afl-gcc'\, ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/local/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Oct 22 2015 15:44:40 @INC: /usr/local/perl-afl/lib/site_perl/5.23.5/x86_64-linux-ld /usr/local/perl-afl/lib/site_perl/5.23.5 /usr/local/perl-afl/lib/5.23.5/x86_64-linux-ld /usr/local/perl-afl/lib/5.23.5 /usr/local/perl-afl/lib/site_perl/5.23.4 /usr/local/perl-afl/lib/site_perl .

p5pRT commented 9 years ago

From @khwilliamson

Thanks for finding this. Now fixed by a82f4918f5debccfb7e9a7047d2c2e558df538cd

-- Karl Williamson

p5pRT commented 9 years ago

@khwilliamson - Status changed from 'open' to 'pending release'

p5pRT commented 8 years ago

@mauke - Status changed from 'pending release' to 'resolved'

Perl / perl5