Assert fail/segfault in Perl_sv_pvn_force_flags

p5pRT commented 8 years ago

Migrated from rt.perl.org#126614 (status was 'resolved')

Searchable as RT126614$

p5pRT commented 8 years ago

From @dcollinsn

Greetings Porters\,

I have compiled bleadperl with the afl-gcc compiler using:

./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des AFL_HARDEN=1 make && make test

And then fuzzed the resulting binary using:

AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@

After reducing testcases using `afl-tmin` and performing additional minimization by hand\, I have located the following testcase that triggers an assert fail in DEBUGGING perls without any other symptoms in the normal perl interpreter. The testcase is the file:

/(?{})|(??{U:0})/|s|||g

dcollins@nightshade64:/usr/local/perl-afl/out$ ~/perldebug/perl -e '/(?{})|(??{U:0})/|s|||g' perl: sv.c:9875: Perl_sv_pvn_force_flags: Assertion `PL_valid_types_PVX[((svtype)((_svpvx)->sv_flags & 0xff)) & 0xf]' failed. Aborted

On non-debugging perls\, this eventually leads to a segfault:

==11561== Invalid read of size 8 ==11561== at 0x6630A5: Perl_pregfree2 (regcomp.c:17498) ==11561== by 0x7B2CE7: Perl_sv_clear (sv.c:6458) ==11561== by 0x7ADD0F: Perl_sv_free2 (sv.c:6885) ==11561== by 0x6E9504: S_SvREFCNT_dec (inline.h:166) ==11561== by 0x6E9504: S_mg_free_struct (mg.c:549) ==11561== by 0x6E9504: Perl_mg_free (mg.c:571) ==11561== by 0x7B2A0F: Perl_sv_clear (sv.c:6428) ==11561== by 0x7ADD0F: Perl_sv_free2 (sv.c:6885) ==11561== by 0x433874: S_SvREFCNT_dec (inline.h:166) ==11561== by 0x433874: Perl_op_clear (op.c:896) ==11561== by 0x43412B: Perl_op_free (op.c:779) ==11561== by 0x433FCB: Perl_op_free (op.c:762) ==11561== by 0x433808: Perl_op_clear (op.c:954) ==11561== by 0x43412B: Perl_op_free (op.c:779) ==11561== by 0x433FCB: Perl_op_free (op.c:762) ==11561== Address 0x28 is not stack'd\, malloc'd or (recently) free'd

**GDB**

(gdb) run Starting program: /home/dcollins/perldebug/perl -e /$\?\{\}$\|$\?\?\{U:0\}$/\|s\|\|\|g [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". perl: sv.c:9875: Perl_sv_pvn_force_flags: Assertion `PL_valid_types_PVX[((svtype)((_svpvx)->sv_flags & 0xff)) & 0xf]' failed.

Program received signal SIGABRT\, Aborted. 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007ffff6cf54e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007ffff6ced226 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007ffff6ced2d2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x00000000009d23dd in Perl_sv_pvn_force_flags (sv=sv@entry=0x11d8560\, lp=lp@entry=0x7fffffffdfd0\, flags=flags@entry=16384) at sv.c:9875 #5 0x00000000009dbb3e in Perl_sv_catpvn_flags (dsv=dsv@entry=0x11d8560\, sstr=0x11d2460 ""\, slen=0\, flags=16384) at sv.c:5361 #6 0x00000000008f2bf3 in Perl_pp_subst () at pp_hot.c:3133 #7 0x00000000007c90bf in Perl_runops_debug () at dump.c:2224 #8 0x0000000000532411 in S_run_body (oldscope=1) at perl.c:2464 #9 perl_run (my_perl=\) at perl.c:2387 #10 0x0000000000428928 in main (argc=3\, argv=0x7fffffffe338\, env=0x7fffffffe358) at perlmain.c:116 (gdb) f 4 #4 0x00000000009d23dd in Perl_sv_pvn_force_flags (sv=sv@entry=0x11d8560\, lp=lp@entry=0x7fffffffdfd0\, flags=flags@entry=16384) at sv.c:9875 9875 SvGROW(sv\, len + 1); (gdb) info locals _svpvx = 0x11d8560 s = \ len = 6 __PRETTY_FUNCTION__ = "Perl_sv_pvn_force_flags"

**VALGRIND**

==5662== Memcheck\, a memory error detector ==5662== Copyright (C) 2002-2015\, and GNU GPL'd\, by Julian Seward et al. ==5662== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==5662== Command: /home/dcollins/perldebug/perl -e /(?{})|(??{U:0})/|s|||g ==5662== perl: sv.c:9875: Perl_sv_pvn_force_flags: Assertion `PL_valid_types_PVX[((svtype)((_svpvx)->sv_flags & 0xff)) & 0xf]' failed. ==5662== ==5662== Process terminating with default action of signal 6 (SIGABRT) ==5662== at 0x5BDC107: raise (raise.c:56) ==5662== by 0x5BDD4E7: abort (abort.c:89) ==5662== by 0x5BD5225: __assert_fail_base (assert.c:92) ==5662== by 0x5BD52D1: __assert_fail (assert.c:101) ==5662== by 0x9D23DC: Perl_sv_pvn_force_flags (sv.c:9875) ==5662== by 0x9DBB3D: Perl_sv_catpvn_flags (sv.c:5361) ==5662== by 0x8F2BF2: Perl_pp_subst (pp_hot.c:3133) ==5662== by 0x7C90BE: Perl_runops_debug (dump.c:2224) ==5662== by 0x532410: S_run_body (perl.c:2464) ==5662== by 0x532410: perl_run (perl.c:2387) ==5662== by 0x428927: main (perlmain.c:116) ==5662== ==5662== HEAP SUMMARY: ==5662== in use at exit: 131\,879 bytes in 671 blocks ==5662== total heap usage: 858 allocs\, 187 frees\, 158\,569 bytes allocated ==5662== ==5662== LEAK SUMMARY: ==5662== definitely lost: 320 bytes in 1 blocks ==5662== indirectly lost: 2\,599 bytes in 38 blocks ==5662== possibly lost: 0 bytes in 0 blocks ==5662== still reachable: 128\,960 bytes in 632 blocks ==5662== suppressed: 0 bytes in 0 blocks ==5662== Rerun with --leak-check=full to see details of leaked memory ==5662== ==5662== For counts of detected and suppressed errors\, rerun with: -v ==5662== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Aborted

**PERL -V**

dcollins@nightshade64:/usr/local/perl-afl$ ./bin/perl -V Summary of my perl5 (revision 5 version 23 subversion 5) configuration: Commit id: 7195e5da55a40d15e29ad80562668bdd6895441f Platform: osname=linux\, osvers=3.16.0-4-amd64\, archname=x86_64-linux-ld uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux ' config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=define usemymalloc=n\, bincompat5005=undef Compiler: cc='ccache afl-gcc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-g'\, cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion=''\, gccversion='5.2.0'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='long double'\, nvsize=16\, Off_t='off_t'\, lseeksize=8 alignbytes=16\, prototype=define Linker and Libraries: ld='ccache afl-gcc'\, ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/local/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Oct 22 2015 15:44:40 @INC: /usr/local/perl-afl/lib/site_perl/5.23.5/x86_64-linux-ld /usr/local/perl-afl/lib/site_perl/5.23.5 /usr/local/perl-afl/lib/5.23.5/x86_64-linux-ld /usr/local/perl-afl/lib/5.23.5 /usr/local/perl-afl/lib/site_perl/5.23.4 /usr/local/perl-afl/lib/site_perl .

p5pRT commented 8 years ago

From @tonycoz

On Tue Nov 10 19:04:55 2015\, dcollinsn@gmail.com wrote:

After reducing testcases using `afl-tmin` and performing additional minimization by hand\, I have located the following testcase that triggers an assert fail in DEBUGGING perls without any other symptoms in the normal perl interpreter. The testcase is the file:

/(?{})|(??{U:0})/|s|||g

This can be simplified further to:

s/(?{})|(??{U:0})//g

dcollins@nightshade64:/usr/local/perl-afl/out$ ~/perldebug/perl -e '/(?{})|(??{U:0})/|s|||g' perl: sv.c:9875: Perl_sv_pvn_force_flags: Assertion `PL_valid_types_PVX[((svtype)((_svpvx)->sv_flags & 0xff)) & 0xf]' failed. Aborted

There's two problems in the original:

1) it looks like #124368 is biting us in a different way in that the regexp being used by pp_subst is the one from the match

2) pp_subst() (in either example) creates a new temp sv (line 3078):

dstr = newSVpvn_flags(orig\, s-orig\, SVs_TEMP | (DO_UTF8(TARG) ? SVf_UTF8 : 0));

then later calls CALLREGEXEC() (line 3129):

} while (CALLREGEXEC(rx\, s\, strend\, orig\, s == m\, /* Yields minend of 0 or 1 */ TARG\, NULL\, REXEC_NOT_FIRST|REXEC_IGNOREPOS|REXEC_FAIL_ON_UNDERFLOW));

and the pp_nextstate called when running the C\<U:0> code calls FREETMPS\, releasing dstr\, and so the following (line 3133) tries to work with a freed SV:

sv_catpvn_nomg_maybeutf8(dstr\, s\, strend - s\, DO_UTF8(TARG));

and asserts.

Tony

p5pRT commented 8 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 8 years ago

From @tonycoz

On Wed Dec 02 19:45:43 2015\, tonyc wrote:

1) it looks like #124368 is biting us in a different way in that the regexp being used by pp_subst is the one from the match

Or not\, s/// has the same empty regexp behaviour as m//

Tony

p5pRT commented 7 years ago

From @hvds

I can't reproduce this with blead. Bisect using:

% Porting/bisect.pl --start=7195e5da55a40d15e29ad80562668bdd6895441f --crash --expect-fail --target=miniperl -e '/(?{})|(??{U:0})/|s|||g'

..leads to 1dfbe6b4ad which is partway through the huge "revamp context system" from DaveM merged in February (at 9d876b687d):

It sounds reasonable to me that the context work fixes what appears to be a premature free\, so I'm resolving this ticket.

Hugo

p5pRT commented 7 years ago

@hvds - Status changed from 'open' to 'resolved'

Perl / perl5