Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.97k stars 560 forks source link

SIGFPE integer divide by zero in S_make_trie #14945

Closed p5pRT closed 8 years ago

p5pRT commented 9 years ago

Migrated from rt.perl.org#126206 (status was 'resolved')

Searchable as RT126206$

p5pRT commented 9 years ago

From @dcollinsn

Greetings Porters\,

I have compiled bleadperl with the afl-gcc compiler using​:

./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitint -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des AFL_HARDEN=1 make && make test

And then fuzzed the resulting binary using​:

AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @​@​

After reducing testcases using `afl-tmin` and performing additional minimization by hand\, I have located the following testcase that triggers a floating point exception in the perl interpreter. The testcase is the attached 11-character file​:

dcollins@​nagios​:\~/perl$ od -c /usr/local/perl-afl/out/allcrash/f3i000002 0000000 t . / \ N { } 337 | / i

This is very similar to [perl #125990]\, however this testcase never triggered that bug\, and was presumably missed when the fuzzer found it alongside the many testcases that did. A debugging perl produces no other interesting output.

The version of 5.20.2 that shipped with my Debian does not have this bug. The result of a bisect run is shown below\, and the problem does arise from the same commit\, where \N{} forces /d to /u.

**GDB**

dcollins@​nagios​:\~/perl$ gdb --args ./perl /usr/local/perl-afl/out/allcrash/f3i000002 GNU gdb (GDB) 7.10 Copyright (C) 2015 Free Software Foundation\, Inc. License GPLv3+​: GNU GPL version 3 or later \<http​://gnu.org/licenses/gpl.html> This is free software​: you are free to change and redistribute it. There is NO WARRANTY\, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions\, please see​: \<http​://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at​: \<http​://www.gnu.org/software/gdb/documentation/>. For help\, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./perl...done. (gdb) run Starting program​: /home/dcollins/perl/perl /usr/local/perl-afl/out/allcrash/f3i000002 bt[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/libthread_db.so.1".

Program received signal SIGFPE\, Arithmetic exception. 0x08119636 in S_make_trie (pRExC_state=0xbfffed64\, startbranch=0x8477d8c\,   first=0x8477d8c\, last=0x8477da4\, tail=0x8477da4\, word_count=2\, flags=35\,   depth=1) at regcomp.c​:2775 2775 accept_state = TRIE_NODENUM( state ); (gdb) bt #0 0x08119636 in S_make_trie (pRExC_state=0xbfffed64\, startbranch=0x8477d8c\,   first=0x8477d8c\, last=0x8477da4\, tail=0x8477da4\, word_count=2\, flags=35\,   depth=1) at regcomp.c​:2775 #1 0x08123811 in S_study_chunk (pRExC_state=0xbfffed64\, scanp=0xbfffeb54\,   minlenp=0xbfffeb5c\, deltap=0xbfffeb68\, last=0x8477da8\, data=0xbfffece4\,   stopparen=-1\, recursed_depth=0\, and_withp=0x8478f60\, flags=12288\, depth=0)   at regcomp.c​:4368 #2 0x08132920 in Perl_re_op_compile (patternp=0x0\, pat_count=1\,   expr=0x8477cdc\, eng=0x83917e0 \<PL_core_reg_engine>\, old_re=0x0\,   is_bare_re=0x0\, orig_rx_flags=4\, pm_flags=4) at regcomp.c​:7296 #3 0x08071183 in Perl_pmruntime (o=0x8477ca4\, expr=0x8477cdc\, repl=0x0\,   isreg=true\, floor=0) at op.c​:5579 #4 0x08107385 in Perl_yyparse (gramtype=258) at perly.y​:1038 #5 0x0809b04e in S_parse_body (env=0x0\, xsinit=0x80612f8 \<xs_init>)   at perl.c​:2304 #6 0x0809983a in perl_parse (my_perl=0x845d008\, xsinit=0x80612f8 \<xs_init>\,   argc=2\, argv=0xbffff4a4\, env=0x0) at perl.c​:1634 #7 0x08061252 in main (argc=2\, argv=0xbffff4a4\, env=0xbffff4b0)   at perlmain.c​:114 (gdb) info locals uc = 0x8477d94 "\002%\004" e = 0x8477d94 "\002%\004" state = 1 accept_state = 0 wordlen = 0 noper = 0x8477d90 charid = 0 trie = 0x8478fe0 widecharmap = 0x0 revcharmap = 0x8470614 cur = 0x8477d8c len = 0 uvc = 0 curword = 0 next_alloc = 1 jumper = 0x0 nextbranch = 0x0 convert = 0x8477d8c prev_states = 0x8477d40 folder = 0x842e540 \<PL_fold_latin1> "" data_slot = 0 trie_words = 0x0 re_trie_maxbuff = 0x8470600 re_debug_flags = 0 __PRETTY_FUNCTION__ = "S_make_trie"

**VALGRIND**

dcollins@​nagios​:\~/perl$ valgrind ./perl /usr/local/perl-afl/out/allcrash/f3i000002 ==16731== Memcheck\, a memory error detector ==16731== Copyright (C) 2002-2013\, and GNU GPL'd\, by Julian Seward et al. ==16731== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info ==16731== Command​: ./perl /usr/local/perl-afl/out/allcrash/f3i000002 ==16731== ==16731== ==16731== Process terminating with default action of signal 8 (SIGFPE) ==16731== Integer divide by zero at address 0x6356C3A2 ==16731== at 0x811963B​: S_make_trie (regcomp.c​:2775) ==16731== by 0x8123810​: S_study_chunk (regcomp.c​:4368) ==16731== by 0x813291F​: Perl_re_op_compile (regcomp.c​:7296) ==16731== by 0x8071182​: Perl_pmruntime (op.c​:5579) ==16731== by 0x8107384​: Perl_yyparse (perly.y​:1038) ==16731== by 0x809B04D​: S_parse_body (perl.c​:2304) ==16731== by 0x8099839​: perl_parse (perl.c​:1634) ==16731== by 0x8061251​: main (perlmain.c​:114) ==16731== ==16731== HEAP SUMMARY​: ==16731== in use at exit​: 108\,665 bytes in 658 blocks ==16731== total heap usage​: 752 allocs\, 94 frees\, 113\,684 bytes allocated ==16731== ==16731== LEAK SUMMARY​: ==16731== definitely lost​: 156 bytes in 1 blocks ==16731== indirectly lost​: 2\,593 bytes in 37 blocks ==16731== possibly lost​: 0 bytes in 0 blocks ==16731== still reachable​: 105\,916 bytes in 620 blocks ==16731== suppressed​: 0 bytes in 0 blocks ==16731== Rerun with --leak-check=full to see details of leaked memory ==16731== ==16731== For counts of detected and suppressed errors\, rerun with​: -v ==16731== ERROR SUMMARY​: 0 errors from 0 contexts (suppressed​: 0 from 0) Floating point exception

**BISECT**

75697d6e4ef98ece405210de48e7529d01b619bf is the first bad commit commit 75697d6e4ef98ece405210de48e7529d01b619bf Author​: Karl Williamson \khw@&#8203;cpan\.org Date​: Sun Dec 21 22​:02​:30 2014 -0700

  Empty \N{} in regex pattern should force /d to /u  
  \N{} is for Unicode names\, even if the name is actually omitted.   (Accepting an empty name is\, I believe\, an accident\, and now is   supported only for backwards compatibility.)

:100644 100644 e6ddb803aae9f37f00d2f0f38fd109f05bf0bcb6 c2521a97ccee8d15370822c3b975be2628fcddb2 M regcomp.c :040000 040000 218758eff11e9ad6989600b1a4a93ca23c3da3cf a6a3a35946820a1f6e9b6f99f414edc78099b63b M t bisect run success

**PERL -V**

dcollins@​nagios​:\~/perl$ ./perl -V Summary of my perl5 (revision 5 version 23 subversion 4) configuration​:   Commit id​: 7a36e618ad808bf649080137e3fb56386d8420e3   Platform​:   osname=linux\, osvers=2.6.32-5-686\, archname=i686-linux-64int-ld   uname='linux nagios 2.6.32-5-686 #1 smp tue may 13 16​:33​:32 utc 2014 i686 gnulinux '   config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitint -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=undef\, usemultiplicity=undef   use64bitint=define\, use64bitall=undef\, uselongdouble=define   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='ccache afl-gcc'\, ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,   optimize='-g'\,   cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'   ccversion=''\, gccversion='5.2.0'\, gccosandvers=''   intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=12345678\, doublekind=3   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12\, longdblkind=3   ivtype='long long'\, ivsize=8\, nvtype='long double'\, nvsize=12\, Off_t='off_t'\, lseeksize=8   alignbytes=4\, prototype=define   Linker and Libraries​:   ld='ccache afl-gcc'\, ldflags =' -fstack-protector-strong -L/usr/local/lib'   libpth=/usr/local/lib /usr/local/lib/gcc/i686-pc-linux-gnu/5.2.0/include-fixed /usr/lib /lib/../lib /usr/lib/../lib /lib /usr/lib/i486-linux-gnu /usr/lib64   libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc   perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc   libc=libc-2.22.so\, so=so\, useshrplib=false\, libperl=libperl.a   gnulibc_version='2.22'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E'   cccdlflags='-fPIC'\, lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:   Compile-time options​: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE   PERL_DONT_CREATE_GVSV   PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP   PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_INT   USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE   USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME   USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF   Built under linux   Compiled at Sep 25 2015 21​:59​:04   @​INC​:   /usr/local/perl-afl/lib/site_perl/5.23.4/i686-linux-64int-ld   /usr/local/perl-afl/lib/site_perl/5.23.4   /usr/local/perl-afl/lib/5.23.4/i686-linux-64int-ld   /usr/local/perl-afl/lib/5.23.4   /usr/local/perl-afl/lib/site_perl/5.23.3   /usr/local/perl-afl/lib/site_perl/5.23.2   /usr/local/perl-afl/lib/site_perl   .

p5pRT commented 9 years ago

From @dcollinsn

f3i000002

p5pRT commented 9 years ago

From @dcollinsn

A simpler testcase​:

/\N{}Ss|/i

** GDB **

Starting program​: /usr/local/perl-afl/bin/perl -e /\\N\{\}Ss\|/i [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGFPE\, Arithmetic exception. 0x00000000007497ce in S_make_trie (pRExC_state=0x7fffffffddc0\, startbranch=0x0\, first=0x0\, last=0x1e\, tail=0x3\, word_count=18933704\, flags=35\, depth=1) at regcomp.c​:2847 2847 const U32 laststate = TRIE_NODENUM( next_alloc ); (gdb) bt #0 0x00000000007497ce in S_make_trie (pRExC_state=0x7fffffffddc0\, startbranch=0x0\, first=0x0\, last=0x1e\, tail=0x3\, word_count=18933704\, flags=35\, depth=1) at regcomp.c​:2847 #1 0x0000000000769d6c in S_study_chunk (pRExC_state=0x7fffffffdce0\, scanp=0x0\, minlenp=0x0\, deltap=0x1e\, last=0x3\, data=0x120e7b0\, stopparen=-1\, recursed_depth=0\, flags=10240\, depth=0\, and_withp=\) at regcomp.c​:4368 #2 0x00000000007998f8 in Perl_re_op_compile (patternp=0x120eae0\, pat_count=0\, expr=0x120e7b0\, eng=0x1e\, old_re=0x7\, is_bare_re=0x7fffffffdd80\, orig_rx_flags=4\, pm_flags=4) at regcomp.c​:7296 #3 0x00000000004e8872 in Perl_pmruntime (o=0x120e978\, expr=0x120e938\, repl=0x0\, isreg=30\, floor=7938400) at op.c​:5572 #4 0x000000000066f2ed in Perl_yyparse (gramtype=18934136) at perly.y​:1038 #5 0x000000000053a8d5 in S_parse_body (env=env@​entry=0x0\, xsinit=xsinit@​entry=0x42c840 \<xs_init>) at perl.c​:2304 #6 0x0000000000542653 in perl_parse (my_perl=\\, xsinit=xsinit@​entry=0x42c840 \<xs_init>\, argc=\\, argv=\\, env=env@​entry=0x0) at perl.c​:1634 #7 0x000000000042c468 in main (argc=3\, argv=0x7fffffffe648\, env=0x7fffffffe668) at perlmain.c​:114 (gdb) info locals state = 18933704 trie = 0x120fdd0 len = 0 uvc = 0 next_alloc = 1 folder = 0xfbab40 \<PL_fold_latin1> "" re_debug_flags = 0 __PRETTY_FUNCTION__ = "S_make_trie"

** VALGRIND **

==15719== Memcheck\, a memory error detector ==15719== Copyright (C) 2002-2015\, and GNU GPL'd\, by Julian Seward et al. ==15719== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==15719== Command​: ../bin/perl -e /\\N{}Ss|/i ==15719== ==15719== ==15719== Process terminating with default action of signal 8 (SIGFPE) ==15719== Integer divide by zero at address 0x80418779B ==15719== at 0x7497BC​: S_make_trie (regcomp.c​:2847) ==15719== by 0x769D6B​: S_study_chunk.constprop.51 (regcomp.c​:4368) ==15719== by 0x7998F7​: Perl_re_op_compile (regcomp.c​:7296) ==15719== by 0x4E8871​: Perl_pmruntime (op.c​:5572) ==15719== by 0x66F2EC​: Perl_yyparse (perly.y​:1038) ==15719== by 0x53A8D4​: S_parse_body (perl.c​:2304) ==15719== by 0x542652​: perl_parse (perl.c​:1634) ==15719== by 0x42C467​: main (perlmain.c​:114) ==15719== ==15719== HEAP SUMMARY​: ==15719== in use at exit​: 122\,004 bytes in 573 blocks ==15719== total heap usage​: 662 allocs\, 89 frees\, 130\,332 bytes allocated ==15719== ==15719== LEAK SUMMARY​: ==15719== definitely lost​: 176 bytes in 1 blocks ==15719== indirectly lost​: 1\,993 bytes in 20 blocks ==15719== possibly lost​: 16 bytes in 1 blocks ==15719== still reachable​: 119\,819 bytes in 551 blocks ==15719== suppressed​: 0 bytes in 0 blocks ==15719== Rerun with --leak-check=full to see details of leaked memory ==15719== ==15719== For counts of detected and suppressed errors\, rerun with​: -v ==15719== ERROR SUMMARY​: 0 errors from 0 contexts (suppressed​: 0 from 0) Floating point exception

** PERL -V **

Summary of my perl5 (revision 5 version 23 subversion 4) configuration​:   Commit id​: 94757bf7ac8451b3588578f287f9c511a04011bf   Platform​:   osname=linux\, osvers=3.16.0-4-amd64\, archname=x86_64-linux-ld   uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux '   config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=undef\, usemultiplicity=undef   use64bitint=define\, use64bitall=define\, uselongdouble=define   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='ccache afl-gcc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,   optimize='-g'\,   cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'   ccversion=''\, gccversion='4.9.2'\, gccosandvers=''   intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3   ivtype='long'\, ivsize=8\, nvtype='long double'\, nvsize=16\, Off_t='off_t'\, lseeksize=8   alignbytes=16\, prototype=define   Linker and Libraries​:   ld='ccache afl-gcc'\, ldflags =' -fstack-protector-strong -L/usr/local/lib'   libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib   libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc   perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc   libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a   gnulibc_version='2.19'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E'   cccdlflags='-fPIC'\, lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:   Compile-time options​: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE   PERL_DONT_CREATE_GVSV   PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP   PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL   USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE   USE_LOCALE_COLLATE USE_LOCALE_CTYPE   USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE   USE_PERLIO USE_PERL_ATOF   Built under linux   Compiled at Sep 30 2015 16​:39​:59   @​INC​:   /usr/local/perl-afl/lib/site_perl/5.23.4/x86_64-linux-ld   /usr/local/perl-afl/lib/site_perl/5.23.4   /usr/local/perl-afl/lib/5.23.4/x86_64-linux-ld   /usr/local/perl-afl/lib/5.23.4   .

p5pRT commented 8 years ago

@khwilliamson - Status changed from 'new' to 'open'

p5pRT commented 8 years ago

From @demerphq

\N{} was creating an EMPTY node\, which was tickling a secondary issue with trie construction from alternations with EXACTFU_SS nodes in them.

commit 944e05e3d149b21eef2ceb8d255da390afb267a2 Author​: Yves Orton \demerphq@&#8203;gmail\.com Date​: Wed Mar 9 20​:00​:53 2016 +0100

  Fix Perl #126206​: handle NOTHING regops and EXACTFU_SS regops in make_trie() properly  
  ... and avoid dereffing non-EXACT nodes unnecessarily at the same   time. This fixes https://rt-archive.perl.org/perl5/Ticket/Display.html?id=126206

p5pRT commented 8 years ago

@demerphq - Status changed from 'open' to 'resolved'