Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.99k stars 559 forks source link

core dump in /...(?{ code }) #4673

Closed p5pRT closed 13 years ago

p5pRT commented 22 years ago

Migrated from rt.perl.org#8027 (status was 'resolved')

Searchable as RT8027$

p5pRT commented 22 years ago

From john@pc.xs4all.nl

Created by john@pc.xs4all.nl

This is a bug report for perl from john@​pc.xs4all.nl\, generated with the help of perlbug 1.33 running under perl v5.7.2.

---------------------------------------------------------------- In some circumstances\, perl dumps core after executing code embedded in a regular expression\, with (?{ code... }). The following program is an example of this.

The crash happens with perl 5.6.1\, perl 5.7.2\, and the latest bleadperl. The code doesn't compile with perl 5.005_03\, the (?{ code }) construct wasn't mature enough back then\, apparently.

I haven't really tried to minimize this. If this is a known issue\, I'd rather not waste too much time on it. However if someone is interested in fixing this\, I can try to reduce the example to a bare minimum.

The workaround is to move code out of the regex itself\, and only do a minimum amount of work inside the re. So I did manage to match the $sillytimestamp\, using a slightly modified version.

Here is the program​:

----8\<-BEGIN-PROGRAM------- #!/usr/bin/perl

use Time​::Local; use strict;

use re 'debug';

sub match {   my $str = shift;   $str =~ m{   ^   (\d|[1-5]\d) # seconds   (\d|[1-5]\d) # minutes   (\d|1\d|2[0-3]) # hours   ([1-9]|[12]\d|3[01]) # mday   (\d|1[01]) # mon   ([2-9]|[1-9]\d|1[0-2]\d|13[0-7]) # year   ([0-6]) # wday   (\d|[1-9]\d|[12]\d\d|3[0-5]\d|36[0-5]) # yday   ([01]) # isdst   $   (?{   print "Possible match​: $1-$2-$3-$4-$5-$6-$7-$8-$9\n";   my $t = timelocal($1\, $2\, $3\, $4\, $5\, 1900 + $6);   print "Possible date​: $t\n";   my @​lt = localtime($t);   print "Localtime is @​lt\n";   if ( $lt[6] == $7 && $lt[7] == $8 && $lt[8] == $9 ) {   print "MATCH! "\, scalar localtime $t\, "\n";   }   })   }x; }

my $sillytimestamp = "251013211010133240"; match($sillytimestamp); ----END-PROGRAM->8--------

This is (a portion of) the output\, when run under yesterday's bleadperl compiled with -DDEBUGGING.

----8\<-BEGIN-OUTPUT------- pc​:\~/perl/bleadperl $ ./perl -Ilib -w ../localtime_play/recrash Compiling REx `   ^   (\d|[1-5]\d) # seconds   (\d|[1-5]\d) # minutes   (\d|1\d|2[0-3]) # hours   ([1-9]|[12]\d|3[01]) # mday   (\d|1[01]) # mon   ([2-9]|[1-9]\d|1[0-2]\d|13[0-7]) # year   ([0-6]) # wday   (\d|[1-9]\d|[12]\d\d|3[0-5]\d|36[0-5]) # yday   ([01]) # isdst   $   (?{   print "Possible match​: $1-$2-$3-$4-$5-$6-$7-$8-$9\n";   my $t = timelocal($1\, $2\, $3\, $4\, $5\, 1900 + $6);   print "Possible date​: $t\n";   my @​lt = localtime($t);   print "Localtime is @​lt\n";   if ( $lt[6] == $7 && $lt[7] == $8 && $lt[8] == $9 ) {   print "MATCH! "\, scalar localtime $t\, "\n";   }   })   ' size 280 Got 2244 bytes for offset annotations. first at 2 rarest char at 0   1​: BOL(2) [...] EXECUTING... [...]   17 \<21101013324> \<0> |264​: ANYOF[01]   18 \<211010133240> \<> |275​: CLOSE9   18 \<211010133240> \<> |277​: EOL   18 \<211010133240> \<> |278​: EVAL   re_eval 0x81a6040 Possible match​: 25-10-13-21-10-101-3-324-0 Possible date​: 1006344625 Localtime is 25 10 13 21 10 101 3 324 0 MATCH! Wed Nov 21 13​:10​:25 2001   18 \<211010133240> \<> |280​: END Match successful! Segmentation fault (core dumped) ----END-OUTPUT->8--------

And here is what gdb says about the generated core file​:

pc​:\~/perl/bleadperl $ gdb perl core GNU gdb 19991004 Copyright 1998 Free Software Foundation\, Inc. GDB is free software\, covered by the GNU General Public License\, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux"... Core was generated by `./perl -Ilib -w ../localtime_play/recrash'. Program terminated with signal 11\, Segmentation fault. Reading symbols from /lib/libnsl.so.1...done. Reading symbols from /lib/libdl.so.2...done. Reading symbols from /lib/libm.so.6...done. Reading symbols from /lib/libc.so.6...done. Reading symbols from /lib/libcrypt.so.1...done. Reading symbols from /lib/libutil.so.1...done. Reading symbols from /lib/ld-linux.so.2...done. Reading symbols from /usr/home/john/perl/bleadperl/lib/auto/re/re.so...done. #0 0x80f3b9c in Perl_leave_scope (base=13) at scope.c​:825 825 if (SvREFCNT(sv) \<= 1 && !SvOBJECT(sv)) { (gdb) bt #0 0x80f3b9c in Perl_leave_scope (base=13) at scope.c​:825 #1 0x80c5eb7 in Perl_pp_match () at pp_hot.c​:1375 #2 0x80ac7f4 in Perl_runops_debug () at dump.c​:1388 #3 0x805ff89 in S_run_body (oldscope=1) at perl.c​:1595 #4 0x805fb6b in perl_run (my_perl=0x816ae68) at perl.c​:1516 #5 0x805c912 in main (argc=4\, argv=0xbffff804\, env=0xbffff818)   at perlmain.c​:76 (gdb) x sv 0x2​: Cannot access memory at address 0x2

Perl Info ``` Flags: category=core severity=medium Site configuration information for perl v5.7.2: Configured by john at Sun Dec 9 23:05:55 CET 2001. Summary of my perl5 (revision 5.0 version 7 subversion 2 patch 13539) configuration: Platform: osname=linux, osvers=2.2.19, archname=i686-linux uname='linux pc.xs4all.nl 2.2.19 #1 sun jun 24 15:06:07 cest 2001 i686 unknown ' config_args='-Doptimize=-g -Dusedevel -des' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=define Compiler: cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-g', cppflags='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include' ccversion='', gccversion='egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lndbm -lgdbm -ldb -ldl -lm -lc -lposix -lcrypt -lutil perllibs=-lnsl -ldl -lm -lc -lposix -lcrypt -lutil libc=/lib/libc-2.1.3.so, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Locally applied patches: DEVEL13539 @INC for perl v5.7.2: lib /usr/local/lib/perl5/5.7.2/i686-linux /usr/local/lib/perl5/5.7.2 /usr/local/lib/perl5/site_perl/5.7.2/i686-linux /usr/local/lib/perl5/site_perl/5.7.2 /usr/local/lib/perl5/site_perl . Environment for perl v5.7.2: HOME=/usr/home/john LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/home/john/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin:/usr/games:/usr/local/sbin:/usr/local/pgsql/bin PERL_BADLANG (unset) SHELL=/bin/bash ```
p5pRT commented 22 years ago

From @vanstyn

Jan-Pieter Cornet \john@&#8203;pc\.xs4all\.nl wrote​: :In some circumstances\, perl dumps core after executing code embedded :in a regular expression\, with (?{ code... }).

This is sorta known. I just dug some more\, and it turns out that this occurs because we switch curpads when performing the eval block (in regmatch()\, case EVAL)\, but because the eval isn't protected by an ENTER/LEAVE we've left stuff on the stack referring to the wrong curpad. Using a shorter example\, I did this​:

  (gdb) set args -we 'our$t="";""=~/(?{print "\<$t>\n";my@​lt=()})/'   (gdb) break Perl_leave_scope   Breakpoint 1 at 0x80db059​: file scope.c\, line 634.   (gdb) cond 1 base == 3   (gdb) run   \<>

  Breakpoint 1\, Perl_leave_scope (base=3) at scope.c​:634   634 if (base \< -1)   (gdb) p ((void**)PL_savestack)[0]@​PL_savestack_ix   $1 = {0xffffffff\, 0x8133b38\, 0x4\, 0x0\, 0x8138cdc\, 0x6\, 0xffffffff\, 0x8133b38\,   0x4\, 0x8137380\, 0x813cd48\, 0x8\, 0x8102980\, 0x0\, 0x1e\, 0x5\, 0x12\, 0x81373bc\,   0x81373c8\, 0x1}   (gdb)

... in which the \<\< 0x5\, 0x12 >> is a SAVEt_CLEARSV of curpad[5]. By the time we finally LEAVE_SCOPE at the end of pp_match()\, the curpad is restored to something which doesn't point to an SV at [5].

Ilya strongly objected to an ENTER/LEAVE in re_eval processing on speed grounds\, but there is another problem​: we need localised variables to retain their value for later in the regexp. This is demonstrated by the regexp test #469​:   (?{$a=2})a*aa(?{local$a=$a+1})k*c(?{$b=$a}) yaaxxaaaacd y $b 3 ... which expects $a == 3 in the third eval block\, and indeed adding an ENTER/LEAVE pair causes re_tests #469 and #470 to fail\, and no other.

Now\, the curpad shenanigans here have always been a mess\, and I don't know of an easy solution. It may be that we can fix the curpad once at the start of any regexp that includes an eval\, and do a single ENTER/LEAVE for the entire regexp before restoring; I'm not quite sure what's involved with that - Sarathy might have more answers.

That does suggest you have a better chance if you avoid using pad variables within the eval\, and indeed if I move the my() declaration of $t and @​lt from your original test case out to the enclosing sub it no longer cores. I don't know how much of a guarantee that is ...

Hugo

p5pRT commented 15 years ago

From @cpansprout

This may be another manifestation of bug #8027.

I have observed this in 5.8.8/9\, 5.10.0 and 5.11.0@​34471. The
following program prints ‘undef’\, when it should print ‘self’​:

#!/usr/local/bin/perl5.10.0

use warnings; use strict;

#my $foo; my $self = "self";

"i" =~ /(?{ my @​m })/;

warn "Should be self​: "\, $self // "undef";

It seems that the ‘my @​m’ in the regular expression is overwriting
$self. If I uncomment ‘my $foo’\, then $self is left untouched.

I have two workarounds​: (?{{ ... }}) and (?{do{{ ... }}}). I’m not
sure why those work\, as (?{do{...}}) and {"i" =~ /(?{...})/} do not.


Flags​:   category=core   severity=medium


This perlbug was built using Perl v5.8.8 - Tue Jan 13 10​:01​:03 PST 2009 It is being executed now by Perl v5.10.0 - Sun Dec 23 12​:08​:07 PST
2007.

Site configuration information for perl v5.10.0​:

Configured by sprout at Sun Dec 23 12​:08​:07 PST 2007.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration​:   Platform​:   osname=darwin\, osvers=9.0.0\, archname=darwin-2level   uname='darwin pint.local 9.0.0 darwin kernel version 9.0.0​: tue
oct 9 21​:35​:55 pdt 2007; root​:xnu-1228~1release_i386 i386 '   config_args='-de'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=undef\, usemultiplicity=undef   useperlio=define\, d_sfio=undef\, uselargefiles=define\,
usesocks=undef   use64bitint=undef\, use64bitall=undef\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-fno-common -DPERL_DARWIN -no-cpp-precomp -fno- strict-aliasing -pipe -I/usr/local/include'\,   optimize='-O3'\,   cppflags='-no-cpp-precomp -fno-common -DPERL_DARWIN -no-cpp- precomp -fno-strict-aliasing -pipe -I/usr/local/include'   ccversion=''\, gccversion='4.0.1 (Apple Inc. build 5465)'\,
gccosandvers=''   intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16   ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\,
Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'\, ldflags =' -L/usr/ local/lib'   libpth=/usr/local/lib /usr/lib   libs=-ldbm -ldl -lm -lutil -lc   perllibs=-ldl -lm -lutil -lc   libc=/usr/lib/libc.dylib\, so=dylib\, useshrplib=false\,
libperl=libperl.a   gnulibc_version=''   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=bundle\, d_dlsymun=undef\, ccdlflags=' '   cccdlflags=' '\, lddlflags=' -bundle -undefined dynamic_lookup -L/ usr/local/lib'

Locally applied patches​:   fix for regcomp CVE-2007-5116 security vulnerability   fix for double-free CVE-2008-1927 security vulnerability


@​INC for perl v5.10.0​:   /usr/local/lib/perl5/5.10.0/darwin-2level   /usr/local/lib/perl5/5.10.0   /usr/local/lib/perl5/site_perl/5.10.0/darwin-2level   /usr/local/lib/perl5/site_perl/5.10.0   /usr/local/lib/perl5/site_perl/5.8.6   /usr/local/lib/perl5/site_perl   .


Environment for perl v5.10.0​:   DYLD_LIBRARY_PATH (unset)   HOME=/Users/sprout   LANG=en_US.UTF-8   LANGUAGE (unset)   LD_LIBRARY_PATH (unset)   LOGDIR (unset)   PATH=/usr/bin​:/bin​:/usr/sbin​:/sbin​:/usr/local/bin​:/usr/X11/bin​:/ usr/local/bin   PERL_BADLANG (unset)   SHELL=/bin/bash

p5pRT commented 15 years ago

From p5p@spam.wizbit.be

On Sun Apr 26 12​:37​:42 2009\, sprout@​cpan.org wrote​:

This may be another manifestation of bug #8027.

I have observed this in 5.8.8/9\, 5.10.0 and 5.11.0@​34471. The
following program prints ‘undef’\, when it should print ‘self’​:

#!/usr/local/bin/perl5.10.0

use warnings; use strict;

#my $foo; my $self = "self";

"i" =~ /(?{ my @​m })/;

warn "Should be self​: "\, $self // "undef";

It seems that the ‘my @​m’ in the regular expression is overwriting
$self. If I uncomment ‘my $foo’\, then $self is left untouched.

I have two workarounds​: (?{{ ... }}) and (?{do{{ ... }}}). I’m not
sure why those work\, as (?{do{...}}) and {"i" =~ /(?{...})/} do not.

This is indeed the same bug as #8027 except that #8027 core dumps where your test case corrups memory (basically the same thing). (So I'm merging the two tickets)

Unfortunally there does not appear to be an easy fix :( (As a side note​: this behaviour goes back to at least perl-5.00504)

Best regards\,

Bram

p5pRT commented 15 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 13 years ago

From @rspier

manually unmerged with 38639

p5pRT commented 13 years ago

From @cpansprout

This was fixed by commit 9e103e269.

p5pRT commented 13 years ago

@cpansprout - Status changed from 'open' to 'resolved'