search

Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.94k stars 554 forks source link

Perl debugger runs out of memory, hangs or segfaults on XML::Parser::Lite #9753

Closed p5pRT closed 15 years ago

p5pRT commented 15 years ago

Migrated from rt.perl.org#66110 (status was 'resolved')

Searchable as RT66110$

p5pRT commented 15 years ago

From @nwc10

Avar mailed p5p in 51dd1af80807190107h30b8626ct6d4d0a825abe4b3b@​mail.gmail.com http​://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-07/msg00382.html

perl 5.10 and blead will do various combinations of running of of memory\, hanging or segfaulting when running on a program using XML​::Parser​::Lite\, attached is a stripped down version of X​::P​::L which demonstrates the problem​:

Dave notes​:

possibly a 5.10.0 regression

p5pRT commented 15 years ago

From p5p@spam.wizbit.be

On Thu May 28 08​:14​:52 2009\, nicholas wrote​:

Avar mailed p5p in 51dd1af80807190107h30b8626ct6d4d0a825abe4b3b@​mail.gmail.com http​://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008- 07/msg00382.html

perl 5.10 and blead will do various combinations of running of of memory\, hanging or segfaulting when running on a program using XML​::Parser​::Lite\, attached is a stripped down version of X​::P​::L which demonstrates the problem​:

Dave notes​:

possibly a 5.10.0 regression

Change 1​: Leaking memory when run interactive in the debugger. When the PERLDB_OPTS is set to NonStop=1 then it does not seem to leak

----Program---- #!/usr/bin/perl -l

$ENV{PERL_SIGNALS} = "unsafe";

my $parent = $$;

my $pid = fork();

if ($pid) {   exec("$^X -d /tmp/rt-66110.pl"); } else {   sleep 3;   system("kill -9 $parent");   exit 0; }

----Output of .../pXYMq0N/perl-5.9.1@​23371/bin/perl----

----EOF ($?='0')---- ----Output of .../pFt3hxR/perl-5.9.1@​23372/bin/perl----

----EOF ($?='9')---- Need a perl between 23371 and 23372

http​://public.activestate.com/cgi-bin/perlbrowse/p/23372 Change 23372 by rgs@​valis on 2004/10/15 14​:09​:54

  Implement a new -dt command-line flag\, to enable threads under the   debugger (bug #31666).  
  Subject​: RE​: [PATCH] debugger handles threads [perl #31666]   From​: \richard\.foley@​ubs\.com   Date​: Wed\, 13 Oct 2004 13​:01​:18 +0200   Message-ID​: \B374141B0A424D4F9CF143CC51B3ADD903FB9E12@​NZURC900PEX1\.ubsgs\.ubsgroup\.net  
  Subject​: Re​: [PATCH] debugger handles threads [perl #31666]   From​: Yitzchak Scott-Thoennes \sthoenna@​efn\.org   Date​: Wed\, 13 Oct 2004 02​:49​:58 -0700   Message-ID​: \20041013094957\.GA17184@​efn\.org

$ perl-p-5.9.1\@​23371/perl -d /tmp/rt-66110.pl

Loading DB routines from perl5db.pl version 1.27 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(/tmp/rt-66110.pl​:71)​: setHandlers();   DB\<1> r Debugged program terminated. Use q to quit or R to restart\,   use O inhibit_exit to avoid stopping after program termination\,   h q\, h R or h O to get additional info.   DB\<1> q

$ perl-p-5.9.1\@​23372/perl -d /tmp/rt-66110.pl

Loading DB routines from perl5db.pl version 1.28 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(/tmp/rt-66110.pl​:71)​: setHandlers();   DB\<1> r

=> Keeps running (100% CPU) and starts leaking memory

(Small note​: /tmp/rt-66110.pl does not contain a shebang line)

Change 2​: Segmentation fault (but not always)

(Small note​: /tmp/rt-66110.pl contains the shebang #!/usr/bin/perl -d)

$ PERLDB_OPTS="NonStop=1" perl-p-5.9.2@​26537/perl /tmp/rt-66110.pl # returns immediatly $ perl-p-5.9.2@​26537/perl /tmp/rt-66110.pl # loops forever

$ PERLDB_OPTS="NonStop=1" perl-p-5.9.2@​26539/perl /tmp/rt-66110.pl Segmentation fault $ perl-p-5.9.2@​26539/perl /tmp/rt-66110.pl Segmentation fault

http​://public.activestate.com/cgi-bin/perlbrowse/p/26538 Change 26538 by nicholas@​entropy on 2005/12/30 14​:57​:00

  In the shared string table\, if we add new entries to the head of the   linked list chain\, then we don't need as much bookkeeping in the loop.

Not sure if this last change is really the cause of the segmentation fault or just a side effect.. the segfault seems very sensitive to changes in the script so it probably isn't...

Best regards\,

Bram

p5pRT commented 15 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 15 years ago

From @craigberry

On Thu\, May 28\, 2009 at 10​:14 AM\, Nicholas Clark \perlbug\-followup@&#8203;perl\.org wrote​:

# New Ticket Created by  Nicholas Clark # Please include the string​:  [perl #66110] # in the subject line of all future correspondence about this issue. # \<URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=66110 >

Avar mailed p5p in 51dd1af80807190107h30b8626ct6d4d0a825abe4b3b@​mail.gmail.com http​://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-07/msg00382.html

perl 5.10 and blead will do various combinations of running of of memory\, hanging or segfaulting when running on a program using XML​::Parser​::Lite\, attached is a stripped down version of X​::P​::L which demonstrates the problem​:

Dave notes​:

possibly a 5.10.0 regression

In Perl_regexec_flags (called from Perl_pp_match)\, after the got_it​: label\, we end up calling savepvn repeatedly with the same large value for the length. The value of i when we hit the code below gets rather large values\, like 3\,938\,416 or 3\,943\,248 (it seems to keep growing).

  if (flags & REXEC_COPY_STR) {   const I32 i = PL_regeol - startpos + (stringarg - strbeg);

\< ifdef snipped>   {   RX_MATCH_COPIED_on(rx);   s = savepvn(strbeg\, i);   prog->subbeg = s;   }   prog->sublen = i;   }

Some values of interest\, including those that make up i\, are as follows​:

REGEXEC\Perl_regexec_flags\my_perl->Ireg_state.re_state_regeol​: 15380736 REGEXEC\Perl_regexec_flags\startpos​: 11437488 REGEXEC\Perl_regexec_flags\stringarg​: 11437488 REGEXEC\Perl_regexec_flags\strbeg​: 11437488 REGEXEC\Perl_regexec_flags\strend​: 11437502

The string we are currently matching is​:

*PP_HOT\Perl_pp_match\s​: "\bar\"

To me that looks just a tad less than 3.9 million bytes :-)\, but it is in fact the string that ends at the current value of strend. The string that ends at the current value of PL_regeol (aka my_perl->Ireg_state.re_state_regeol) is "CODE(0xb18940)". If I set a watchpoint for PL_regeol\, it keeps toggling back and forth between the ends of these two different strings. So it really looks as though two different regex operations are going on at once in interleaved fashion and keep hijacking the value of PL_regeol from each other.

I think that's about as far as I'm going to get with this but thought I'd pass along my observations.

p5pRT commented 15 years ago

From @craigberry

On Sat\, May 30\, 2009 at 9​:39 AM\, Craig A. Berry \craig\.a\.berry@&#8203;gmail\.com wrote​:

On Thu\, May 28\, 2009 at 10​:14 AM\, Nicholas Clark \perlbug\-followup@&#8203;perl\.org wrote​:

# New Ticket Created by  Nicholas Clark # Please include the string​:  [perl #66110] # in the subject line of all future correspondence about this issue. # \<URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=66110 >

Avar mailed p5p in 51dd1af80807190107h30b8626ct6d4d0a825abe4b3b@​mail.gmail.com http​://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-07/msg00382.html

perl 5.10 and blead will do various combinations of running of of memory\, hanging or segfaulting when running on a program using XML​::Parser​::Lite\, attached is a stripped down version of X​::P​::L which demonstrates the problem​:

Dave notes​:

possibly a 5.10.0 regression

In Perl_regexec_flags (called from Perl_pp_match)\, after the got_it​: label\, we end up calling savepvn repeatedly with the same large value for the length.  The value of i when we hit the code below gets rather large values\, like 3\,938\,416 or 3\,943\,248 (it seems to keep growing).

       if (flags & REXEC_COPY_STR) {            const I32 i = PL_regeol - startpos + (stringarg - strbeg);

\< ifdef snipped>            {                RX_MATCH_COPIED_on(rx);                s = savepvn(strbeg\, i);                prog->subbeg = s;            }            prog->sublen = i;        }

Some values of interest\, including those that make up i\, are as follows​:

REGEXEC\Perl_regexec_flags\my_perl->Ireg_state.re_state_regeol​: 15380736 REGEXEC\Perl_regexec_flags\startpos​:    11437488 REGEXEC\Perl_regexec_flags\stringarg​:   11437488 REGEXEC\Perl_regexec_flags\strbeg​:      11437488 REGEXEC\Perl_regexec_flags\strend​:      11437502

The string we are currently matching is​:

*PP_HOT\Perl_pp_match\s​:        "\bar\"

To me that looks just a tad less than 3.9 million bytes :-)\, but it is in fact the string that ends at the current value of strend.  The string that ends at the current value of PL_regeol (aka my_perl->Ireg_state.re_state_regeol) is "CODE(0xb18940)".  If I set a watchpoint for PL_regeol\, it keeps toggling back and forth between the ends of these two different strings.  So it really looks as though two different regex operations are going on at once in interleaved fashion and keep hijacking the value of PL_regeol from each other.

I think that's about as far as I'm going to get with this but thought I'd pass along my observations.

Curiosity kept me going a bit farther\, so here's some more analysis with blead. Still nowhere near a solution. Here is what the call stack looks like when the value of PL_regeol gets hijacked​:

DBG> sh calls module name routine name line rel PC abs PC *REGEXEC Perl_re_intuit_start   85298 0000000000002001 000000000049F821 *PP_HOT Perl_pp_match   84290 0000000000010360 0000000000346890 *DUMP Perl_runops_debug   93949 0000000000014190 00000000002B9A50 *REGEXEC S_regmatch 88644 000000000002CA40 00000000004CA260 *REGEXEC S_regtry 87270 000000000001CE00 00000000004BA620 *REGEXEC Perl_regexec_flags   87069 0000000000019660 00000000004B6E80 *PP_HOT Perl_pp_match   84303 0000000000010A60 0000000000346F90 *DUMP Perl_runops_debug   93949 0000000000014190 00000000002B9A50 *PERL S_run_body 85382 000000000000B3B0 00000000000730C0 *PERL perl_run 85313 000000000000A930 0000000000072640

The noteworthy bit is that Perl_pp_match appears twice\, so we are doing a regex op inside of a regex op. In S_regmatch\, we are in a section that starts with​:

  case EVAL​: /* /(?{A})B/ /(??{A})B/ and /(?(?{A})X|Y)B/ */

and does​:

  CALLRUNOPS(aTHX); /* Scalar context. */

which in turn initiates another match operation. So we have inner and an outer regex matches going on at once\, but they share global variables like PL_regeol and so stomp on each other.

It may not be a general case of a problem with evals in regexen\, but the particular extreme to which evals are taken in the test case. If the one-line sub declaration in the test case is moved outside of an eval as below\, everything looks peachy.

Inline Patch ```diff --- xpl-testcase.pl;-0 2009-05-28 06:59:18 -0500 +++ xpl-testcase.pl 2009-05-31 10:29:23 -0500 @@ -61,7 +61,7 @@ sub regexp { sub compile { local $^W; # try regexp as it should be, apply patch if doesn't work foreach (regexp(), regexp('??')) { - eval qq{sub parse_re { use re "eval"; 1 while \$_[0] =~ m{$_}go ```

}; 1} or die; + sub parse_re { use re "eval"; 1 while \$_[0] =~ m{$_}go }   last if eval { parse_re('\bar\'); 1 }   };

[end]

I don't know why that sub declaration was inside an eval in the first place.

p5pRT commented 15 years ago

From p5p@spam.wizbit.be

A trimmed down version attached.

When you run it you may or may not get an 'Out of memory error!'. In some of my tests adding or removing a newline was enough to make the error go away. (It may also give a segfault. Not seen with this test but with ealier version I did get it )

Summary​: running the test seems to depend on​: - the version of perl\, (- the configuration of perl\,) - the user running it\, - the filename\, - the command used in the debugger\,

Test 1​: perl-5.10.0\, file​: test.pl​: user 'perl' and using 'next' in the debugger -> error Test 2​: perl-5.10.0\, file​: test.pl​: user 'perl' and using 'run' in the debugger -> error Test 3​: perl-5.10.0\, file​: test.pl​: user 'perl' and running with NonStop=1 -> no error

Test 4​: perl-5.10.0\, file​: test.pl​: user 'bram' and using 'next' in the debugger -> no error Test 5​: perl-5.10.0\, file​: test.pl​: user 'bram' and using 'run' in the debugger -> no error Test 6​: perl-5.10.0\, file​: test.pl​: user 'bram' and running with NonStop=1 -> error

Test 7​: blead\, file​: test.pl​: user 'perl' and using 'next' in the debugger -> no error Test 8​: blead\, file​: test.pl​: user 'perl' and using 'run' in the debugger -> no error Test 9​: blead\, file​: test.pl​: user 'perl' and running with NonStop=1 -> no error Test 10​: blead\, file​: test.pl​: user 'bram' and using 'next' in the debugger -> no error Test 11​: blead\, file​: test.pl​: user 'bram' and using 'run' in the debugger -> no error Test 12​: blead\, file​: test.pl​: user 'bram' and running with NonStop=1 -

no error

Test 13​: blead\, file​: test_15.pl​: user 'perl' and using 'next' in the debugger -> error Test 14​: blead\, file​: test_15.pl​: user 'perl' and using 'run' in the debugger -> no error Test 15​: blead\, file​: test_15.pl​: user 'perl' and running with NonStop=1 -> no error Test 16​: blead\, file​: test_15.pl​: user 'bram' and using 'next' in the debugger -> error Test 17​: blead\, file​: test_15.pl​: user 'bram' and using 'run' in the debugger -> no error Test 18​: blead\, file​: test_15.pl​: user 'bram' and running with NonStop=1 -> no error

(Details are at end)

Valgrind (from Vincent)​: $ ~/utils/valgrind/bin/valgrind perl5.10.0-dbg-thr -d z.pl ==20460== Memcheck\, a memory error detector. ==20460== Copyright (C) 2002-2009\, and GNU GPL'd\, by Julian Seward et al. ==20460== Using LibVEX rev 1900M\, a library for dynamic binary translation. ==20460== Copyright (C) 2004-2009\, and GNU GPL'd\, by OpenWorks LLP. ==20460== Using valgrind-3.5.0.SVN\, a dynamic binary instrumentation framework. ==20460== Copyright (C) 2000-2009\, and GNU GPL'd\, by Julian Seward et al. ==20460== For more details\, rerun with​: -v ==20460==

Loading DB routines from perl5db.pl version 1.30000000000000004 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(z.pl​:1)​: *c = sub {};   DB\<1> n main​::(z.pl​:2)​: '' =~ m/(?{ c() })/;   DB\<1> n main​::((reeval 4)[z.pl​:2]​:1)​:   DB\<1> n ==20460== Invalid read of size 1 ==20460== at 0x4C22660​: memcpy (mc_replace_strmem.c​:482) ==20460== by 0x478DE2​: Perl_savepvn (util.c​:944) ==20460== by 0x55403B​: Perl_regexec_flags (regexec.c​:2174) ==20460== by 0x4AE129​: Perl_pp_match (pp_hot.c​:1330) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x4A0FE0​: S_run_body (perl.c​:2379) ==20460== by 0x4A090A​: perl_run (perl.c​:2302) ==20460== by 0x421735​: main (perlmain.c​:113) ==20460== Address 0x6c2c457 is 55 bytes inside a block of size 57 free'd ==20460== at 0x4C20DA0​: free (vg_replace_malloc.c​:323) ==20460== by 0x477781​: Perl_safesysfree (util.c​:250) ==20460== by 0x504CDA​: Perl_leave_scope (scope.c​:767) ==20460== by 0x4AEC4B​: Perl_pp_match (pp_hot.c​:1506) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x55AFF4​: S_regmatch (regexec.c​:3687) ==20460== by 0x554AF3​: S_regtry (regexec.c​:2325) ==20460== by 0x553EB5​: Perl_regexec_flags (regexec.c​:2137) ==20460== by 0x4AE129​: Perl_pp_match (pp_hot.c​:1330) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x4A0FE0​: S_run_body (perl.c​:2379) ==20460== by 0x4A090A​: perl_run (perl.c​:2302) ==20460== ==20460== Invalid read of size 1 ==20460== at 0x4C22668​: memcpy (mc_replace_strmem.c​:482) ==20460== by 0x478DE2​: Perl_savepvn (util.c​:944) ==20460== by 0x55403B​: Perl_regexec_flags (regexec.c​:2174) ==20460== by 0x4AE129​: Perl_pp_match (pp_hot.c​:1330) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x4A0FE0​: S_run_body (perl.c​:2379) ==20460== by 0x4A090A​: perl_run (perl.c​:2302) ==20460== by 0x421735​: main (perlmain.c​:113) ==20460== Address 0x6c2c456 is 54 bytes inside a block of size 57 free'd ==20460== at 0x4C20DA0​: free (vg_replace_malloc.c​:323) ==20460== by 0x477781​: Perl_safesysfree (util.c​:250) ==20460== by 0x504CDA​: Perl_leave_scope (scope.c​:767) ==20460== by 0x4AEC4B​: Perl_pp_match (pp_hot.c​:1506) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x55AFF4​: S_regmatch (regexec.c​:3687) ==20460== by 0x554AF3​: S_regtry (regexec.c​:2325) ==20460== by 0x553EB5​: Perl_regexec_flags (regexec.c​:2137) ==20460== by 0x4AE129​: Perl_pp_match (pp_hot.c​:1330) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x4A0FE0​: S_run_body (perl.c​:2379) ==20460== by 0x4A090A​: perl_run (perl.c​:2302) ==20460== ==20460== Invalid read of size 1 ==20460== at 0x4C22672​: memcpy (mc_replace_strmem.c​:482) ==20460== by 0x478DE2​: Perl_savepvn (util.c​:944) ==20460== by 0x55403B​: Perl_regexec_flags (regexec.c​:2174) ==20460== by 0x4AE129​: Perl_pp_match (pp_hot.c​:1330) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x4A0FE0​: S_run_body (perl.c​:2379) ==20460== by 0x4A090A​: perl_run (perl.c​:2302) ==20460== by 0x421735​: main (perlmain.c​:113) ==20460== Address 0x6c2c455 is 53 bytes inside a block of size 57 free'd ==20460== at 0x4C20DA0​: free (vg_replace_malloc.c​:323) ==20460== by 0x477781​: Perl_safesysfree (util.c​:250) ==20460== by 0x504CDA​: Perl_leave_scope (scope.c​:767) ==20460== by 0x4AEC4B​: Perl_pp_match (pp_hot.c​:1506) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x55AFF4​: S_regmatch (regexec.c​:3687) ==20460== by 0x554AF3​: S_regtry (regexec.c​:2325) ==20460== by 0x553EB5​: Perl_regexec_flags (regexec.c​:2137) ==20460== by 0x4AE129​: Perl_pp_match (pp_hot.c​:1330) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x4A0FE0​: S_run_body (perl.c​:2379) ==20460== by 0x4A090A​: perl_run (perl.c​:2302) ==20460== ==20460== Invalid read of size 1 ==20460== at 0x4C2267C​: memcpy (mc_replace_strmem.c​:482) ==20460== by 0x478DE2​: Perl_savepvn (util.c​:944) ==20460== by 0x55403B​: Perl_regexec_flags (regexec.c​:2174) ==20460== by 0x4AE129​: Perl_pp_match (pp_hot.c​:1330) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x4A0FE0​: S_run_body (perl.c​:2379) ==20460== by 0x4A090A​: perl_run (perl.c​:2302) ==20460== by 0x421735​: main (perlmain.c​:113) ==20460== Address 0x6c2c454 is 52 bytes inside a block of size 57 free'd ==20460== at 0x4C20DA0​: free (vg_replace_malloc.c​:323) ==20460== by 0x477781​: Perl_safesysfree (util.c​:250) ==20460== by 0x504CDA​: Perl_leave_scope (scope.c​:767) ==20460== by 0x4AEC4B​: Perl_pp_match (pp_hot.c​:1506) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x55AFF4​: S_regmatch (regexec.c​:3687) ==20460== by 0x554AF3​: S_regtry (regexec.c​:2325) ==20460== by 0x553EB5​: Perl_regexec_flags (regexec.c​:2137) ==20460== by 0x4AE129​: Perl_pp_match (pp_hot.c​:1330) ==20460== by 0x476A10​: Perl_runops_debug (dump.c​:1931) ==20460== by 0x4A0FE0​: S_run_body (perl.c​:2379) ==20460== by 0x4A090A​: perl_run (perl.c​:2302) Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.
  DB\<1>

The details​:

perl -V used in the following tests​: Summary of my perl5 (revision 5 version 10 subversion 0) configuration​:   Platform​:   osname=linux\, osvers=2.4.33.3\, archname=i686-linux   uname='linux wizbit 2.4.33.3 #1 sun jan 21 11​:41​:03 cet 2007 i686 pentium3 i386 gnulinux '   config_args='-Dprefix=/opt/perl/5.10.0 -Dusedevel=y -de'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=undef\, usemultiplicity=undef   useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef   use64bitint=undef\, use64bitall=undef\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include - D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,   optimize='-O2'\,   cppflags='-fno-strict-aliasing -pipe -I/usr/local/include'   ccversion=''\, gccversion='3.4.6'\, gccosandvers=''   intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12   ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=4\, prototype=define   Linker and Libraries​:   ld='cc'\, ldflags =' -L/usr/local/lib'   libpth=/usr/local/lib /lib /usr/lib   libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc   perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc   libc=/lib/libc-2.3.6.so\, so=so\, useshrplib=false\, libperl=libperl.a   gnulibc_version='2.3.6'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E'   cccdlflags='-fPIC'\, lddlflags='-shared -O2 -L/usr/local/lib'

Characteristics of this binary (from libperl)​:   Compile-time options​: PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP   USE_LARGE_FILES USE_PERLIO   Built under linux   Compiled at Feb 27 2008 15​:59​:27   @​INC​:   /opt/perl/5.10.0/lib/5.10.0/i686-linux   /opt/perl/5.10.0/lib/5.10.0   /opt/perl/5.10.0/lib/site_perl/5.10.0/i686-linux   /opt/perl/5.10.0/lib/site_perl/5.10.0   .

Test 1​: perl-5.10.0\, file​: test.pl​: user 'perl' and using 'next' in the debugger -> error perl@​wizbit​:/tmp/RT-66110$ /opt/perl/bin/perl5100 -d test.pl

Loading DB routines from perl5db.pl version 1.3 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test.pl​:1)​: *c = sub {};   DB\<1> n main​::(test.pl​:2)​: '' =~ m/(?{ c() })/;   DB\<1> n main​::((reeval 4)[test.pl​:2]​:1)​:   DB\<1> n Out of memory! Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q *** glibc detected *** corrupted double-linked list​: 0xb7edd878 *** Aborted perl@​wizbit​:/tmp/RT- 66110$

Test 2​: perl-5.10.0\, file​: test.pl​: user 'perl' and using 'run' in the debugger -> error perl@​wizbit​:/tmp/RT-66110$ /opt/perl/bin/perl5100 -d test.pl

Loading DB routines from perl5db.pl version 1.3 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test.pl​:1)​: *c = sub {};   DB\<1> r Out of memory! Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q *** glibc detected *** corrupted double-linked list​: 0xb7ea7878 *** Aborted perl@​wizbit​:/tmp/RT- 66110$

Test 3​: perl-5.10.0\, file​: test.pl​: user 'perl' and running with NonStop=1 -> no error perl@​wizbit​:/tmp/RT-66110$ PERLDB_OPTS="NonStop=1" /opt/perl/bin/ perl5100 -d test.pl perl@​wizbit​:/tmp/RT- 66110$

Test 4​: perl-5.10.0\, file​: test.pl​: user 'bram' and using 'next' in the debugger -> no error

Loading DB routines from perl5db.pl version 1.3 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test.pl​:1)​: *c = sub {};   DB\<1> n main​::(test.pl​:2)​: '' =~ m/(?{ c() })/;   DB\<1> n main​::((reeval 4)[test.pl​:2]​:1)​:   DB\<1> n Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q bram@​wizbit​:/tmp/RT-66110$

Test 5​: perl-5.10.0\, file​: test.pl​: user 'bram' and using 'run' in the debugger -> no error

Loading DB routines from perl5db.pl version 1.3 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test.pl​:1)​: *c = sub {};   DB\<1> r Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q bram@​wizbit​:/tmp/RT-66110$

Test 6​: perl-5.10.0\, file​: test.pl​: user 'bram' and running with NonStop=1 -> error bram@​wizbit​:/tmp/RT-66110$ PERLDB_OPTS="NonStop=1" /opt/perl/bin/ perl5100 -d test.pl Out of memory! bram@​wizbit​:/tmp/RT-66110$

Test 7​: blead\, file​: test.pl​: user 'perl' and using 'next' in the debugger -> no error perl@​wizbit​:/tmp/RT-66110$ /opt/perl/2009-06-18/perl -I /opt/perl/2009- 06-18/lib -d test.pl

Loading DB routines from perl5db.pl version 1.33 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test.pl​:1)​: *c = sub {};   DB\<1> n main​::(test.pl​:2)​: '' =~ m/(?{ c() })/;   DB\<1> n main​::((reeval 4)[test.pl​:2]​:1)​:   DB\<1> n Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q perl@​wizbit​:/tmp/RT-66110$

Test 8​: blead\, file​: test.pl​: user 'perl' and using 'run' in the debugger -> no error perl@​wizbit​:/tmp/RT-66110$ /opt/perl/2009-06-18/perl -I /opt/perl/2009- 06-18/lib -d test.pl

Loading DB routines from perl5db.pl version 1.33 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test.pl​:1)​: *c = sub {};   DB\<1> r Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q perl@​wizbit​:/tmp/RT-66110$

Test 9​: blead\, file​: test.pl​: user 'perl' and running with NonStop=1 -> no error perl@​wizbit​:/tmp/RT-66110$ PERLDB_OPTS="NonStop=1" /opt/perl/2009-06-18/ perl -I /opt/perl/2009-06-18/lib -d test.pl perl@​wizbit​:/tmp/RT-66110$

Test 10​: blead\, file​: test.pl​: user 'bram' and using 'next' in the debugger -> no error bram@​wizbit​:/tmp/RT-66110$ /opt/perl/2009-06-18/perl -I /opt/perl/2009- 06-18/lib -d test.pl

Loading DB routines from perl5db.pl version 1.33 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test.pl​:1)​: *c = sub {};   DB\<1> n main​::(test.pl​:2)​: '' =~ m/(?{ c() })/;   DB\<1> n main​::((reeval 4)[test.pl​:2]​:1)​:   DB\<1> n Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q bram@​wizbit​:/tmp/RT-66110$

Test 11​: blead\, file​: test.pl​: user 'bram' and using 'run' in the debugger -> no error bram@​wizbit​:/tmp/RT-66110$ /opt/perl/2009-06-18/perl -I /opt/perl/2009- 06-18/lib -d test.pl

Loading DB routines from perl5db.pl version 1.33 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test.pl​:1)​: *c = sub {};   DB\<1> r Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q bram@​wizbit​:/tmp/RT- 66110$

Test 12​: blead\, file​: test.pl​: user 'bram' and running with NonStop=1 -

no error bram@​wizbit​:/tmp/RT-66110$ PERLDB_OPTS="NonStop=1" /opt/perl/2009-06-18/ perl -I /opt/perl/2009-06-18/lib -d test.pl bram@​wizbit​:/tmp/RT-66110$

Changing the name of the test script​: test.pl -> test_15.pl

$ md5sum test.pl test_15.pl 950ddaee20bd8f2c6275abb4f252f4a3 test.pl 950ddaee20bd8f2c6275abb4f252f4a3 test_15.pl

$ diff -Naur test.pl test_15.pl $

Test 13​: blead\, file​: test_15.pl​: user 'perl' and using 'next' in the debugger -> error perl@​wizbit​:/tmp/RT-66110$ /opt/perl/2009-06-18/perl -I /opt/perl/2009- 06-18/lib -d test_15.pl

Loading DB routines from perl5db.pl version 1.33 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test_15.pl​:1)​: *c = sub {};   DB\<1> n main​::(test_15.pl​:2)​: '' =~ m/(?{ c() })/;   DB\<1> n main​::((reeval 4)[test_15.pl​:2]​:1)​: 1​:   DB\<1> n Out of memory! Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q *** glibc detected *** corrupted double-linked list​: 0xb8030878 *** Aborted perl@​wizbit​:/tmp/RT-66110$

Test 14​: blead\, file​: test_15.pl​: user 'perl' and using 'run' in the debugger -> no error perl@​wizbit​:/tmp/RT-66110$ /opt/perl/2009-06-18/perl -I /opt/perl/2009- 06-18/lib -d test_15.pl

Loading DB routines from perl5db.pl version 1.33 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test_15.pl​:1)​: *c = sub {};   DB\<1> r Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q perl@​wizbit​:/tmp/RT-66110$

Test 15​: blead\, file​: test_15.pl​: user 'perl' and running with NonStop=1 -> no error perl@​wizbit​:/tmp/RT-66110$ PERLDB_OPTS="NonStop=1" /opt/perl/2009-06-18/ perl -I /opt/perl/2009-06-18/lib -d test_15.pl perl@​wizbit​:/tmp/RT-66110$

Test 16​: blead\, file​: test_15.pl​: user 'bram' and using 'next' in the debugger -> error bram@​wizbit​:/tmp/RT-66110$ /opt/perl/2009-06-18/perl -I /opt/perl/2009- 06-18/lib -d test_15.pl

Loading DB routines from perl5db.pl version 1.33 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test_15.pl​:1)​: *c = sub {};   DB\<1> n main​::(test_15.pl​:2)​: '' =~ m/(?{ c() })/;   DB\<1> n main​::((reeval 4)[test_15.pl​:2]​:1)​: 1​:   DB\<1> n Out of memory! Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q *** glibc detected *** corrupted double-linked list​: 0xb7f9b878 *** Aborted bram@​wizbit​:/tmp/RT-66110$

Test 17​: blead\, file​: test_15.pl​: user 'bram' and using 'run' in the debugger -> no error bram@​wizbit​:/tmp/RT-66110$ /opt/perl/2009-06-18/perl -I /opt/perl/2009- 06-18/lib -d test_15.pl

Loading DB routines from perl5db.pl version 1.33 Editor support available.

Enter h or `h h' for help\, or `man perldebug' for more help.

main​::(test_15.pl​:1)​: *c = sub {};   DB\<1> r Debugged program terminated. Use q to quit or R to restart\,   use o inhibit_exit to avoid stopping after program termination\,   h q\, h R or h o to get additional info.   DB\<1> q bram@​wizbit​:/tmp/RT-66110$

Test 18​: blead\, file​: test_15.pl​: user 'bram' and running with NonStop=1 -> no error bram@​wizbit​:/tmp/RT-66110$ PERLDB_OPTS="NonStop=1" /opt/perl/2009-06-18/ perl -I /opt/perl/2009-06-18/lib -d test_15.pl bram@​wizbit​:/tmp/RT-66110$

Best regards\,

Bram

p5pRT commented 15 years ago

From p5p@spam.wizbit.be

rt-66110.pl

p5pRT commented 15 years ago

From p5p@perl.wizbit.be

Avar mailed p5p in
51dd1af80807190107h30b8626ct6d4d0a825abe4b3b@​mail.gmail.com http​://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-07/msg00382.html

perl 5.10 and blead will do various combinations of running of of memory\, hanging or segfaulting when running on a program using XML​::Parser​::Lite\, attached is a stripped down version of X​::P​::L which demonstrates the problem​:

Dave notes​:

possibly a 5.10.0 regression

Some more debugging (and trimming perl5db.pl to 19 lines) revealed the
problem.

In perl5db.pl in sub sub​:

http​://perl5.git.perl.org/perl.git/blob/85bdf03b25729816eedfea55a7f2c32c4bb80fba​:/lib/perl5db.pl 3650 if ($sub =~ /^threads​::new$/ && $ENV{PERL5DB_THREADED}) { 3651 print "creating new thread\n"; 3652 }

The trimmed test case from some earlier debugging​:

*c = sub {}; '' =~ m/(?{ c() })/;

What happens​: - c() is called inside a regex. - DB​::sub is called - DB​::sub starts a new regex

The end result is that a new regex is being executed inside a regex
and the regex engine can't handle that.

This also means it is reproduciable without the debugger​:

#!/usr/bin/perl -l

print $]; *c = sub { "foo" =~ m/bar/ }; sub parse_re {   $_[0] =~ m{(?{ c() })}; } parse_re(); __END__ 5.011000 Segmentation fault

(Of course since it is a memory corruption it may not be reproducible
on every system).

I'll submit a patch and a test case later today.

Best regards\,

Bram

p5pRT commented 15 years ago

From p5p@perl.wizbit.be

Citeren Bram \p5p@&#8203;perl\.wizbit\.be​:

Avar mailed p5p in
51dd1af80807190107h30b8626ct6d4d0a825abe4b3b@​mail.gmail.com http​://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-07/msg00382.html

perl 5.10 and blead will do various combinations of running of of memory\, hanging or segfaulting when running on a program using XML​::Parser​::Lite\, attached is a stripped down version of X​::P​::L which demonstrates the problem​:

Dave notes​:

possibly a 5.10.0 regression

Some more debugging (and trimming perl5db.pl to 19 lines) revealed the problem.

In perl5db.pl in sub sub​:

http​://perl5.git.perl.org/perl.git/blob/85bdf03b25729816eedfea55a7f2c32c4bb80fba​:/lib/perl5db.pl 3650 if ($sub =~ /^threads​::new$/ && $ENV{PERL5DB_THREADED}) { 3651 print "creating new thread\n"; 3652 }

The trimmed test case from some earlier debugging​:

*c = sub {}; '' =~ m/(?{ c() })/;

What happens​: - c() is called inside a regex. - DB​::sub is called - DB​::sub starts a new regex

The end result is that a new regex is being executed inside a regex and the regex engine can't handle that.

This also means it is reproduciable without the debugger​:

#!/usr/bin/perl -l

print $]; *c = sub { "foo" =~ m/bar/ }; sub parse_re { $_[0] =~ m{(?{ c() })}; } parse_re(); __END__ 5.011000 Segmentation fault

(Of course since it is a memory corruption it may not be reproducible on every system).

I'll submit a patch and a test case later today.

Patch attached.

Best regards\,

Bram

p5pRT commented 15 years ago

From p5p@perl.wizbit.be

0001-perl-66110-Do-not-use-a-regex-in-DB-sub.patch ```diff From f33b9071bf2d82a22bb2cb2ea5115d362f881159 Mon Sep 17 00:00:00 2001 From: Bram Date: Sat, 25 Jul 2009 16:26:45 +0200 Subject: [PATCH] [perl #66110]: Do not use a regex in DB::sub --- MANIFEST | 1 + lib/perl5db.pl | 4 +++- lib/perl5db.t | 10 +++++++++- lib/perl5db/t/rt-66110 | 36 ++++++++++++++++++++++++++++++++++++ 4 files changed, 49 insertions(+), 2 deletions(-) create mode 100644 lib/perl5db/t/rt-66110 diff --git a/MANIFEST b/MANIFEST index f26ec98..90f1156 100644 --- a/MANIFEST +++ b/MANIFEST @@ -2807,6 +2807,7 @@ lib/perl5db/t/eval-line-bug Tests for the Perl debugger lib/perl5db/t/lvalue-bug Tests for the Perl debugger lib/perl5db/t/proxy-constants Tests for the Perl debugger lib/perl5db/t/rt-61222 Tests for the Perl debugger +lib/perl5db/t/rt-66110 Tests for the Perl debugger lib/perl5db/t/symbol-table-bug Tests for the Perl debugger lib/PerlIO.pm PerlIO support module lib/PerlIO/via/QuotedPrint.pm PerlIO::via::QuotedPrint diff --git a/lib/perl5db.pl b/lib/perl5db.pl index 03ef2a2..33bbc47 100644 --- a/lib/perl5db.pl +++ b/lib/perl5db.pl @@ -3639,6 +3639,8 @@ arguments with which the subroutine was invoked =cut sub sub { + # Do not use a regex in this subroutine -> results in corrupted memory + # See: [perl #66110] # lock ourselves under threads lock($DBGR); @@ -3647,7 +3649,7 @@ sub sub { # sub's return value in (if needed), and an array to put the sub's # return value in (if needed). my ( $al, $ret, @ret ) = ""; - if ($sub =~ /^threads::new$/ && $ENV{PERL5DB_THREADED}) { + if ($sub eq 'threads::new' && $ENV{PERL5DB_THREADED}) { print "creating new thread\n"; } diff --git a/lib/perl5db.t b/lib/perl5db.t index 6e57c9f..59acd7a 100644 --- a/lib/perl5db.t +++ b/lib/perl5db.t @@ -27,7 +27,7 @@ my $dev_tty = '/dev/tty'; } } -plan(7); +plan(8); sub rc { open RC, ">", ".perldb" or die $!; @@ -160,6 +160,14 @@ SKIP: { } +# [perl #66110] Call a subroutine inside a regex +{ + local $ENV{PERLDB_OPTS} = "ReadLine=0 NonStop=1"; + my $output = runperl(switches => [ '-d' ], stderr => 1, progfile => '../lib/perl5db/t/rt-66110'); + like($output, "All tests successful.", "[perl #66110]"); +} + + # clean up. END { diff --git a/lib/perl5db/t/rt-66110 b/lib/perl5db/t/rt-66110 new file mode 100644 index 0000000..7ba6c36 --- /dev/null +++ b/lib/perl5db/t/rt-66110 @@ -0,0 +1,36 @@ +#!/usr/bin/perl +# +# This code is used by lib/perl5db.t !!! +# + +$all_ok = 1; +*c = sub { }; + +if ("abcdefghi" =~ m/(abc)(def)(?{ c() })(ghi)/) { + print "ok 1\n"; + + $all_ok = 0, print "not " if $1 ne 'abc'; + print "ok 2\n"; + + $all_ok = 0, print "not " if $2 ne 'def'; + print "ok 3\n"; + + $all_ok = 0, print "not " if $3 ne 'ghi'; + print "ok 4\n"; + + $all_ok = 0, print "not " if $& ne 'abcdefghi'; + print "ok 5\n"; +} +else { + $all_ok = 0; + print "not ok 1\n"; + print "not ok 2\n"; + print "not ok 3\n"; + print "not ok 4\n"; + print "not ok 5\n"; +} + +if ($all_ok) { + print "All tests successful."; +} + -- 1.6.3.2 ```
p5pRT commented 15 years ago

From @craigberry

On Sat\, Jul 25\, 2009 at 9​:38 AM\, Bram\p5p@&#8203;perl\.wizbit\.be wrote​:

Patch attached.

Thanks\, now in blead at​:

http​://perl5.git.perl.org/perl.git/commitdiff/b7bfa85

p5pRT commented 15 years ago

p5p@spam.wizbit.be - Status changed from 'open' to 'resolved'

p5pRT commented 15 years ago

From @craigberry

On Sat\, Jul 25\, 2009 at 2​:03 PM\, Craig A. Berry\craig\.a\.berry@&#8203;gmail\.com wrote​:

On Sat\, Jul 25\, 2009 at 9​:38 AM\, Bram\p5p@&#8203;perl\.wizbit\.be wrote​:

Patch attached.

Thanks\, now in blead at​:

http​://perl5.git.perl.org/perl.git/commitdiff/b7bfa85

Bram's change prevents the debugger from exercising the pathology reported.

My change here​:

http​://perl5.git.perl.org/perl.git/commitdiff/d80618d

prevents the regex engine from losing track of where its strings end when there is a regex inside of a regex\, thus guarding against the memory errors observed.

Note that other aspects of regex state are not saved and restored and a regex inside of a regex is still unsupported and is unlikely to produce correct results. perltodo.pod already mentions the desired behavior of full reentrancy for the regex engine.

p5pRT commented 15 years ago

From @druud62

Craig A. Berry wrote​:

a regex inside of a regex is still unsupported

IIRC\, code similar to the following is somewhere in the documentation. (but maybe including the e-modifier doesn't apply to the above?)

perl -wle'   my $s = "abc 123232323 def";   $s =~ s/([0-9]+)/ (my $t = $1) =~ s!.!*!g; $t /e;   print $s; ' abc ********* def

-- Ruud

p5pRT commented 15 years ago

From @jbenjore

"Dr.Ruud" \rvtol\+usenet@&#8203;isolution\.nl writes​:

Craig A. Berry wrote​:

a regex inside of a regex is still unsupported

IIRC\, code similar to the following is somewhere in the documentation. (but maybe including the e-modifier doesn't apply to the above?)

perl -wle' my $s = "abc 123232323 def"; $s =~ s/([0-9]+)/ (my $t = $1) =~ s!.!*!g; $t /e; print $s; ' abc ********* def

No. That's ok. What's dodgy is​:

  m{   (?{ m\<...> })   }

This shows up when calling functions within (?{}) and (??{}).

Josh

p5pRT commented 15 years ago

From @demerphq

2009/8/2 Josh ben Jore \twists@&#8203;gmail\.com​:

"Dr.Ruud" \rvtol\+usenet@&#8203;isolution\.nl writes​:

Craig A. Berry wrote​:

a regex inside of a regex is still unsupported

IIRC\, code similar to the following is somewhere in the documentation. (but maybe including the e-modifier doesn't apply to the above?)

perl -wle'     my $s = "abc 123232323 def";     $s =~ s/([0-9]+)/ (my $t = $1) =~ s!.!*!g; $t /e;     print $s; ' abc ********* def

No. That's ok. What's dodgy is​:

   m{       (?{ m\<...> })    }

This shows up when calling functions within (?{}) and (??{}).

Hmm\, i remember it being dodgy too. Are you sure its not?

Yves

-- perl -Mre=debug -e "/just|another|perl|hacker/"