search

Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.99k stars 559 forks source link

Memory leak from C<eval "sub { \$foo = 22 "> #8122

Closed p5pRT closed 18 years ago

p5pRT commented 19 years ago

Migrated from rt.perl.org#37231 (status was 'resolved')

Searchable as RT37231$

p5pRT commented 19 years ago

From adavies@ptc.com

Created by adavies@arbortext.com

I'm seeing a memory leak from C\ in the case​: eval "sub { \$foo = 22 "; # missing closing }

eg. # Watch memory usage grow​: $| = 1; while (1) { foreach (1 .. 1_000) {   eval "sub { \$foo = "; } print "Pausing...\n"; sleep 1; }

Perl Info ``` Flags: category=core severity=low Site configuration information for perl v5.8.7: Configured by Richard_Jinks at Fri Aug 12 10:36:13 2005. Summary of my perl5 (revision 5 version 8 subversion 7) configuration: Platform: osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX', optimize='-MD -DNDEBUG -O1', cppflags='-DWIN32' ccversion='12.00.8804', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -release -libpath:"C:\Perl3B2v9\lib\CORE" -machine:x86' libpth=\lib libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib gnulibc_version='undef' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release -libpath:"C:\Perl3B2v9\lib\CORE" -machine:x86' Locally applied patches: @INC for perl v5.8.7: C:/Perl3B2v9/lib C:/Perl3B2v9/site/lib . Environment for perl v5.8.7: HOME=C:\alex LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=C:\perl3\bin;C:\Program Files\Tcl-8.4.8.0\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\alex\bin;C:\Program Files\ATI Technologies\ATI Control Panel;C:\Program Files\Perforce;C:\cygwin\bin;C:\Program Files\Crimson Editor;C:\Program Files\Microsoft Visual Studio\VC98\Bin;C:\Program Files\Microsoft Visual Studio\Common\Tools;C:\Program Files\Vantive32;C:\Program Files\Sun\AppServer\bin;C:\ghc\ghc-6.2.2\bin PERL_BADLANG (unset) SHELL (unset) ```
p5pRT commented 18 years ago

From @smpeters

[adavies@​ptc.com - Thu Sep 22 04​:34​:01 2005]​:

This is a bug report for perl from adavies@​arbortext.com\, generated with the help of perlbug 1.35 running under perl v5.8.7.

----------------------------------------------------------------- [Please enter your report here]

I'm seeing a memory leak from C\ in the case​: eval "sub { \$foo = 22 "; # missing closing }

eg. # Watch memory usage grow​: $| = 1; while (1) { foreach (1 .. 1_000) { eval "sub { \$foo = "; } print "Pausing...\n"; sleep 1; }

Again\, here's another case of where Perl's memory allocation has been recently been improved. While a good deal of memory has been memory has been allocated\, it is greatly decreased\, but the majority of the memory is still reachable.

Summary of Perl-5.8.7​: ==2398== LEAK SUMMARY​: ==2398== definitely lost​: 133\,902 bytes in 13 blocks. ==2398== indirectly lost​: 107\,341 bytes in 537 blocks. ==2398== possibly lost​: 0 bytes in 0 blocks. ==2398== still reachable​: 0 bytes in 0 blocks. ==2398== suppressed​: 0 bytes in 0 blocks. ==2398== Reachable blocks (those to which a pointer was found) are not shown.

Summary of bleadperl ==2613== LEAK SUMMARY​: ==2613== definitely lost​: 92 bytes in 1 blocks. ==2613== indirectly lost​: 1\,252 bytes in 21 blocks. ==2613== possibly lost​: 0 bytes in 0 blocks. ==2613== still reachable​: 68\,027 bytes in 541 blocks. ==2613== suppressed​: 0 bytes in 0 blocks. ==2613== Reachable blocks (those to which a pointer was found) are not shown. ==2613== To see them\, rerun with​: --show-reachable=yes

p5pRT commented 18 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 18 years ago

@smpeters - Status changed from 'open' to 'resolved'

p5pRT commented 18 years ago

From @iabyn

On Thu\, Sep 22\, 2005 at 04​:34​:02AM -0700\, Davies\, Alex wrote​:

I'm seeing a memory leak from C\ in the case​: eval "sub { \$foo = 22 "; # missing closing }

This is now mostly fixed in the development branch of Perl by the change below

  eval "sub {";

no longer leaks\, while

  eval "sub { \$foo = 22 ";

still leaks slightly (I need to look into this further).

The CV created at the start of a new sub parse would be leaked if the sub parse didn't complete. The solution is to SAVEFREESV(PL_compcv) at the start; then if we arrive at the end\, undo it with SvREFCNT(PL_cpompcv)++.

The logical place for these two operations is in start_subparse() and newATTRSUB()\, but since they are used by code in the wild\, I've instead added the diddling directly to perly.y (and madly.y). A bit ugly.

Dave.

-- The warp engines start playing up a bit\, but seem to sort themselves out after a while without any intervention from boy genius Wesley Crusher.   -- Things That Never Happen in "Star Trek" #17

Change 28314 by davem@​davem-splatty on 2006/05/26 18​:50​:34

  stop eval "sub{" leaking

Affected files ...

... //depot/perl/madly.act#3 edit ... //depot/perl/madly.h#3 edit ... //depot/perl/madly.tab#3 edit ... //depot/perl/madly.y#3 edit ... //depot/perl/perly.act#15 edit ... //depot/perl/perly.h#26 edit ... //depot/perl/perly.tab#12 edit ... //depot/perl/perly.y#70 edit

Differences ...

==== //depot/perl/perly.y#70 (text) ====

@​@​ -341\,7 +341\,8 @​@​   ;

format : FORMAT startformsub formname block - { newFORM($2\, $3\, $4); } + { SvREFCNT_inc(PL_compcv); + newFORM($2\, $3\, $4); }   ;

formname​: WORD { $$ = $1; } @​@​ -350\,24 +351\,29 @​@​

/* Unimplemented "my sub foo { }" */ mysubrout​: MYSUB startsub subname proto subattrlist subbody - { newMYSUB($2\, $3\, $4\, $5\, $6); } + { SvREFCNT_inc(PL_compcv); + newMYSUB($2\, $3\, $4\, $5\, $6); }   ;

/* Subroutine definition */ subrout : SUB startsub subname proto subattrlist subbody - { newATTRSUB($2\, $3\, $4\, $5\, $6); } + { SvREFCNT_inc(PL_compcv); + newATTRSUB($2\, $3\, $4\, $5\, $6); }   ;

startsub​: /* NULL */ /* start a regular subroutine scope */ - { $$ = start_subparse(FALSE\, 0); } + { $$ = start_subparse(FALSE\, 0); + SAVEFREESV(PL_compcv); }   ;

startanonsub​: /* NULL */ /* start an anonymous subroutine scope */ - { $$ = start_subparse(FALSE\, CVf_ANON); } + { $$ = start_subparse(FALSE\, CVf_ANON); + SAVEFREESV(PL_compcv); }   ;

startformsub​: /* NULL */ /* start a format subroutine scope */ - { $$ = start_subparse(TRUE\, 0); } + { $$ = start_subparse(TRUE\, 0); + SAVEFREESV(PL_compcv); }   ;

/* Name of a subroutine - must be a bareword\, could be special */ @​@​ -412\,7 +418\,8 @​@​ use : USE startsub   { CvSPECIAL_on(PL_compcv); /* It's a BEGIN {} */ }   WORD WORD listexpr ';' - { utilize($1\, $2\, $4\, $5\, $6); } + { SvREFCNT_inc(PL_compcv); + utilize($1\, $2\, $4\, $5\, $6); }   ;

/* Ordinary expressions; logical combinations */ @​@​ -464\,7 +471\,8 @​@​   | FUNC '(' listexprcom ')' /* print (@​args) */   { $$ = convert($1\, 0\, $3); }   | LSTOPSUB startanonsub block /* sub f(&@​); f { foo } ... */ - { $3 = newANONATTRSUB($2\, 0\, Nullop\, $3); } + { SvREFCNT_inc(PL_compcv); + $3 = newANONATTRSUB($2\, 0\, Nullop\, $3); }   listexpr %prec LSTOP /* ... @​bar */   { $$ = newUNOP(OP_ENTERSUB\, OPf_STACKED\,   append_elem(OP_LIST\, @​@​ -593\,7 +601\,8 @​@​   | HASHBRACK ';' '}' %prec '(' /* { } (';' by tokener) */   { $$ = newANONHASH(Nullop); }   | ANONSUB startanonsub proto subattrlist block %prec '(' - { $$ = newANONATTRSUB($2\, $3\, $4\, $5); } + { SvREFCNT_inc(PL_compcv); + $$ = newANONATTRSUB($2\, $3\, $4\, $5); }

  ;

(other files snipped for clarity)

p5pRT commented 18 years ago

From @iabyn

On Fri\, May 26\, 2006 at 07​:54​:24PM +0100\, Dave Mitchell wrote​:

On Thu\, Sep 22\, 2005 at 04​:34​:02AM -0700\, Davies\, Alex wrote​:

I'm seeing a memory leak from C\ in the case​: eval "sub { \$foo = 22 "; # missing closing }

This is now mostly fixed in the development branch of Perl by the change below

eval "sub \{";

no longer leaks\, while

eval "sub \{ \\$foo = 22 ";

still leaks slightly (I need to look into this further).

This leaks\, because during error recovery\, bison pops states from the parsee stack\, and if any of those those states refence an op\, that op gets leaked. The change below fixes this leak (and in fact probably fixes the bulk of eval syntax error leaks)\, by​:

* hacking regen_perly.y to add an extra table to perly.tab that indicates for each state\, whether is is of type opval

* in perly.c\, every time a state is popped from the stack\, checks to see * if there is an op associates with it\, and if so\, frees it.

Dave.

-- The Enterprise is involved in a bizarre time-warp experience which is in some way unconnected with the Late 20th Century.   -- Things That Never Happen in "Star Trek" #14

Change 28315 by davem@​davem-splatty on 2006/05/27 00​:31​:33

  stop OPs leaking in eval "syntax error"   When bison pops states during error recovery\, any states holding   an OP would leak the OP. Create an extra YY table that tells us   which states are of type opval\, and when popping one of those\,   free the op.

Affected files ...

... //depot/perl/madly.tab#4 edit ... //depot/perl/perly.c#80 edit ... //depot/perl/perly.tab#13 edit ... //depot/perl/regen_perly.pl#8 edit

Differences ...

==== //depot/perl/madly.tab#4 (text) ====

@​@​ -876\,3 +876\,30 @​@​   60\, 96\, 96\, 105\, 105\, 99\, 96\, 88\, 96\, 108\,   105\, 4\, 112\, 105\, 113\, 87\, 87\, 96\, 96\, 102 }; +/* which symbols are of type opval */ +static const int yy_is_opval[] = +{ + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 1\, 1\, 1\, 1\, + 1\, 1\, 1\, 1\, 1\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 1\, 0\, 0\, 0\, 1\, 0\, + 0\, 1\, 1\, 1\, 1\, 1\, 1\, 1\, + 1\, 1\, 0\, 1\, 1\, 1\, 1\, 1\, + 1\, 0\, 1\, 1\, 1\, 1\, 1\, + 1\, 0\, 0\, 0\, 1\, + 1\, 1\, 1\, 1\, 1\, 1\, 0\, + 1\, 1\, 1\, 0\, 1\, 1\, 1\, + 1\, 1\, 1\, 1\, 1\, 1\, + 1\, 1\, 1\, 1\, 1\, 1\, 1\, + 1\, 1\, 1\, 0 + +};

==== //depot/perl/perly.c#80 (text) ====

@​@​ -613\,6 +613\,10 @​@​   /* Pop the rest of the stack. */   while (yyss \< yyssp) {   YYDSYMPRINTF ("Error​: popping"\, yystos[*yyssp]\, yyvsp); + if (yy_is_opval[yystos[*yyssp]]) { + YYDPRINTF ((Perl_debug_log\, "(freeing op)\n")); + op_free(yyvsp->opval); + }   YYPOPSTACK;   }   YYABORT; @​@​ -650\,6 +654\,10 @​@​   YYABORT;

  YYDSYMPRINTF ("Error​: popping"\, yystos[*yyssp]\, yyvsp); + if (yy_is_opval[yystos[*yyssp]]) { + YYDPRINTF ((Perl_debug_log\, "(freeing op)\n")); + op_free(yyvsp->opval); + }   yyvsp--; #ifdef DEBUGGING   yynsp--;

==== //depot/perl/perly.tab#13 (text) ====

@​@​ -884\,3 +884\,30 @​@​   96\, 105\, 105\, 99\, 96\, 78\, 96\, 108\, 105\, 81\,   112\, 105\, 113\, 77\, 77\, 96\, 96\, 102 }; +/* which symbols are of type opval */ +static const int yy_is_opval[] = +{ + 0\, 0\, 0\, 0\, 1\, 1\, 1\, + 1\, 1\, 1\, 1\, 1\, 1\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, 0\, + 0\, 0\, 1\, 0\, 0\, 0\, 1\, + 0\, 0\, 1\, 1\, 1\, 1\, 1\, + 1\, 1\, 1\, 1\, 0\, 1\, 1\, 1\, + 1\, 1\, 1\, 0\, 0\, 0\, 1\, + 1\, 0\, 0\, 0\, 0\, + 1\, 1\, 1\, 1\, 1\, 0\, + 0\, 0\, 1\, 1\, 1\, 0\, 1\, 1\, + 1\, 1\, 1\, 1\, 1\, 1\, + 1\, 1\, 1\, 1\, 1\, 1\, + 1\, 1\, 1\, 1\, 1\, 0 + +};

==== //depot/perl/regen_perly.pl#8 (text) ====

@​@​ -11\,6 +11\,7 @​@​ # #line directives plus adding a #ifdef PERL_CORE # # perly.tab the parser table C definitions extracted from the bison output +# plus an extra table generated by this script. # # perly.act the action case statements extracted from the bison output # @​@​ -87\,6 +88\,8 @​@​

my ($actlines\, $tablines) = extract($clines);

+$tablines .= make_opval_tab($y_file\, $tablines); + chmod 0644\, $act_file; open ACTFILE\, ">$act_file" or die "can't open $act_file​: $!\n"; print ACTFILE $actlines; @​@​ -169\,6 +172\,45 @​@​   return $actlines. "\n"\, $tablines. "\n"; }

+# read a .y file and extract a list of all the token names and +# non-terminal names that are declared to be of type opval +# then scan the string $tablines for the table yytname which gives +# the token index of each token/non-terminal\, then use this to +# create a new table\, indexed by token number\, which indicates +# whether that token is of type opval. +# +# ie given +# %token \ A B +# %type \ C D +# +# and yytname[] = { "A" "B"\, "C"\, "D"\, "E"\, "F" }; +# +# then return +# static const int yy_is_opval[] = { 1\, 1\, 1\, 1\, 0\, 0 } + +sub make_opval_tab { + my ($y_file\, $tablines) = @​_; + my %tokens; + open my $fh\, '\<'\, $y_file or die "Can't open $y_file​: $!\n"; + while (\<$fh>) { + next unless s/^%(token|type)\s+\\s+//; + $tokens{$_} =1 for (split ' '\, $_); + } + + $tablines =~ /^\Qstatic const char *const yytname[] =\E\n + {\n + (.*?) + ^}; + /xsm + or die "Can't extract yytname[] from table string\n"; + my $fields = $1; + $fields =~ s/"([^"]+)"/$tokens{$1}||0/ge; + return + "/* which symbols are of type opval */\n" . + "static const int yy_is_opval[] =\n{\n" . $fields . "\n};\n"; +} + + sub my_system {   system(@​_);   if ($? == -1) {

p5pRT commented 18 years ago

From @nwc10

On Sat\, May 27\, 2006 at 01​:37​:30AM +0100\, Dave Mitchell wrote​:

This leaks\, because during error recovery\, bison pops states from the parsee stack\, and if any of those those states refence an op\, that op gets leaked. The change below fixes this leak (and in fact probably fixes the bulk of eval syntax error leaks)\, by​:

* hacking regen_perly.y to add an extra table to perly.tab that indicates for each state\, whether is is of type opval

* in perly.c\, every time a state is popped from the stack\, checks to see * if there is an op associates with it\, and if so\, frees it.

Very nice.

I have managed to find one thing that still leaks - ./perl -e '1; use abc'

gives

==28442== 28 bytes in 1 blocks are definitely lost in loss record 4 of 7 ==28442== at 0x1B9037A5​: calloc (vg_replace_malloc.c​:176) ==28442== by 0x8066A2D​: Perl_newLISTOP (op.c​:2664) ==28442== by 0x806699D​: Perl_prepend_elem (op.c​:2398) ==28442== by 0x806BC14​: Perl_newSTATEOP (op.c​:3969) ==28442== by 0x8211FEA​: Perl_yyparse (perly.y​:174) ==28442== by 0x80B994B​: S_parse_body (perl.c​:2242) ==28442== by 0x80B7FE3​: perl_parse (perl.c​:1607) ==28442== by 0x806036A​: main (perlmain.c​:101)

Also\, I was hoping that your 2 changes would solve this one (unthreaded build)​:

$ PERL_DESTRUCT_LEVEL=2 ./perl -e 'BEGIN {$^H{a}++}; 1; use 6' Perl v6.0.0 required--this is only v5.9.4\, stopped at -e line 1. BEGIN failed--compilation aborted at -e line 1. Unbalanced string table refcount​: (1) for "a" during global destruction.

The unbalanced string table refcount is because the refcount on the struct refcounted_he isn't being dropped properly. But I'm not sure how to arrange that it is dropper properly. (It's not "leaked"\, in as much as valgrind reports that it's still reachable\, but arguably that is a leak\, in that everything should be destroyed and freed up prior to interpreter exit)

Nicholas Clark

p5pRT commented 18 years ago

From @iabyn

On Sat\, May 27\, 2006 at 05​:16​:11PM +0100\, Nicholas Clark wrote​:

I have managed to find one thing that still leaks - ./perl -e '1; use abc' ... $ PERL_DESTRUCT_LEVEL=2 ./perl -e 'BEGIN {$^H{a}++}; 1; use 6'

Pah! Your mother is an hamster and your father smells of elderberries!

Hopefully the change below fixes those too. My previous patch coped with syntax errors. Your examples are where the parser calls code which dies\, so the current parse stack is just abandoned.

I've added a SAVEDESTRUCTOR_X function to clean up the parse stack on premature exit.

Hopefully there should be very few remaining eval leaks. (Nothing like saying that on a public mailing list to encourage people to prove you wrong....)

-- Atheism is a religion like not collecting stamps is a hobby

Change 28319 by davem@​davem-splatty on 2006/05/27 21​:16​:30

  fix eval qw(BEGIN{die}) style leaks.   death while exdcuting code while parsing meant that the current   parse stack got quiety abandonded\, thus leaking a bunch of OPs.   Register a destructor to be called when this happens.

Affected files ...

... //depot/perl/perly.c#82 edit

Differences ...

==== //depot/perl/perly.c#82 (text) ====

@​@​ -245\,6 +245\,38 @​@​

#endif /* !YYERROR_VERBOSE */

+ +/* a snapshot of the current stack position variables for use by + * S_clear_yystack */ + +typedef struct { + short *yyss; + short *yyssp; + YYSTYPE *yyvsp; + int yylen; +} yystack_positions; + +/* called during cleanup (via SAVEDESTRUCTOR_X) to free any items on the + * parse stack\, thus avoiding leaks if we die */ + +static void +S_clear_yystack(pTHX_ const void *p) +{ + yystack_positions *y = (yystack_positions*) p; + + if (!y->yyss) + return; + YYDPRINTF ((Perl_debug_log\, "clearing the parse stack\n")); + y->yyvsp -= y->yylen; /* ignore the tokens that have just been reduced */ + y->yyssp -= y->yylen; + while (y->yyssp > y->yyss) { + if (yy_is_opval[yystos[*y->yyssp]]) + op_free(y->yyvsp->opval); + y->yyvsp--; + y->yyssp--; + } +} + /*----------. | yyparse. | `----------*/ @​@​ -283\,6 +315\,8 @​@​   /* for ease of re-allocation and automatic freeing\, have two SVs whose   * SvPVX points to the stacks */   SV *yyss_sv\, *yyvs_sv; + SV *ss_save_sv; + yystack_positions *ss_save;

#ifdef DEBUGGING   /* maintain also a stack of token/rule names for debugging with -Dpv */ @​@​ -320\,10 +354\,18 @​@​

  yyss_sv = newSV(YYINITDEPTH * sizeof(short));   yyvs_sv = newSV(YYINITDEPTH * sizeof(YYSTYPE)); + ss_save_sv = newSV(sizeof(yystack_positions));   SAVEFREESV(yyss_sv);   SAVEFREESV(yyvs_sv); + SAVEFREESV(ss_save_sv);   yyss = (short *) SvPVX(yyss_sv);   yyvs = (YYSTYPE *) SvPVX(yyvs_sv); + ss_save = (yystack_positions *) SvPVX(ss_save_sv); + + ss_save->yyss = NULL; /* disarm stack cleanup */ + /* cleanup the parse stack on premature exit */ + SAVEDESTRUCTOR_X(S_clear_yystack\, (void*) ss_save); +   /* note that elements zero of yyvs and yyns are not used */   yyssp = yyss;   yyvsp = yyvs; @​@​ -340\,8 +382\,6 @​@​   yynerrs = 0;   yychar = YYEMPTY; /* Cause a token to be read. */

- -   YYDPRINTF ((Perl_debug_log\, "Entering state %d\n"\, yystate));

  goto yysetstate; @​@​ -507\,6 +547\,15 @​@​

  YY_REDUCE_PRINT (yyn); + + /* running external code may trigger a die (eg 'use nosuchmodule')​: + * record the current stack state so that an unwind will + * free all the pesky OPs lounging around on the parse stack */ + ss_save->yyss = yyss; + ss_save->yyssp = yyssp; + ss_save->yyvsp = yyvsp; + ss_save->yylen = yylen; +   switch (yyn) {

/* contains all the rule actions; auto-generated from perly.y */ @​@​ -716\,7 +765\,8 @​@​

  yyreturn​:

- LEAVE; /* force stack free before we return */ + ss_save->yyss = NULL; /* disarm parse stack cleanup */ + LEAVE; /* force stack free before we return */

  return yyresult; }

p5pRT commented 18 years ago

From @nwc10

On Sat\, May 27\, 2006 at 10​:21​:50PM +0100\, Dave Mitchell wrote​:

On Sat\, May 27\, 2006 at 05​:16​:11PM +0100\, Nicholas Clark wrote​:

I have managed to find one thing that still leaks - ./perl -e '1; use abc' ... $ PERL_DESTRUCT_LEVEL=2 ./perl -e 'BEGIN {$^H{a}++}; 1; use 6'

Pah! Your mother is an hamster and your father smells of elderberries!

And you're a very nice man. Thank you.

Hopefully the change below fixes those too.

Seems to.

My previous patch coped with syntax errors. Your examples are where the parser calls code which dies\, so the current parse stack is just abandoned.

Hopefully there should be very few remaining eval leaks. (Nothing like saying that on a public mailing list to encourage people to prove you wrong....)

Well\, I wasn't setting out to prove you wrong. It was more that I had this ongoing problem that I was stuck on. And you've solved it.

Nicholas Clark

p5pRT commented 18 years ago

From @nwc10

On Sat\, May 27\, 2006 at 11​:43​:17PM +0100\, Nicholas Clark wrote​:

On Sat\, May 27\, 2006 at 10​:21​:50PM +0100\, Dave Mitchell wrote​:

On Sat\, May 27\, 2006 at 05​:16​:11PM +0100\, Nicholas Clark wrote​:

I have managed to find one thing that still leaks - ./perl -e '1; use abc' ... $ PERL_DESTRUCT_LEVEL=2 ./perl -e 'BEGIN {$^H{a}++}; 1; use 6'

Pah! Your mother is an hamster and your father smells of elderberries!

And you're a very nice man. Thank you.

And for your next insult? :-)

Hopefully there should be very few remaining eval leaks. (Nothing like saying that on a public mailing list to encourage people to prove you wrong....)

Well\, I wasn't setting out to prove you wrong. It was more that I had this ongoing problem that I was stuck on. And you've solved it.

Er\, well\, there is *this* one​:

PERL_DESTRUCT_LEVEL=2 valgrind --tool=memcheck --leak-check=yes --num-callers=25 ./perl -e 'eval q% s/a/"b"}/e %' ==29874== Memcheck\, a memory error detector for x86-linux. ==29874== Copyright (C) 2002-2004\, and GNU GPL'd\, by Julian Seward et al. ==29874== Using valgrind-2.2.0\, a program supervision framework for x86-linux. ==29874== Copyright (C) 2000-2004\, and GNU GPL'd\, by Julian Seward et al. ==29874== For more details\, rerun with​: -v ==29874== ==29874== ==29874== ERROR SUMMARY​: 0 errors from 0 contexts (suppressed​: 25 from 1) ==29874== malloc/free​: in use at exit​: 192 bytes in 5 blocks. ==29874== malloc/free​: 833 allocs\, 828 frees\, 99062 bytes allocated. ==29874== For counts of detected errors\, rerun with​: -v ==29874== searching for pointers to 5 not-freed blocks. ==29874== checked 3831948 bytes. ==29874== ==29874== 24 bytes in 1 blocks are definitely lost in loss record 1 of 5 ==29874== at 0x1B9037A5​: calloc (vg_replace_malloc.c​:176) ==29874== by 0x8064822​: Perl_newUNOP (op.c​:2722) ==29874== by 0x81EC2FA​: Perl_yyparse (perly.y​:613) ==29874== by 0x81303F9​: S_doeval (pp_ctl.c​:2931) ==29874== by 0x8133E78​: Perl_pp_entereval (pp_ctl.c​:3497) ==29874== by 0x8087ED5​: Perl_runops_debug (dump.c​:1738) ==29874== by 0x80B067B​: S_run_body (perl.c​:2391) ==29874== by 0x80AFCD1​: perl_run (perl.c​:2311) ==29874== by 0x805EA12​: main (perlmain.c​:103) ==29874== ==29874== LEAK SUMMARY​: ==29874== definitely lost​: 24 bytes in 1 blocks. ==29874== possibly lost​: 0 bytes in 0 blocks. ==29874== still reachable​: 168 bytes in 4 blocks. ==29874== suppressed​: 0 bytes in 0 blocks. ==29874== Reachable blocks (those to which a pointer was found) are not shown. ==29874== To see them\, rerun with​: --show-reachable=yes

Found because it shows up like this on an unthreaded build​:

$ LC_ALL=en_GB.UTF-8 PERL_UNICODE="" PERL_DESTRUCT_LEVEL=2 ./perl -e 'eval q% s/a/"b"}/e %' Unbalanced string table refcount​: (1) for "open" during global destruction.

which in turn is a really cut down version to t/op/subst.t

Nicholas Clark

p5pRT commented 18 years ago

From @iabyn

On Mon\, Jun 05\, 2006 at 11​:38​:10PM +0100\, Nicholas Clark wrote​:

And for your next insult? :-)

You sir\, are a knave; a rascal; an eater of broken meats; a base\, proud\, shallow\, beggarly\, three-suited\, hundred-pound\, filthy\, worsted-stocking knave; a lily-livered\, action-taking knave\, a whoreson\, glass-gazing\, super-serviceable finical rogue; one-trunk-inheriting slave; one that wouldst be a bawd\, in way of good service\, and art nothing but the composition of a knave\, beggar\, coward\, pandar\, and the son and heir of a mongrel bitch​: one whom I will beat into clamorous whining\, if thou deniest the least syllable of thy addition.

(Lear\, act II scene ii)

Er\, well\, there is *this* one​:

PERL_DESTRUCT_LEVEL=2 valgrind --tool=memcheck --leak-check=yes --num-callers=25 ./perl -e 'eval q% s/a/"b"}/e %'

Oh\, *that* one.

find_leak("eval"\, F_NODELAY) = -1 ENOTIME (real life interrupt)

-- In defeat\, indomitable; in victory\, insufferable   -- Churchill on Montgomery

p5pRT commented 18 years ago

From @nwc10

On Tue\, Jun 06\, 2006 at 12​:49​:09AM +0100\, Dave Mitchell wrote​:

On Mon\, Jun 05\, 2006 at 11​:38​:10PM +0100\, Nicholas Clark wrote​:

Er\, well\, there is *this* one​:

PERL_DESTRUCT_LEVEL=2 valgrind --tool=memcheck --leak-check=yes --num-callers=25 ./perl -e 'eval q% s/a/"b"}/e %'

Oh\, *that* one.

And this one​:

$ PERL_DESTRUCT_LEVEL=2 valgrind --tool=memcheck --leak-check=yes --num-callers=20 ./perl -Ilib -e 'use warnings FATAL => qw(void); length "abc";' ==12010== Memcheck\, a memory error detector for x86-linux. ==12010== Copyright (C) 2002-2004\, and GNU GPL'd\, by Julian Seward et al. ==12010== Using valgrind-2.2.0\, a program supervision framework for x86-linux. ==12010== Copyright (C) 2000-2004\, and GNU GPL'd\, by Julian Seward et al. ==12010== For more details\, rerun with​: -v ==12010== Useless use of length in void context at -e line 1. ==12010== ==12010== ERROR SUMMARY​: 0 errors from 0 contexts (suppressed​: 25 from 1) ==12010== malloc/free​: in use at exit​: 208 bytes in 6 blocks. ==12010== malloc/free​: 4285 allocs\, 4279 frees\, 250471 bytes allocated. ==12010== For counts of detected errors\, rerun with​: -v ==12010== searching for pointers to 6 not-freed blocks. ==12010== checked 3831996 bytes. ==12010== ==12010== 28 bytes in 1 blocks are definitely lost in loss record 4 of 6 ==12010== at 0x1B9037A5​: calloc (vg_replace_malloc.c​:176) ==12010== by 0x806458B​: Perl_newLISTOP (op.c​:2664) ==12010== by 0x8064507​: Perl_prepend_elem (op.c​:2398) ==12010== by 0x8068A97​: Perl_newSTATEOP (op.c​:3969) ==12010== by 0x81EAC81​: Perl_yyparse (perly.y​:174) ==12010== by 0x80AFA59​: S_parse_body (perl.c​:2242) ==12010== by 0x80AE3AD​: perl_parse (perl.c​:1607) ==12010== by 0x805E9F8​: main (perlmain.c​:101) ==12010== ==12010== LEAK SUMMARY​: ==12010== definitely lost​: 28 bytes in 1 blocks. ==12010== possibly lost​: 0 bytes in 0 blocks. ==12010== still reachable​: 180 bytes in 5 blocks. ==12010== suppressed​: 0 bytes in 0 blocks. ==12010== Reachable blocks (those to which a pointer was found) are not shown. ==12010== To see them\, rerun with​: --show-reachable=yes

I think both are related\, in that they are due to death somewhere while building the optree\, but I don't know enough to figure a better connection or a cure.

find_leak("eval"\, F_NODELAY) = -1 ENOTIME (real life interrupt)

Oh. :-(

Nicholas Clark

p5pRT commented 18 years ago

From @nwc10

On Wed\, Jun 07\, 2006 at 06​:43​:06PM +0100\, Nicholas Clark wrote​:

On Tue\, Jun 06\, 2006 at 12​:49​:09AM +0100\, Dave Mitchell wrote​:

On Mon\, Jun 05\, 2006 at 11​:38​:10PM +0100\, Nicholas Clark wrote​:

Er\, well\, there is *this* one​:

PERL_DESTRUCT_LEVEL=2 valgrind --tool=memcheck --leak-check=yes --num-callers=25 ./perl -e 'eval q% s/a/"b"}/e %'

Oh\, *that* one.

And this one​:

$ PERL_DESTRUCT_LEVEL=2 valgrind --tool=memcheck --leak-check=yes --num-callers=20 ./perl -Ilib -e 'use warnings FATAL => qw(void); length "abc";'

And this one​:

$ PERL_DESTRUCT_LEVEL=2 valgrind --tool=memcheck --leak-check=yes --num-callers=20 ./perl -Ilib -e 'use warnings FATAL => qw(void); $a = sort @​b;' ==12077== Memcheck\, a memory error detector for x86-linux. ==12077== Copyright (C) 2002-2004\, and GNU GPL'd\, by Julian Seward et al. ==12077== Using valgrind-2.2.0\, a program supervision framework for x86-linux. ==12077== Copyright (C) 2000-2004\, and GNU GPL'd\, by Julian Seward et al. ==12077== For more details\, rerun with​: -v ==12077== Useless use of sort in scalar context at -e line 1. ==12077== ==12077== ERROR SUMMARY​: 0 errors from 0 contexts (suppressed​: 25 from 1) ==12077== malloc/free​: in use at exit​: 208 bytes in 7 blocks. ==12077== malloc/free​: 4287 allocs\, 4280 frees\, 250485 bytes allocated. ==12077== For counts of detected errors\, rerun with​: -v ==12077== searching for pointers to 7 not-freed blocks. ==12077== checked 3831996 bytes. ==12077== ==12077== 24 bytes in 1 blocks are definitely lost in loss record 3 of 6 ==12077== at 0x1B9037A5​: calloc (vg_replace_malloc.c​:176) ==12077== by 0x8064822​: Perl_newUNOP (op.c​:2722) ==12077== by 0x8070308​: Perl_newSVREF (op.c​:5792) ==12077== by 0x81ECB71​: Perl_yyparse (perly.y​:780) ==12077== by 0x80AFA59​: S_parse_body (perl.c​:2242) ==12077== by 0x80AE3AD​: perl_parse (perl.c​:1607) ==12077== by 0x805E9F8​: main (perlmain.c​:101) ==12077== ==12077== LEAK SUMMARY​: ==12077== definitely lost​: 24 bytes in 1 blocks. ==12077== possibly lost​: 0 bytes in 0 blocks. ==12077== still reachable​: 184 bytes in 6 blocks. ==12077== suppressed​: 0 bytes in 0 blocks. ==12077== Reachable blocks (those to which a pointer was found) are not shown. ==12077== To see them\, rerun with​: --show-reachable=yes

I think both are related\, in that they are due to death somewhere while building the optree\, but I don't know enough to figure a better connection or a cure.

Well\, these two are definitely related as they're warnings triggered while building the optree that have been made fatal.

Nicholas Clark

p5pRT commented 18 years ago

From @nwc10

I can't even work out why

On Wed\, Jun 07\, 2006 at 06​:43​:06PM +0100\, Nicholas Clark wrote​:

==12010== 28 bytes in 1 blocks are definitely lost in loss record 4 of 6 ==12010== at 0x1B9037A5​: calloc (vg_replace_malloc.c​:176) ==12010== by 0x806458B​: Perl_newLISTOP (op.c​:2664)

this op is lost

==12010== by 0x8064507​: Perl_prepend_elem (op.c​:2398) ==12010== by 0x8068A97​: Perl_newSTATEOP (op.c​:3969)

but this op that points to it (or so I thought) is reachable.

(adding --show-reachable=yes shows that quite a few ops aren't freed. But I'm confused as to why that particular op isn't linked from any of those ops)

Nicholas Clark