Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.91k stars 542 forks source link

@_ gets corrupted when F(@X) shortens @X #4924

Open p5pRT opened 22 years ago

p5pRT commented 22 years ago

Migrated from rt.perl.org#8358 (status was 'open')

Searchable as RT8358$

p5pRT commented 22 years ago

From vallon@bear.com

Created by vallon@bear.com

(Also reproduced in 5.005)

SAMPLE CODE​: Here is a sample program​:  
  local @​X;   local @​Y;

  sub F {   print join('\,'\, @​_)\, "\n";   @​X = ();   print join('\,'\, @​_)\, "\n";   @​Y = qw/ magic /;   print join('\,'\, @​_)\, "\n";   }  
  @​X = qw/ A B C D E /;   F('start'\, @​X\, 'end');

With the curious output​:

  start\,A\,B\,C\,D\,E\,end   start\,\,\,\,\,\,end   start\,magic\,\,\,\,\,end

BUG​: The bug is that in F\, after @​X is emptied\, @​_ seems to contain dangling references. The allocation of 'magic' seems to reuse some recently released memory\, and @​_ (still) references that memory.

EXPECTED BEHAVIOR​: How is this supposed to work? What should @​_ reference if @​X gets shorter? My guess would be that it evaluates to undef for @​_ elements that reference truncated values of @​X\, and assignment to non-existent values automatically extends @​X (ala lvalue array entries).

BIGGER EXAMPLE​: In the following\, @​_ ends up referencing something that causes a croak from within Perl_sv_setsv ("Bizarre copy of..."). As with all memory corruption\, actual behavior is undefined.

#!/usr/local/bin/perl5.6.1 -w

local @​X;

sub Main {   print 'orig​: '\, join('\,'\, @​_)\, "\n"; # 'start'\, 'A'..'M'\, 'end'

  @​X = ();

  print 'clear​: '\, join('\,'\, @​_)\, "\n"; # @​_ seems ok\, but isn't

  my @​z = qw/ MAGIC STRINGS /;

  print 'magic​: '\, join('\,'\, @​_)\, "\n"; # oops\, dangling pointer???

  require Carp; # massage heap

  print 'trash​: '\, join('\,'\, @​_)\, "\n"; # oops\, more trash from load

  my @​a = @​_; # crash (maybe?) : Bizarre copy of CODE in aassign at ./test2 line 9. }

@​X = qw/ A B C D E F G H I J K L M /; Main('start'\, @​X\, 'end');

Perl Info ``` Flags: category=core severity=medium Site configuration information for perl v5.6.1: Configured by software at Wed Dec 12 09:52:28 EST 2001. Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration: Platform: osname=hpux, osvers=10.20, archname=PA-RISC2.0-multi uname='hp-ux devsrv1 b.10.20 a 9000785 2004553983 two-user license ' config_args='-d -e -Dperl5=/usr/local/bin/perl5 -Dprefix=/usr/local/public/perl-5.6.1 -Uusemymalloc -Uinstallusrbinperl -Dusemultiplicity -Dccflags=-Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit ' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=define useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef Compiler: cc='cc', ccflags =' -Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit -D_HPUX_SOURCE -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 ', optimize='-O', cppflags='-Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit -D_HPUX_SOURCE -Aa -I/usr/local/include' ccversion='A.10.32.30', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, usemymalloc=n, prototype=define Linker and Libraries: ld='ld', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib libs=-lnsl_s -lndbm -lgdbm -lmalloc -ldld -lm -lc -lndir -lcrypt -lsec perllibs=-lnsl_s -lmalloc -ldld -lm -lc -lndir -lcrypt -lsec libc=/lib/libc.sl, so=sl, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_hpux.xs, dlext=sl, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-B,deferred ' cccdlflags='+z', lddlflags='-b +vnocompatwarnings -L/usr/local/lib' Locally applied patches: @INC for perl v5.6.1: /usr/derivs/public/lib/perl /a/vallon/lib/perl /usr/local/public/perl-5.6.1/lib/5.6.1/PA-RISC2.0-multi /usr/local/public/perl-5.6.1/lib/5.6.1 /usr/local/public/perl-5.6.1/lib/site_perl/5.6.1/PA-RISC2.0-multi /usr/local/public/perl-5.6.1/lib/site_perl/5.6.1 /usr/local/public/perl-5.6.1/lib/site_perl . Environment for perl v5.6.1: HOME=/a/vallon LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/derivs/public/bin:/a/vallon/jv/bin:/usr/local/licensed/autosys/HP-UX/autosys/bin:/a/vallon/bin:/a/vallon/import/bin:/a/vallon/public/bin:/usr/local/bin:/usr/bin/X11:/usr/local/bin/X11:/usr/ucb:/bin:/usr/bin:/opt/softbench/bin:/usr/bin:/usr/ccs/bin:/usr/contrib/bin:/opt/ansic/bin:/opt/nettladm/bin:/opt/pd/bin:/opt/upgrade/bin:/usr/bin/X11:/usr/contrib/bin/X11:/opt/hparray/bin:/opt/resmon/bin:/opt/perf/bin:/opt/langtools/bin:/opt/imake/bin:/opt/aCC/bin:/opt/fortran/bin:/usr/local/X11R5/bin:/usr/atria/bin:/usr/derivs/prod/bin:/usr/derivs/apps/bin:/usr/local/sybase-11.1.1/bin:/a/derivs/bin PERL5LIB=/usr/derivs/public/lib/perl:/a/vallon/lib/perl PERL_BADLANG (unset) SHELL=/bin/ksh SHLIB_PATH (unset) **************************************************************** Bear Stearns is not responsible for any recommendation, solicitation, offer or agreement or any information about any transaction, customer account or account activity contained in this communication. *********************************************************************** ```
p5pRT commented 22 years ago

From @schwern

Confirmed from bleadperl back to 5.003_07 at least.

On Mon\, Jan 28\, 2002 at 01​:30​:13PM -0500\, Justin Vallon wrote​:

This is a bug report for perl from vallon@​bear.com\, generated with the help of perlbug 1.33 running under perl v5.6.1.

----------------------------------------------------------------- [Please enter your report here]

(Also reproduced in 5.005)

SAMPLE CODE​: Here is a sample program​:

local @​X;
local @​Y;

The local doesn't appear to be necessary for the bug to manifest.

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}

@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​:

start\,A\,B\,C\,D\,E\,end
start\,\,\,\,\,\,end
start\,magic\,\,\,\,\,end

BUG​: The bug is that in F\, after @​X is emptied\, @​_ seems to contain dangling references. The allocation of 'magic' seems to reuse some recently released memory\, and @​_ (still) references that memory.

EXPECTED BEHAVIOR​: How is this supposed to work? What should @​_ reference if @​X gets shorter? My guess would be that it evaluates to undef for @​_ elements that reference truncated values of @​X\, and assignment to non-existent values automatically extends @​X (ala lvalue array entries).

BIGGER EXAMPLE​: In the following\, @​_ ends up referencing something that causes a croak from within Perl_sv_setsv ("Bizarre copy of..."). As with all memory corruption\, actual behavior is undefined.

#!/usr/local/bin/perl5.6.1 -w

local @​X;

sub Main { print 'orig​: '\, join('\,'\, @​_)\, "\n"; # 'start'\, 'A'..'M'\, 'end'

@​X = \(\);

           print 'clear​: '\, join\('\,'\, @​\_\)\, "\\n"; \# @​\_ seems ok\, but isn't

my @​z = qw/ MAGIC STRINGS /;

           print 'magic​: '\, join\('\,'\, @​\_\)\, "\\n"; \# oops\, dangling pointer???

require Carp; \# massage heap

           print 'trash​: '\, join\('\,'\, @​\_\)\, "\\n"; \# oops\, more trash from load

my @​a = @​\_; \# crash \(maybe?\) : Bizarre copy of CODE in aassign at \./test2 line 9\.

}

@​X = qw/ A B C D E F G H I J K L M /; Main('start'\, @​X\, 'end');

[Please do not change anything below this line] ----------------------------------------------------------------- --- Flags​: category=core severity=medium --- Site configuration information for perl v5.6.1​:

Configured by software at Wed Dec 12 09​:52​:28 EST 2001.

Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration​: Platform​: osname=hpux\, osvers=10.20\, archname=PA-RISC2.0-multi uname='hp-ux devsrv1 b.10.20 a 9000785 2004553983 two-user license ' config_args='-d -e -Dperl5=/usr/local/bin/perl5 -Dprefix=/usr/local/public/perl-5.6.1 -Uusemymalloc -Uinstallusrbinperl -Dusemultiplicity -Dccflags=-Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit ' hint=recommended\, useposix=true\, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=define useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef Compiler​: cc='cc'\, ccflags =' -Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit -D_HPUX_SOURCE -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 '\, optimize='-O'\, cppflags='-Ae -O +DAportable +DS2.0a +ESlit +Oentrysched +Ofastaccess +Olibcalls +Onolimit -D_HPUX_SOURCE -Aa -I/usr/local/include' ccversion='A.10.32.30'\, gccversion=''\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=4321 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, usemymalloc=n\, prototype=define Linker and Libraries​: ld='ld'\, ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib libs=-lnsl_s -lndbm -lgdbm -lmalloc -ldld -lm -lc -lndir -lcrypt -lsec perllibs=-lnsl_s -lmalloc -ldld -lm -lc -lndir -lcrypt -lsec libc=/lib/libc.sl\, so=sl\, useshrplib=false\, libperl=libperl.a Dynamic Linking​: dlsrc=dl_hpux.xs\, dlext=sl\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-B\,deferred ' cccdlflags='+z'\, lddlflags='-b +vnocompatwarnings -L/usr/local/lib'

Locally applied patches​:

--- @​INC for perl v5.6.1​: /usr/derivs/public/lib/perl /a/vallon/lib/perl /usr/local/public/perl-5.6.1/lib/5.6.1/PA-RISC2.0-multi /usr/local/public/perl-5.6.1/lib/5.6.1 /usr/local/public/perl-5.6.1/lib/site_perl/5.6.1/PA-RISC2.0-multi /usr/local/public/perl-5.6.1/lib/site_perl/5.6.1 /usr/local/public/perl-5.6.1/lib/site_perl .

--- Environment for perl v5.6.1​: HOME=/a/vallon LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/derivs/public/bin​:/a/vallon/jv/bin​:/usr/local/licensed/autosys/HP-UX/autosys/bin​:/a/vallon/bin​:/a/vallon/import/bin​:/a/vallon/public/bin​:/usr/local/bin​:/usr/bin/X11​:/usr/local/bin/X11​:/usr/ucb​:/bin​:/usr/bin​:/opt/softbench/bin​:/usr/bin​:/usr/ccs/bin​:/usr/contrib/bin​:/opt/ansic/bin​:/opt/nettladm/bin​:/opt/pd/bin​:/opt/upgrade/bin​:/usr/bin/X11​:/usr/contrib/bin/X11​:/opt/hparray/bin​:/opt/resmon/bin​:/opt/perf/bin​:/opt/langtools/bin​:/opt/imake/bin​:/opt/aCC/bin​:/opt/fortran/bin​:/usr/local/X11R5/bin​:/usr/atria/bin​:/usr/derivs/prod/bin​:/usr/derivs/apps/bin​:/usr/local/sybase-11.1.1/bin​:/a/derivs/bin PERL5LIB=/usr/derivs/public/lib/perl​:/a/vallon/lib/perl PERL_BADLANG (unset) SHELL=/bin/ksh SHLIB_PATH (unset)

**************************************************************** Bear Stearns is not responsible for any recommendation\, solicitation\, offer or agreement or any information about any transaction\, customer account or account activity contained in this communication. ***********************************************************************

--

Michael G. Schwern \schwern@​pobox\.com http​://www.pobox.com/~schwern/ Perl Quality Assurance \perl\-qa@​perl\.org Kwalitee Is Job One The eye opening delightful morning taste of expired cheese bits in sour milk!

p5pRT commented 22 years ago

From [Unknown Contact. See original ticket]

On Jan 28\, Michael G Schwern said​:

Confirmed from bleadperl back to 5.003_07 at least.

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

  ($x\, $y\, $z) = (1\,2\,3);   foo($x\, $y\, $z);

  sub foo {   print "["\, join("]["\, @​_)\, "]\n"; # [1][2][3]   ($x\, $y\, $z) = ($y\, $z\, $x);   print "["\, join("]["\, @​_)\, "]\n"; # [2][3][1]   }

@​_ is its separate array\, it holds its own elements\, albeit aliases to other scalars. Its size remains the same until you change its size\, but the elements can end up being different when they are changed from their sources\, just as the sources can change when the elements of @​_ are changed​:

  $x = "japhy";   $c = prechop($x);

  sub prechop { substr($_[0]\, 0\, 1\, '') }   print "$c $x\n"; # j aphy

This seems consistent.

-- Jeff "japhy" Pinyan japhy@​pobox.com http​://www.pobox.com/~japhy/ RPI Acacia brother #734 http​://www.perlmonks.org/ http​://www.cpan.org/ ** Look for "Regular Expressions in Perl" published by Manning\, in 2002 ** \ what does y/// stand for? \ why\, yansliterate of course.

p5pRT commented 22 years ago

From @ysth

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

True 'nuf. But look again and explain why changing @​Y affects @​_. (Also note the comment about perl croaking in the OPs second example.)

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}
@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​: start\,A\,B\,C\,D\,E\,end start\,\,\,\,\,\,end start\,magic\,\,\,\,\,end

p5pRT commented 22 years ago

From [Unknown Contact. See original ticket]

On Jan 28\, Yitzchak Scott-Thoennes said​:

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

True 'nuf. But look again and explain why changing @​Y affects @​_. (Also note the comment about perl croaking in the OPs second example.)

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}
@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​: start\,A\,B\,C\,D\,E\,end start\,\,\,\,\,\,end start\,magic\,\,\,\,\,end

Well\, it doesn't appear to futz up in bleadperl...

-- Jeff "japhy" Pinyan japhy@​pobox.com http​://www.pobox.com/~japhy/ RPI Acacia brother #734 http​://www.perlmonks.org/ http​://www.cpan.org/ ** Look for "Regular Expressions in Perl" published by Manning\, in 2002 ** \ what does y/// stand for? \ why\, yansliterate of course.

p5pRT commented 22 years ago

From @schwern

On Mon\, Jan 28\, 2002 at 10​:18​:43PM -0500\, Jeff 'japhy' Pinyan wrote​:

On Jan 28\, Yitzchak Scott-Thoennes said​:

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

True 'nuf. But look again and explain why changing @​Y affects @​_. (Also note the comment about perl croaking in the OPs second example.)

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}
@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​: start\,A\,B\,C\,D\,E\,end start\,\,\,\,\,\,end start\,magic\,\,\,\,\,end

Well\, it doesn't appear to futz up in bleadperl...

It does @​14190.

--

Michael G. Schwern \schwern@​pobox\.com http​://www.pobox.com/~schwern/ Perl Quality Assurance \perl\-qa@​perl\.org Kwalitee Is Job One "His plagiarism was limited only by this faulty technique."   -- Peter Schickele

p5pRT commented 22 years ago

From @iabyn

sub F \{
                      print join\('\,'\, @​\_\)\, "\\n";
    @​X = \(\);
                      print join\('\,'\, @​\_\)\, "\\n";
    @​Y = qw/ magic /;
                      print join\('\,'\, @​\_\)\, "\\n";
\}
@​X = qw/ A B C D E /;
F\('start'\, @​X\, 'end'\);

With the curious output​: start\,A\,B\,C\,D\,E\,end start\,\,\,\,\,\,end start\,magic\,\,\,\,\,end

Yes it's a bug\, and no\, it can't easily be fixed. It's all because stuff pushed onto the stack (and copied into @​_) isn't reference counted.

The individual SVs that make up the elements of @​X start off with a refcnt of 1. They are then pushed onto the stack (still with refcnt == 1)\, then the pointers on the stack are copied into @​_. So now both @​X and @​_ have pointers to the SVs (but with refcnt == 1). Now the elements of @​X are deleted​: since they have a refcnt of 1\, the are freed up. @​_ now contains pointers to SVs in the free SV slabs !!! When a new SV is created ("magic")\, one of the free SVs is grabbed\, which happens to be the same as that pointed to by the first slot in @​_. Bad\, bad\, bad.

Fixed in Perl 6? :-(

p5pRT commented 22 years ago

From [Unknown Contact. See original ticket]

However\, in my sample code\, @​_ is corrupt\, not modified. If you pass an array (@​X) to your foo function\, and the array (@​X) is shortened by side-effect in foo\, then the elements of @​_ that pointed to now-truncated elements of the array point to corrupt memory. You may core dump. That is a bug.

It appears that the aliasing of the elements that occurs when @​_ is created becomes corrupted if the aliases arrays are shorted.

-Justin vallon@​bear.com

-----Original Message----- From​: Jeff 'japhy' Pinyan [SMTP​:jeffp@​crusoe.net] Sent​: Monday\, January 28\, 2002 8​:34 PM To​: Michael G Schwern Cc​: Justin Vallon; perl5-porters@​perl.org Subject​: Re​: [ID 20020128.003] @​_ gets corrupted when F(@​X) shortens @​X

On Jan 28\, Michael G Schwern said​:

Confirmed from bleadperl back to 5.003_07 at least.

I'm not sure this is a bug.

The elements of @​_ are aliases to the elements passed to the function.

  ($x\, $y\, $z) = (1\,2\,3);   foo($x\, $y\, $z);

  sub foo {   print "["\, join("]["\, @​_)\, "]\n"; # [1][2][3]   ($x\, $y\, $z) = ($y\, $z\, $x);   print "["\, join("]["\, @​_)\, "]\n"; # [2][3][1]   }

@​_ is its separate array\, it holds its own elements\, albeit aliases to other scalars. Its size remains the same until you change its size\, but the elements can end up being different when they are changed from their sources\, just as the sources can change when the elements of @​_ are changed​:

  $x = "japhy";   $c = prechop($x);

  sub prechop { substr($_[0]\, 0\, 1\, '') }   print "$c $x\n"; # j aphy

This seems consistent.

-- Jeff "japhy" Pinyan japhy@​pobox.com http​://www.pobox.com/~japhy/ RPI Acacia brother #734 http​://www.perlmonks.org/ http​://www.cpan.org/ ** Look for "Regular Expressions in Perl" published by Manning\, in 2002 ** \ what does y/// stand for? \ why\, yansliterate of course.

*********************************************************************** Bear Stearns is not responsible for any recommendation\, solicitation\, offer or agreement or any information about any transaction\, customer account or account activity contained in this communication. ***********************************************************************