Open p5pRT opened 17 years ago
Hi all\,
There is a problem with the orassign (||=) implementation (though the same is probably true for other similar operators) when the right hand side expression "deletes" what is being referenced by the lvalue\, and the result is used (though I'm not 100% sure about that last part).
Test case:
-- begin --
my @array;
sub fn { @array = (); }
$array[0] = 1;
my $var = $array[1] ||= fn();
-- end --
On my platform this causes a segfault during execution. In some other instances it will appear to work\, but the copy SV will be bogus and will usually cause a crash later on (if it doesn't segfault right away\, a Dump or Dumper on $var should crash).
The fix is probably to increment the refcount on the lvalue before evaluating the right hand side (though I'm not sure this would give the right result?)\, or maybe to re-"evaluate" (dereference?) the lvalue after the right hand site has been evaluated (like a regular sassign would do?). Not familiar enough with perl internals to really say more...
jc@oxado.com (via RT) wrote:
Test case:
-- begin --
my @array;
sub fn { @array = (); }
$array[0] = 1;
my $var = $array[1] ||= fn();
-- end --
On my platform this causes a segfault during execution.
Segfault confirmed with 5.8.8\, 5.6.2 and 5.5.5 maint. bleadperl produces...
panic: attempt to copy value 0 to a freed scalar 1804cc0 at - line 10.
Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=darwin\, osvers=8.8.1\, archname=darwin-thread-multi-2level uname='darwin windhund.local 8.8.1 darwin kernel version 8.8.1: mon sep 25 19:42:00 pdt 2006; root:xnu-792.13.8.obj~1release_i386 i386 i386 macbook1\,1 darwin ' config_args='-Dprefix=/usr/local/perl/5.8.8 -Duseithreads -Dccflags=-I/sw/include -Dldflags=-L/sw/lib -Dperladmin=schwern@pobox.com -Dcf_email=schwern@pobox.com -Dmyhostname=windhund -Dmydomain=.schwern.org -Dprivlib=/usr/local/perl/5.8.8/lib -Dsiteprefix=/usr/local -Dsitelib=/usr/local/lib/site_perl -Dlibpth=/usr/local/lib /sw/lib /opt/local/lib /usr/lib -Dnetdb_host_type=const char * -Dnetdb_hlen_type=socklen_t -Uinstallusrbinperl' hint=recommended\, useposix=true\, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-I/sw/include -fno-common -DPERL_DARWIN -no-cpp-precomp -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/opt/local/include'\, optimize='-O3'\, cppflags='-no-cpp-precomp -I/sw/include -fno-common -DPERL_DARWIN -no-cpp-precomp -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/opt/local/include' ccversion=''\, gccversion='4.0.1 (Apple Computer\, Inc. build 5363)'\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'\, ldflags ='-L/sw/lib -L/usr/local/lib -L/opt/local/lib' libpth=/usr/local/lib /sw/lib /opt/local/lib /usr/lib libs=-lgdbm -ldbm -ldl -lm -lc perllibs=-ldl -lm -lc libc=/usr/lib/libc.dylib\, so=dylib\, useshrplib=false\, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=bundle\, d_dlsymun=undef\, ccdlflags=' ' cccdlflags=' '\, lddlflags='-L/sw/lib -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib'
Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API Built under darwin Compiled at Oct 14 2006 18:50:36 %ENV: PERL5LIB="/sw/lib/perl5:/sw/lib/perl5/darwin" @INC: /sw/lib/perl5 /sw/lib/perl5/darwin /usr/local/perl/5.8.8/lib/darwin-thread-multi-2level /usr/local/perl/5.8.8/lib /usr/local/lib/site_perl/5.8/darwin-thread-multi-2level /usr/local/lib/site_perl /usr/local/lib/site_perl .
The RT System itself - Status changed from 'new' to 'open'
On Tue\, 03 Jul 2007 09:31:32 -0700\, "jc@oxado.com (via RT)" \perlbug\-followup@​perl\.org wrote:
There is a problem with the orassign (||=) implementation (though the same is probably true for other similar operators) when the right hand side expression "deletes" what is being referenced by the lvalue\, and the result is used (though I'm not 100% sure about that last part).
Same for dor-assign:
# cat xx.pl #!/pro/bin/perl
use strict; use warnings;
my @array;
sub fn { @array = (); }
$array[0] = 1;
my $var = $array[1] //= fn (); #
perl-5.8.8-dor
# perl xx.pl panic: sv_upgrade to unknown type 255 at xx.pl line 15.
perl-5.9.x-31526
# bleadperl xx.pl panic: attempt to copy value 0 to a freed scalar 81583ec at xx.pl line 15.
-- H.Merijn Brand Amsterdam Perl Mongers (http://amsterdam.pm.org/) using & porting perl 5.6.2\, 5.8.x\, 5.9.x on HP-UX 10.20\, 11.00\, 11.11\, & 11.23\, SuSE 10.0 & 10.2\, AIX 4.3 & 5.2\, and Cygwin. http://qa.perl.org http://mirrors.develooper.com/hpux/ http://www.test-smoke.org http://www.goldmark.org/jeff/stupid-disclaimers/
On Tue\, Jul 03\, 2007 at 02:01:16PM -0700\, Michael G Schwern wrote:
Segfault confirmed with 5.8.8\, 5.6.2 and 5.5.5 maint. bleadperl produces...
panic: attempt to copy value 0 to a freed scalar 1804cc0 at - line 10.
I believe this is the 4\,527th reported bug that is caused by the perl stack not being refcounted. Ah\, the joys of premature optimisation.
Making the stack be refcounted is something on my vague list of "things I might look into but probably won't because something more easy or interesting has popped up in the meantime" for 5.12.
-- "You may not work around any technical limitations in the software" -- Windows Vista license
On Jul 3\, 2007\, at 3:12 PM\, Dave Mitchell wrote:
On Tue\, Jul 03\, 2007 at 02:01:16PM -0700\, Michael G Schwern wrote:
Segfault confirmed with 5.8.8\, 5.6.2 and 5.5.5 maint. bleadperl
produces...panic: attempt to copy value 0 to a freed scalar 1804cc0 at - line
10.I believe this is the 4\,527th reported bug that is caused by the perl stack not being refcounted. Ah\, the joys of premature optimisation.
Making the stack be refcounted is something on my vague list of
"things I might look into but probably won't because something more easy or interesting has popped up in the meantime" for 5.12.
Is that optimisation premature? :)
artur
In article \20070703221234\.GA29232@​iabyn\.com\, davem@iabyn.com (Dave Mitchell) wrote:
On Tue\, Jul 03\, 2007 at 02:01:16PM -0700\, Michael G Schwern wrote:
Segfault confirmed with 5.8.8\, 5.6.2 and 5.5.5 maint. bleadperl produces...
panic: attempt to copy value 0 to a freed scalar 1804cc0 at - line 10.
I believe this is the 4\,527th reported bug that is caused by the perl stack not being refcounted. Ah\, the joys of premature optimisation.
Making the stack be refcounted is something on my vague list of "things I might look into but probably won't because something more easy or interesting has popped up in the meantime" for 5.12.
Can it even be done without breaking old XS modules?
Also in the "wouldn't it be nice if" category: refcounting the inputs to foreach\, map\, grep\, and sort.
/Bo Lindbergh
On Wed\, Jul 04\, 2007 at 12:25:09PM +0200\, Bo Lindbergh wrote:
In article \20070703221234\.GA29232@​iabyn\.com\, davem@iabyn.com (Dave Mitchell) wrote:
On Tue\, Jul 03\, 2007 at 02:01:16PM -0700\, Michael G Schwern wrote:
Segfault confirmed with 5.8.8\, 5.6.2 and 5.5.5 maint. bleadperl produces...
panic: attempt to copy value 0 to a freed scalar 1804cc0 at - line 10.
I believe this is the 4\,527th reported bug that is caused by the perl stack not being refcounted. Ah\, the joys of premature optimisation.
Making the stack be refcounted is something on my vague list of "things I might look into but probably won't because something more easy or interesting has popped up in the meantime" for 5.12.
Can it even be done without breaking old XS modules?
I envison duplicating the stack frame for old XS - ie just before calling into an XS function\, do a block copy of the top N addresses on the stack - so the function gets its own non-refcounted list of SVs to play with and dispose of.
Also in the "wouldn't it be nice if" category: refcounting the inputs to foreach\, map\, grep\, and sort.
That would mostly happen automatically if the stack was refcounted. The only one that wouldn't is for (@array)\, which optimises away pushing all the elements of @array onto the stack.
Anyway\, I haven't given this much thought yet (and don't intend to yet) - recounting the stack may well not be a viable scheme.
-- You live and learn (although usually you just live).
In article \20070704113158\.GD31152@​iabyn\.com\, davem@iabyn.com (Dave Mitchell) wrote:
On Wed\, Jul 04\, 2007 at 12:25:09PM +0200\, Bo Lindbergh wrote:
In article \20070703221234\.GA29232@​iabyn\.com\, davem@iabyn.com (Dave Mitchell) wrote:
Making the stack be refcounted is something on my vague list of "things I might look into but probably won't because something more easy or interesting has popped up in the meantime" for 5.12.
Also in the "wouldn't it be nice if" category: refcounting the inputs to foreach\, map\, grep\, and sort.
That would mostly happen automatically if the stack was refcounted. The only one that wouldn't is for (@array)\, which optimises away pushing all the elements of @array onto the stack.
That case seems to work fine already. The example below generates no errors or warnings and outputs "ACE\n".
{ use strict; use warnings;
my @arr=qw(A B C D E);
foreach my $el (@arr) { splice(@arr\,0\,1); print $el; } print "\n"; }
/Bo Lindbergh
In article \20070704113158\.GD31152@​iabyn\.com\, davem@iabyn.com (Dave Mitchell) wrote:
On Wed\, Jul 04\, 2007 at 12:25:09PM +0200\, Bo Lindbergh wrote:
In article \20070703221234\.GA29232@​iabyn\.com\, davem@iabyn.com (Dave Mitchell) wrote:
Making the stack be refcounted is something on my vague list of "things I might look into but probably won't because something more easy or interesting has popped up in the meantime" for 5.12.
Also in the "wouldn't it be nice if" category: refcounting the inputs to foreach\, map\, grep\, and sort.
That would mostly happen automatically if the stack was refcounted. The only one that wouldn't is for (@array)\, which optimises away pushing all the elements of @array onto the stack.
That case seems to work fine already. The example below generates no errors or warnings and outputs "ACE\n".
{ use strict; use warnings;
my @arr=qw(A B C D E);
foreach my $el (@arr) { splice(@arr\,0\,1); print $el; } print "\n"; }
/Bo Lindbergh
orassign (and probably similar things in perl\, maybe //=?) does not keep a reference to the lvalue\, which can be freed in the rvalue expression\, corrupting memory\, e.g.:
my %hash = (a => 0); my @array; my $w = \$hash{a};
$hash{a} ||= do { %hash=(); 5 };
use Data::Dumper; warn Dumper \%hash;
This will output an empty hash\, which isn't a big problem in itself\, but the problem is that the ||= will assign the 5 to a freed scalar (I think).
I have not found a good testcase to prove this theory\, but somethign is clearly broken\, as this example segfaults here (with both debian lenny's and my own perl)\, and clearly shouldn't:
my %hash = (a => 0); my $array;
$hash{a} ||= do { %hash=(); $array=[1..100]; 5 };
warn join ":"\, @$array;
I think the expected behaviour (on the perl level) would be that orassign would evaluate the lvalue twice\, i.e. $a ||= $b is the same as:
$a = $b unless $a;
I think this would be slow/ugly/annoying to implement. If orassign would keep a reference to the lvalue\, then the hash would stay empty\, but at least perl wouldn't corrupt it's own memory.
(the problem comes uo naturally when you have some kind of cache in %hash and a function that updates the cache which can call itself recursively\, it's not entirely academic\, and cost me quite a few days to debug\, as the symptoms are random memory corruption at completely unrelated places :-).
$ perl-5.9.2 rt-66448.pl 1:[...]:100 at rt-66448.pl line 10.
$ perl-5.9.3 rt-66448.pl 1:[...]:100 at rt-66448.pl line 10. Segmentation fault
(bisect will follow later)
p5p@spam.wizbit.be - Status changed from 'new' to 'open'
$hash{a} is undefined on every version of perl I tested.
Not sure how useful this bisect is...
----Program---- #!/usr/bin/perl -l my %hash = (a => 0); my $array;
$hash{a} ||= do { %hash=(); $array=[1..100]; 5 };
print join ":"\, @$array;
----Output of .../plLVm6w/perl-5.9.2@24615/bin/perl---- 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20:21:22:23:24:25:26:27:28:29:30:31:32:33:34:35:36:37:38:39:40:41:42:43:44:45:46:47:48:49:50:51:52:53:54:55:56:57:58:59:60:61:62:63:64:65:66:67:68:69:70:71:72:73:74:75:76:77:78:79:80:81:82:83:84:85:86:87:88:89:90:91:92:93:94:95:96:97:98:99:100
----EOF ($?='0')---- ----Output of .../pYR6QKD/perl-5.9.2@24617/bin/perl----
----EOF ($?='11')---- Need a perl between 24615 and 24617
http://perl5.git.perl.org/perl.git/commit/ 311a25d9bdebead7c924f8195b0287fe2a60a568
author Nicholas Clark \nick@​ccl4\.org Sun\, 29 May 2005 17:46:39 +0000 (17:46 +0000) committer Nicholas Clark \nick@​ccl4\.org Sun\, 29 May 2005 17:46:39 +0000 (17:46 +0000) commit 311a25d9bdebead7c924f8195b0287fe2a60a568 tree 76ec442d5d0e9d206c50c1c2c9b052e44146b77a tree | snapshot parent b08468124c2de39dce34c2652895d4c7023c7c4d commit | diff
Change the IV to a union. Revert the NV union back to a plain NV Transpose the positions of IV and NV (NV is now first) Don't allocate the NV for PV\,PVIV\,PVAV and PVHV (last 2 non-allocations currently disabled by default)
p4raw-id: //depot/perl@24617
Migrated from rt.perl.org#43501 (status was 'open')
Searchable as RT43501$