Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.92k stars 549 forks source link

Array deref in return statement causes incorrect value in scalar context #7852

Closed p5pRT closed 16 years ago

p5pRT commented 19 years ago

Migrated from rt.perl.org#34582 (status was 'resolved')

Searchable as RT34582$

p5pRT commented 19 years ago

From perl-bug@owl.me.uk

Created by perl-bug@owl.me.uk

Dear all\,

I have noticed that including an array deref in the return statement of a sub causes unusual results when that sub is called in scalar context.

Take the following script as an example​:

--- start --- #!/usr/bin/perl

use strict; use warnings; use English; local $OFS = ' '; local $ORS = "\n";

sub a {   my $foo = ['dick'\, 'harry'];   my @​bar = ('tom'\, @​$foo);   return @​bar; }

sub b {   my $foo = ['tom'\, 'dick'\, 'harry'];   return @​$foo; }

sub c {   my $foo = ['dick'\, 'harry'];   return ('tom'\, @​$foo); }

sub d {   my $foo = ['tom'\, 'dick'];   return (@​$foo\, 'harry'); }

print a; print b; print c; print d;

print scalar a; print scalar b; print scalar c; print scalar d; --- end ---

The output is as follows​:

--- start --- tom dick harry tom dick harry tom dick harry tom dick harry 3 3 2 harry --- end ---

All four subs return the list ('tom'\, 'dick'\, 'harry') when called in array context. However\, only the first two return the value 3 (what I would expect\, the size of this array) when called in array context.

sub c\, which returns a list containing a scalar and a deref'd array ref\, returns 2 instead of 3. I guessed this might be the compiler noticing that two things are returned by c and optimising the return into something like "return wantarray ? ('tom'\, @​$foo) : 2"\, although I doubt this is the behaviour most users would expect\, since the list returned in array context certainly has 3 elements. If this is the intended behaviour\, I would think that the warnings pragma should generate a warning in this case.

sub d is even more interesting\, since that evaluates to 'harry' in scalar context. I don't know if this is intended\, but again it seems counter-intuitive to me\, and it would\, too\, probably benefit from a warning.

Thanks for listening!

~ Rich

Perl Info ``` Flags: category=core severity=high Site configuration information for perl v5.8.4: Configured by Debian Project at Tue Mar 8 20:31:23 EST 2005. Summary of my perl5 (revision 5 version 8 subversion 4) configuration: Platform: osname=linux, osvers=2.4.27-ti1211, archname=i386-linux-thread-multi uname='linux kosh 2.4.27-ti1211 #1 sun sep 19 18:17:45 est 2004 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.4 -Dsitearch=/usr/local/lib/perl/5.8.4 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.4 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include' ccversion='', gccversion='3.3.5 (Debian 1:3.3.5-9)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libperl.so.5.8.4 gnulibc_version='2.3.2' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib' Locally applied patches: @INC for perl v5.8.4: /etc/perl /usr/local/lib/perl/5.8.4 /usr/local/share/perl/5.8.4 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl . Environment for perl v5.8.4: HOME=/home/rich LANG=en_GB LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/rich/bin:/home/rich/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/usr/lib/xscreensaver:/usr/games:/sbin:/usr/sbin:/usr/lib/xscreensaver:/usr/games:/sbin:/usr/sbin PERL_BADLANG (unset) SHELL=/bin/zsh ```
p5pRT commented 19 years ago

From perl-bug@owl.me.uk

Aha\, I have found something in perlfunc(1) which leads me to understand the strangeness of the behaviour​:

  Because "scalar" is unary operator\, if you accidentally use for   EXPR a parenthesized list\, this behaves as a scalar comma   expression\, evaluating all but the last element in void context   and returning the final element evaluated in scalar context.   This is seldom what you want.

So because the return statements of c and d are "comma expressions"\, value returned by the sub is in fact neither array nor scalar\, it is a comma expression\, so c returns 2 because the array referenced by $foo contains 2 elements.

I must admit that I am surprised that expressions of this type can make their way out of a sub and into the calling scope\, and I still believe that one or the other of the following should happen​:

  1. All subs should return arrays or scalar\, and if a list is returned\, it is cast into an array before it is returned\, or   2. A warning is generated whenever someone tries to return a list rather than an array.

~ Rich

p5pRT commented 19 years ago

perl-bug@owl.me.uk - Status changed from 'new' to 'open'

p5pRT commented 19 years ago

From rick@bort.ca

On Sun\, Mar 27\, 2005 at 12​:30​:19PM -0000\, Rich Daley wrote​:

sub c { my $foo = ['dick'\, 'harry']; return ('tom'\, @​$foo); }

sub d { my $foo = ['tom'\, 'dick']; return (@​$foo\, 'harry'); }

[...] print scalar c; print scalar d; --- end ---

The output is as follows​:

--- start --- 2 harry --- end ---

This is correct behaviour. In scalar context\, the comma operator evaluates each expression in scalar context and returns the last one. So

  ('tom'\, @​$foo);

returns the result of @​$foo in scalar context\, which is 2.

  (@​$foo\, 'harry');

returns the result of 'harry' in scalar context\, which is 'harry'.

-- Rick Delaney rick@​bort.ca

p5pRT commented 19 years ago

@rgs - Status changed from 'open' to 'resolved'

p5pRT commented 19 years ago

From perl5-porters@ton.iguana.be

In article \rt\-3\.0\.11\-34582\-109696\.13\.4721151028463@​perl\.org\,   Rich Daley (via RT) \perlbug\-followup@​perl\.org writes​:

Not a bug.

When you see a construct like   @​a = (1\, 2\, 3) it's not () that makes a list\, it's the comma in list context (caused by the array at the left of the =) that makes the list. The () are only there for precedence\, so that the comma operators get executed before the assign.

In

return (foo\, bar\, baz)

the () are unneeded since the precedence is such that the comma operators get executed first anyways\, so it is equivalent to

return foo\, bar\, baz

Like every perl operator\, "\," is really two different operators\, one for scalar context\, and one for list context. Your examples will call the functions in scalar context\, so it's the scalar context comma that counts​: it will evaluate each piece in scalar context and return the last value. So we get​:

sub a {   my $foo = ['dick'\, 'harry'];   my @​bar = ('tom'\, @​$foo);   return @​bar; }

@​bar is executed in scalar context => you get 3

sub c {   my $foo = ['dick'\, 'harry'];   return ('tom'\, @​$foo); }

sub c\, which returns a list containing a scalar and a deref'd array ref\, returns 2 instead of 3. I guessed this might be the compiler

Nope. sub c doesn't return a list. There is no such thing as returning a list in scalar context. The () is irrelevant as explained above.

noticing that two things are returned by c and optimising the return into something like "return wantarray ? ('tom'\, @​$foo) : 2"\, although I doubt this is the behaviour most users would expect\, since the list returned in array context certainly has 3 elements. If this is the intended behaviour\, I would think that the warnings pragma should generate a warning in this case.

sub c executed in scalar context will do 'tom'\, @​$foo in scalar context\, so the first value is evalled and thrown away\, then @​$foo is evalled in scalar context => you get 2

sub d {   my $foo = ['tom'\, 'dick'];   return (@​$foo\, 'harry'); }

sub d is even more interesting\, since that evaluates to 'harry' in scalar context. I don't know if this is intended\, but again it seems counter-intuitive to me\, and it would\, too\, probably benefit from a warning.

@​$foo\, 'harry' is executed in scalar context. First it executes @​$foo in scalar context (giving 2) which is thrown away. Then it executus 'harry' in scalar context => you get 'harry'

p5pRT commented 19 years ago

From RandyS@ThePierianSpring.org

Rich Daley (via RT) wrote​:

# New Ticket Created by Rich Daley # Please include the string​: [perl #34582] # in the subject line of all future correspondence about this issue. # \<URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=34582 >

This is a bug report for perl from perl-bug@​owl.me.uk\, generated with the help of perlbug 1.35 running under perl v5.8.4.

----------------------------------------------------------------- [Please enter your report here]

Dear all\,

I have noticed that including an array deref in the return statement of a sub causes unusual results when that sub is called in scalar context.

Take the following script as an example​:

--- start --- #!/usr/bin/perl

use strict; use warnings; use English; local $OFS = ' '; local $ORS = "\n";

sub a { my $foo = ['dick'\, 'harry']; my @​bar = ('tom'\, @​$foo); return @​bar; }

sub b { my $foo = ['tom'\, 'dick'\, 'harry']; return @​$foo; }

sub c { my $foo = ['dick'\, 'harry']; return ('tom'\, @​$foo); }

sub d { my $foo = ['tom'\, 'dick']; return (@​$foo\, 'harry'); }

print a; print b; print c; print d;

print scalar a; print scalar b; print scalar c; print scalar d; --- end ---

The output is as follows​:

--- start --- tom dick harry tom dick harry tom dick harry tom dick harry 3 3 2 harry --- end ---

All four subs return the list ('tom'\, 'dick'\, 'harry') when called in array context. However\, only the first two return the value 3 (what I would expect\, the size of this array) when called in array context.

sub c\, which returns a list containing a scalar and a deref'd array ref\, returns 2 instead of 3. I guessed this might be the compiler noticing that two things are returned by c and optimising the return into something like "return wantarray ? ('tom'\, @​$foo) : 2"\, although I doubt this is the behaviour most users would expect\, since the list returned in array context certainly has 3 elements. If this is the intended behaviour\, I would think that the warnings pragma should generate a warning in this case.

sub d is even more interesting\, since that evaluates to 'harry' in scalar context. I don't know if this is intended\, but again it seems counter-intuitive to me\, and it would\, too\, probably benefit from a warning.

The return statement in sub c and sub d\, despite appearances is not operation on a list. It's operating on an expression which involves the comma operator. The comma operator evaluates to its right-most value.

return ('tom'\, @​$foo); # same as​: return @​$foo; return (@​$foo\, 'harry'); # same as​: return 'harry';

You might try something like​:

return @​{[ 'tom'\, @​$foo ]}; return @​{[ @​$foo\, 'harry' ]};

This behavior is consistent going back to 5.005 and with perl's C-like roots.

p5pRT commented 19 years ago

From perl-bug@owl.me.uk

Apologies in advance for reopening the ticket.

I understand now that this is the intended behaviour. However\, even perlfunc(1) points out that evaluating the comma in scalar context is "seldom what you want"\, so would it not make sense to include a warning in the warnings pragma for either or both of the following cases​:

  1. If a comma is evaluated in scalar context\, or   2. if a sub returns a comma expression\, since this could produce unexpected results.

I understand that some people would want to use this legitimately\, but I suspect that the majority of us would simply be confused by this slightly unusual behaviour. Somewhat more intuitive behaviour\, such as the behaviour when attempting to use an undefined value in string 'eq'\, does produce warnings under the pragma.

Thanks for your time!

~ Rich

p5pRT commented 19 years ago

perl-bug@owl.me.uk - Status changed from 'resolved' to 'open'

p5pRT commented 19 years ago

From @ysth

On Sun\, Mar 27\, 2005 at 09​:11​:12AM -0500\, Rick Delaney wrote​:

On Sun\, Mar 27\, 2005 at 12​:30​:19PM -0000\, Rich Daley wrote​:

sub c { my $foo = ['dick'\, 'harry']; return ('tom'\, @​$foo); }

sub d { my $foo = ['tom'\, 'dick']; return (@​$foo\, 'harry'); }

[...] print scalar c; print scalar d; --- end ---

The output is as follows​:

--- start --- 2 harry --- end ---

This is correct behaviour. In scalar context\, the comma operator evaluates each expression in scalar context and returns the last one.

Quite correct. I actually would rather scalar context comma evaluated it's right operand in void context\, not scalar\, but it's probably too late to do that. In void context\, comma operator does evaluate both operands in void context. The documentation (shown below for easy reference) could use some clarification on this point\, but I hesitate to add anything given that perlop and perlfunc largely don't mention "void context" at all\, and often mean "scalar or void context" when they say "scalar context".

Note the results from this​:

$ perl -wle'sub ctx{print wantarray?"list"​:defined wantarray?"scalar"​:"void"} sub foo { ctx\,ctx\,ctx } print "void call"; foo for 1; print "scalar call"; grep foo\, 1; print "list call"; map foo\, 1' void call void void void scalar call scalar scalar scalar list call list list list

=head2 Comma Operator

Binary "\," is the comma operator. In scalar context it evaluates its left argument\, throws that value away\, then evaluates its right argument and returns that value. This is just like C's comma operator.

In list context\, it's just the list argument separator\, and inserts both its arguments into the list.

p5pRT commented 19 years ago

From rick@bort.ca

On Sun\, Mar 27\, 2005 at 12​:13​:12PM -0800\, Yitzchak Scott-Thoennes wrote​:

On Sun\, Mar 27\, 2005 at 09​:11​:12AM -0500\, Rick Delaney wrote​:

This is correct behaviour. In scalar context\, the comma operator evaluates each expression in scalar context and returns the last one.

Quite correct. I actually would rather scalar context comma evaluated it's right operand in void context\, not scalar\, but it's probably too late to do that.

That wouldn't be very useful. Then there wouldn't be any result from the last expression to return.

-- Rick Delaney rick@​bort.ca

p5pRT commented 19 years ago

From @ysth

On Sun\, Mar 27\, 2005 at 04​:44​:21PM -0500\, Rick Delaney wrote​:

On Sun\, Mar 27\, 2005 at 12​:13​:12PM -0800\, Yitzchak Scott-Thoennes wrote​:

On Sun\, Mar 27\, 2005 at 09​:11​:12AM -0500\, Rick Delaney wrote​:

This is correct behaviour. In scalar context\, the comma operator evaluates each expression in scalar context and returns the last one.

Quite correct. I actually would rather scalar context comma evaluated it's right operand in void context\, not scalar\, but it's probably too late to do that.

That wouldn't be very useful. Then there wouldn't be any result from the last expression to return.

Arrg. s/right/left/

p5pRT commented 19 years ago

From @ysth

On Sun\, Mar 27\, 2005 at 12​:13​:12PM -0800\, Yitzchak Scott-Thoennes wrote​:

Quite correct. I actually would rather scalar context comma evaluated it's right operand in void context\, not scalar\, but it's probably too s/right/left/ late to do that. In void context\, comma operator does evaluate both operands in void context. The documentation (shown below for easy reference) could use some clarification on this point\, but I hesitate to add anything given that perlop and perlfunc largely don't mention "void context" at all\, and often mean "scalar or void context" when they say "scalar context".

Note the results from this​:

$ perl -wle'sub ctx{print wantarray?"list"​:defined wantarray?"scalar"​:"void"} sub foo { ctx\,ctx\,ctx } print "void call"; foo for 1; print "scalar call"; grep foo\, 1; print "list call"; map foo\, 1' void call void void void scalar call scalar scalar scalar list call list list list

Hmm\, there appears to be a bug in the above. When the context is known at compile-time\, the left operand of scalar-context comma does get void context​:

$ perl -wle'sub ctx{print wantarray?"list"​:defined wantarray?"scalar"​:"void"} 0+(ctx\,ctx\,ctx)' Useless use of addition (+) in void context at -e line 1. void void scalar

p5pRT commented 16 years ago

p5p@spam.wizbit.be - Status changed from 'open' to 'resolved'