Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.95k stars 554 forks source link

Unexpected parsing behavior depending on number of parts in string concatination #15594

Open p5pRT opened 8 years ago

p5pRT commented 8 years ago

Migrated from rt.perl.org#129233 (status was 'open')

Searchable as RT129233$

p5pRT commented 8 years ago

From sullr@cpan.org

Created by sullr@cpan.org

The following statement gets successfully parsed and executed by perl. Note that at the end of line 3 there is a '.' instead of a ';'\, i.e. the author of the code made obviously an error (based on real bug).

  1 my $x = 1;   2 my $y = 'a';   3 warn "foobar $y " . "foo " .   4 $x = 0;

  > 0 at x.pl line 3.

If the code is modified so that the first concationation is skipped the behavior is completly different but actually the expected behavior (i.e. error)​:

  1 my $x = 1;   2 my $y = 'a';   3 warn "foobar $y foo " .   4 $x = 0;

  > Can't modify concatenation (.) or string in scalar assignment at x.pl line 4\, near "0;"   > Execution of x.pl aborted due to compilation errors.

The same behavior can be seen with assignments\, i.e.

  my $z = "foobar $y " . "foo " . $x = 0; # works\, $z is 0   my $z = "foobar $y foo " . $x = 0; # error

It also works if the variable is outside the quotes or even if no quotes at all are used but an additional variable.

  my $z = $x = 0; # works as expected   my $z = $y.$x = 0; # fails as expected   my $z = "".$y.$x = 0; # works unexpectedly   my $z = $y."".$x = 0; # works unexpectedly   my $z = $v.$y.$x = 0; # works unexpectedly

This behavior exists in a wide range of perl versions. I found it in perl 5.8.1 and it is still in the current blead perl 5.25.5.

Perl Info ``` Flags: category=core severity=medium Site configuration information for perl 5.25.5: Configured by work at Thu Sep 8 20:16:29 CEST 2016. Summary of my perl5 (revision 5 version 25 subversion 5) configuration: Snapshot of: 49fc490652d8b428d67872fae3acb10f0b43cff7 Platform: osname=linux osvers=4.4.0-34-generic archname=x86_64-linux uname='linux rincewind 4.4.0-34-generic #53~14.04.1-ubuntu smp wed jul 27 16:56:40 utc 2016 x86_64 x86_64 x86_64 gnulinux ' config_args='-de -Dprefix=/home/work-public/perl/perls/perl-blead -Dusedevel -Aeval:scriptdir=/home/work-public/perl/perls/perl-blead/bin' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2' cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='' gccversion='4.8.4' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc= so=so useshrplib=false libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector' @INC for perl 5.25.5: /home/work-public/perl/perls/perl-blead/lib/site_perl/5.25.5/x86_64-linux /home/work-public/perl/perls/perl-blead/lib/site_perl/5.25.5 /home/work-public/perl/perls/perl-blead/lib/5.25.5/x86_64-linux /home/work-public/perl/perls/perl-blead/lib/5.25.5 Environment for perl 5.25.5: HOME=/home/work LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ADDRESS=de_DE.UTF-8 LC_IDENTIFICATION=de_DE.UTF-8 LC_MEASUREMENT=de_DE.UTF-8 LC_MONETARY=de_DE.UTF-8 LC_NAME=de_DE.UTF-8 LC_NUMERIC=de_DE.UTF-8 LC_PAPER=de_DE.UTF-8 LC_TELEPHONE=de_DE.UTF-8 LC_TIME=de_DE.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/work-public/perl/bin:/home/work-public/perl/perls/perl-blead/bin:/home/Shared/anaconda2/bin:/home/Shared/anaconda3/bin:/home/work/bin/:/home/work/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games PERLBREW_BASHRC_VERSION=0.64 PERLBREW_HOME=/home/work/.perlbrew PERLBREW_MANPATH=/home/work-public/perl/perls/perl-blead/man PERLBREW_PATH=/home/work-public/perl/bin:/home/work-public/perl/perls/perl-blead/bin PERLBREW_PERL=perl-blead PERLBREW_ROOT=/home/work-public/perl PERLBREW_VERSION=0.66 PERL_BADLANG (unset) SHELL=/bin/bash ```
p5pRT commented 8 years ago

From @iabyn

On Thu\, Sep 08\, 2016 at 12​:24​:52PM -0700\, sullr@​cpan.org wrote​:

The same behavior can be seen with assignments\, i.e.

my $z = "foobar $y " . "foo " . $x = 0; # works\, $z is 0 my $z = "foobar $y foo " . $x = 0; # error

It can be reduced to the following difference​:

  (($x . "a") . "b") = 0; # compiles (bug)   (($x + 1 ) + 2 ) = 0; # error (correct behaviour)

-- "Strange women lying in ponds distributing swords is no basis for a system of government. Supreme executive power derives from a mandate from the masses\, not from some farcical aquatic ceremony."   -- Dennis\, "Monty Python and the Holy Grail"

p5pRT commented 8 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 8 years ago

From zefram@fysh.org

Dave Mitchell wrote​:

It can be reduced to the following difference​:

(($x . "a") . "b") = 0; # compiles (bug) (($x + 1 ) + 2 ) = 0; # error (correct behaviour)

Where a concat op has a lhs that is another concat op\, ck_concat sets the OPf_STACKED flag on the outer concat. Addition doesn't get that treatment. It seems that the flag is intended as a signal to the peephole optimiser\, where it suppresses an optimisation that I can't make sense of. But op_lvalue_flags() also looks at OPf_STACKED\, for concat\, add\, and many other non-lvalue op types\, and quite explicitly suppresses the "can't modify" error if it's set. Presumably there's something else that I haven't identified that sets the flag on these ops in some situation that makes them an acceptable lvalue. The two uses of the flag are interfering with each other. Neither is the standard meaning of the flag\, and neither is described in the comment in op.h that describes the flag.

-zefram

p5pRT commented 8 years ago

From @cpansprout

On Fri Sep 09 03​:47​:58 2016\, zefram@​fysh.org wrote​:

Dave Mitchell wrote​:

It can be reduced to the following difference​:

(($x . "a") . "b") = 0; # compiles (bug) (($x + 1 ) + 2 ) = 0; # error (correct behaviour)

Where a concat op has a lhs that is another concat op\, ck_concat sets the OPf_STACKED flag on the outer concat. Addition doesn't get that treatment.

$a.$b.$c is optimised to ($a.$b).=$c to avoid having to copy the value returned by $a.$b to a new temporary scalar to be returned by $c.

It seems that the flag is intended as a signal to the peephole optimiser\, where it suppresses an optimisation that I can't make sense of.

It seems that the ($a.$b).=$c optimisation can interfere in some cases with the removal of the stringify op in "$a$b$c" in some circumstances involving lexical scalar assignment. I don’t understand why though. That checking of the flag in the peephole optimizer is not directly related to this bug.

But op_lvalue_flags() also looks at OPf_STACKED\, for concat\, add\, and many other non-lvalue op types\, and quite explicitly suppresses the "can't modify" error if it's set. Presumably there's something else that I haven't identified that sets the flag on these ops in some situation that makes them an acceptable lvalue.

All the assignment versions of binary ops\, .= += -= etc.\, have the OPf_STACKED flag set.

--

Father Chrysostomos