Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.96k stars 555 forks source link

%+ not scoped properly #15638

Open p5pRT opened 8 years ago

p5pRT commented 8 years ago

Migrated from rt.perl.org#129771 (status was 'open')

Searchable as RT129771$

p5pRT commented 8 years ago

From @nkh

Created by @nkh

Hi\,

based on perlre​:

These special variables\, like the %+ hash and the numbered match variables ($1 \, $2 \, $3 \, etc.) are dynamically scoped until the end of the enclosing block or until the next successful match\, whichever comes first.

the following code parse a string which has the form {block1}* text* {block2} where block may be empty.

I use named capture (b1\, b2) to capture the blocks.

for each test case the following is display

test case

block1 text block2 text text

The problem can already be seen within the first iterations where $+{b1} should be undefined but has kept the previous value

adding an extra block fixes the problem

for (qw( ? {a} a{}? ?a{} ?{a} {a}?{b} {a}{b}{c} {a}{b}{c}{d} {a}bc{d} {a}bcd {?} { } {{} {a}{{{{{{b} {a}{{{{{{b {a}{{{{{{ {a}}}}}}{b} {a}}}}}}{b {a}}}}}}b} )) { #{{{ # perl bug fix print " $_\n" ; my $args = $_ ;

$args =~ s/^ (?'b1' (?​:\{[^\{\}]*\}) ) //x ; print '>' . ($+{b1} // '') . "\n" ;

$args = reverse $args ;

$args =~ s/^ (?'b2' (?​:\}[^\{\}]*\{) ) //x ; print '>' . reverse($+{b2} // '') . "\n" ;

my $t = reverse $args; print ">$t\n" ;

print "-----------------\n\n" ; #}}} }

Perl Info ``` Flags: category=core severity=high Site configuration information for perl 5.22.1: Configured by Debian Project at Sun Mar 13 11:54:18 UTC 2016. Summary of my perl5 (revision 5 version 22 subversion 1) configuration: Platform: osname=linux, osvers=3.16.0, archname=x86_64-linux-gnu-thread-multi uname='linux localhost 3.16.0 #1 smp debian 3.16.0 x86_64 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dcc=x86_64-linux-gnu-gcc -Dcpp=x86_64-linux-gnu-cpp -Dld=x86_64-linux-gnu-gcc -Dccflags=-DDEBIAN -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions -Wl,-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.22 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.22 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.22 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.22.1 -Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.22.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -dEs -Duseshrplib -Dlibperl=libperl.so.5.22.1' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='x86_64-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -g', cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='5.3.1 20160311', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='x86_64-linux-gnu-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=libc-2.21.so, so=so, useshrplib=true, libperl=libperl.so.5.22 gnulibc_version='2.21' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector-strong' Locally applied patches: DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN. DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove overly restrictive DB_File version check. DEBPKG:debian/doc_info - Replace generic man(1) instructions with Debian-specific information. DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories. DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes. DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking DEBPKG:fixes/respect_umask - Respect umask during installation DEBPKG:debian/writable_site_dirs - Set umask approproately for site install directories DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of libperl.a under /usr/lib DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets. DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor. DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy. DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable. DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need. DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038] http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/747628 Point users to Debian packages of deprecated core modules DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the upstream Git repository DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.22.1-9 in patchlevel.h DEBPKG:debian/skip-kfreebsd-crash - http://bugs.debian.org/628493 [perl #96272] Skip a crashing test case in t/op/threads.t on GNU/kFreeBSD DEBPKG:fixes/document_makemaker_ccflags - http://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags} DEBPKG:debian/find_html2text - http://bugs.debian.org/640479 Configure CPAN::Distribution with correct name of html2text DEBPKG:debian/perl5db-x-terminal-emulator.patch - http://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5db.pl DEBPKG:debian/cpan-missing-site-dirs - http://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with nonexisting site dirs if a parent is writable DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790] http://bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option not respected DEBPKG:debian/regen-skip - Skip a regeneration check in unrelated git repositories DEBPKG:debian/makemaker-pasthru - http://bugs.debian.org/758471 Pass LD settings through to subdirectories DEBPKG:fixes/pod_man_reproducible_date - http://bugs.debian.org/759405 Support POD_MAN_DATE in Pod::Man for the left-hand footer DEBPKG:debian/locale-robustness - http://bugs.debian.org/782068 [perl #124310] Make t/run/locale.t survive missing locales masked by LC_ALL DEBPKG:fixes/podman-utc - http://bugs.debian.org/780259 Make the embedded date from Pod::Man reproducible DEBPKG:fixes/podman-utc-docs - http://bugs.debian.org/780259 Documentation and test suite updates for UTC fix DEBPKG:fixes/podman-empty-date - http://bugs.debian.org/780259 Support an empty POD_MAN_DATE environment variable DEBPKG:fixes/podman-pipe - http://bugs.debian.org/777405 Better errors for man pages from standard input DEBPKG:debian/pod2man-customized - Update porting/customized.dat for pod2man modifications DEBPKG:debian/makemaker-manext - http://bugs.debian.org/247370 Make EU::MakeMaker honour MANnEXT settings in generated manpage headers DEBPKG:debian/makemaker_customized - Update t/porting/customized.dat for files patched in Debian DEBPKG:debian/do-not-record-build-date - [6baa8db] http://bugs.debian.org/774422 [perl #125830] Allow overriding the compile time in "perl -V" output DEBPKG:fixes/podman-source-date-epoch - http://bugs.debian.org/801621 Make Pod::Man honor the SOURCE_DATE_EPOCH environment variable DEBPKG:fixes/podman-source-date-epoch-cleanups - http://bugs.debian.org/801621 Coding style and documentation for SOURCE_EPOCH_DATE DEBPKG:fixes/podman-source-date-epoch-testfix - http://bugs.debian.org/807086 Guard for building with SOURCE_DATE_EPOCH or POD_MAN_DATE set DEBPKG:debian/devel-ppport-reproducibility - http://bugs.debian.org/801523 Sort the list of XS code files when generating RealPPPort.xs DEBPKG:fixes/encode-unicode-bom - http://bugs.debian.org/798727 [ rt.cpan.org #107043] Address https://rt.cpan.org/Public/Bug/Display.html?id=107043 DEBPKG:debian/encode-unicode-bom-doc - http://bugs.debian.org/798727 Document Debian backport of Encode::Unicode fix DEBPKG:debian/kfreebsd-softupdates - http://bugs.debian.org/796798 Work around Debian Bug#796798 DEBPKG:fixes/autodie-scope - http://bugs.debian.org/798096 Fix a scoping issue with "no autodie" and the "system" sub DEBPKG:debian/debugperl-compat-fix - [perl #127212] http://bugs.debian.org/810326 Disable PERL_TRACK_MEMPOOL for debugging builds DEBPKG:fixes/CVE-2015-8607_file_spec_taint_fix - http://bugs.debian.org/810719 [perl #126862] ensure File::Spec::canonpath() preserves taint DEBPKG:fixes/mkstemp-umask - http://bugs.debian.org/810924 [perl #127322] [e57270b] Fix umask for mkstemp(3) calls DEBPKG:fixes/crosscompile-no-targethost - [perl #127234] Fix the Configure escape with usecrosscompile but no targethost DEBPKG:fixes/podlators-no-encode - [rt.cpan.org #111156] Degrade gracefully if utf8 is requested but Encode is not available DEBPKG:debian/cross-time-hires - [rt.cpan.org #111391] Add an environment variable to skip running configuration probes DEBPKG:fixes/encode-unicode-pod - Unicode.pm: Fix POD error DEBPKG:fixes/memoize-pod - [rt.cpan.org #89441] Fix POD errors in Memoize DEBPKG:fixes/ok-pod - Added encoding for pod. DEBPKG:fixes/CVE-2016-2381_duplicate_env - remove duplicate environment variables from environ @INC for perl 5.22.1: /home/nadim/perl5/lib/perl5/x86_64-linux-gnu-thread-multi /home/nadim/perl5/lib/perl5 /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base . Environment for perl 5.22.1: HOME=/home/nadim LANG=en_US.UTF-8 LANGUAGE= LC_ADDRESS=fr_FR.UTF-8 LC_IDENTIFICATION=fr_FR.UTF-8 LC_MEASUREMENT=fr_FR.UTF-8 LC_MONETARY=fr_FR.UTF-8 LC_NAME=fr_FR.UTF-8 LC_NUMERIC=fr_FR.UTF-8 LC_PAPER=fr_FR.UTF-8 LC_TELEPHONE=fr_FR.UTF-8 LC_TIME=fr_FR.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/nadim/bin:/home/nadim/.local/bin:/home/nadim/.qfc/bin:/home/nadim/nadim/devel/repositories/rakudo/install/share/perl6/site/bin/:/home/nadim/nadim/devel/repositories/rakudo/install/bin/:/home/nadim/perl5/bin:/home/nadim/nadim/bin/:/home/nadim/bin:/home/nadim/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/nadim/.go/bin:/home/nadim/nadim/devel/repositories/fzf/bin PERL5LIB=/home/nadim/perl5/lib/perl5: PERL_BADLANG (unset) PERL_LOCAL_LIB_ROOT=:/home/nadim/perl5 PERL_MB_OPT=--install_base /home/nadim/perl5 PERL_MM_OPT=INSTALL_BASE=/home/nadim/perl5 SHELL=/bin/bash ```
p5pRT commented 8 years ago

From @iabyn

On Fri\, Sep 30\, 2016 at 05​:51​:27PM -0700\, nadim khemir wrote​:

based on perlre​:

These special variables\, like the %+ hash and the numbered match variables ($1 \, $2 \, $3 \, etc.) are dynamically scoped until the end of the enclosing block or until the next successful match\, whichever comes first.

the following code parse a string which has the form {block1}* text* {block2} where block may be empty.

I use named capture (b1\, b2) to capture the blocks.

for each test case the following is display

test case

block1 text block2 text text

The problem can already be seen within the first iterations where $+{b1} should be undefined but has kept the previous value

This is intended behaviour. From the point of view of the dynamic scope of pattern match captures\, iterating a loop isn't regarded as exiting a scope and starting a new scope; rather it is seen as restarting the current scope.

A similar effect can be seen with this simplified program​:

  for (qw(ab XY)) {   print "\nPRE​: [$1][$+{foo}]\n";   /(.)(?'foo'.)/ or die;   print "POST​: [$1][$+{foo}]\n";   }

which outputs​:

  PRE​: [][]   POST​: [a][b]

  PRE​: [a][b]   POST​: [X][Y]

Adding an extra scope like

  for (qw(ab XY)) {   {   print "\nPRE​: [$1][$+{foo}]\n";   /(.)(?'foo'.)/ or die;   print "POST​: [$1][$+{foo}]\n";   }   }

will cause the output to change to​:

  PRE​: [][]   POST​: [a][b]

  PRE​: [][]   POST​: [X][Y]

-- Fire extinguisher (n) a device for holding open fire doors.

p5pRT commented 8 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 8 years ago

From @nkh

A hint in the documentation would be nice :)

maybe with this example

On Mon\, Oct 3\, 2016 at 12​:09 PM\, Dave Mitchell via RT \< perlbug-followup@​perl.org> wrote​:

On Fri\, Sep 30\, 2016 at 05​:51​:27PM -0700\, nadim khemir wrote​:

based on perlre​:

These special variables\, like the %+ hash and the numbered match variables ($1 \, $2 \, $3 \, etc.) are dynamically scoped until the end of the enclosing block or until the next successful match\, whichever comes first.

the following code parse a string which has the form {block1}* text* {block2} where block may be empty.

I use named capture (b1\, b2) to capture the blocks.

for each test case the following is display

test case

block1 text block2 text text

The problem can already be seen within the first iterations where $+{b1} should be undefined but has kept the previous value

This is intended behaviour. From the point of view of the dynamic scope of pattern match captures\, iterating a loop isn't regarded as exiting a scope and starting a new scope; rather it is seen as restarting the current scope.

A similar effect can be seen with this simplified program​:

for \(qw\(ab XY\)\) \{
    print "\\nPRE&#8203;:  \[$1\]\[$\+\{foo\}\]\\n";
    /\(\.\)\(?'foo'\.\)/ or die;
    print   "POST&#8203;: \[$1\]\[$\+\{foo\}\]\\n";
\}

which outputs​:

PRE&#8203;:  \[\]\[\]
POST&#8203;: \[a\]\[b\]

PRE&#8203;:  \[a\]\[b\]
POST&#8203;: \[X\]\[Y\]

Adding an extra scope like

for \(qw\(ab XY\)\) \{
    \{
        print "\\nPRE&#8203;:  \[$1\]\[$\+\{foo\}\]\\n";
        /\(\.\)\(?'foo'\.\)/ or die;
        print   "POST&#8203;: \[$1\]\[$\+\{foo\}\]\\n";
    \}
\}

will cause the output to change to​:

PRE&#8203;:  \[\]\[\]
POST&#8203;: \[a\]\[b\]

PRE&#8203;:  \[\]\[\]
POST&#8203;: \[X\]\[Y\]

-- Fire extinguisher (n) a device for holding open fire doors.