/(??{...})/ failure/corruption when using 'use bigint;'

p5pRT commented 13 years ago

Migrated from rt.perl.org#99026 (status was 'resolved')

Searchable as RT99026$

p5pRT commented 13 years ago

From jerry@nightwatch.org.uk

Created by jerry@nightwatch.org.uk

It would appear that 'use bigint;' doesn't interact with self generating regexes using the '(??{})' construct. This shows up when the RE construct is used to select a number of characters based upon a preceding capture buffer. Eg the code below should pull out 'abc':

"03abcde" =~ /^(..)((??{sprintf".{%d}"\,hex("0x$1")}))/; print "$`:$&:$' -> [$1] [$2]\n"

Run the same code with 'use bigint;':

use bigint; "03abcde" =~ /^(..)((??{sprintf".{%d}"\,hex("0x$1")}))/; print "$`:$&:$' -> [$1] [$2]\n"

and the RE doesn't match and $' is filled with garbage\, which would appear to be the memory of the process.

If the output of the sprintf() is captured it contains the correct value in both cases\, ie '.{3}'. This has been checked under 5.10.1 and 5.12.1.

Perl Info

``` Flags: category=library severity=medium module=bigint Site configuration information for perl 5.10.1: Configured by Debian Project at Thu Jun 30 19:24:11 UTC 2011. Summary of my perl5 (revision 5 version 10 subversion 1) configuration: Commit id: Platform: osname=linux, osvers=2.6.32-5-686, archname=i486-linux-gnu-thread-multi uname='linux callisto 2.6.32-5-686 #1 smp mon jun 13 04:13:06 utc 2011 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.1 -Dsitearch=/usr/local/lib/perl/5.10.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.1 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -g', cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.4.5', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/lib64 libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.11.2.so, so=so, useshrplib=true, libperl=libperl.so.5.10.1 gnulibc_version='2.11.2' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector' Locally applied patches: DEBPKG:debian/arm_thread_stress_timeout - http://bugs.debian.org/501970 Raise the timeout of ext/threads/shared/t/stress.t to accommodate slower build hosts DEBPKG:debian/cpan_config_path - Set location of CPAN::Config to /etc/perl as /usr may not be writable. DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN. DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove overly restrictive DB_File version check. DEBPKG:debian/doc_info - Replace generic man(1) instructions with Debian-specific information. DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories. DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes. DEBPKG:debian/extutils_hacks - Various debian-specific ExtUtils changes DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets. DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor. DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy. DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable. DEBPKG:debian/m68k_thread_stress - http://bugs.debian.org/495826 Disable some threads tests on m68k for now due to missing TLS. DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian DEBPKG:debian/module_build_man_extensions - http://bugs.debian.org/479460 Adjust Module::Build manual page extensions for the Debian Perl policy DEBPKG:debian/perl_synopsis - http://bugs.debian.org/278323 Rearrange perl.pod DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need. DEBPKG:debian/use_gdbm - Explicitly link against -lgdbm_compat in ODBM_File/NDBM_File. DEBPKG:fixes/assorted_docs - http://bugs.debian.org/443733 [384f06a] Math::BigInt::CalcEmu documentation grammar fix DEBPKG:fixes/net_smtp_docs - http://bugs.debian.org/100195 [rt.cpan.org #36038] Document the Net::SMTP 'Port' option DEBPKG:fixes/processPL - http://bugs.debian.org/357264 [rt.cpan.org #17224] Always use PERLRUNINST when building perl modules. DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local DEBPKG:fixes/pod2man-index-backslash - http://bugs.debian.org/521256 Escape backslashes in .IX entries DEBPKG:debian/disable-zlib-bundling - Disable zlib bundling in Compress::Raw::Zlib DEBPKG:fixes/kfreebsd_cppsymbols - http://bugs.debian.org/533098 [3b910a0] Add gcc predefined macros to $Config{cppsymbols} on GNU/kFreeBSD. DEBPKG:debian/cpanplus_definstalldirs - http://bugs.debian.org/533707 Configure CPANPLUS to use the site directories by default. DEBPKG:debian/cpanplus_config_path - Save local versions of CPANPLUS::Config::System into /etc/perl. DEBPKG:fixes/kfreebsd-filecopy-pipes - http://bugs.debian.org/537555 [16f708c] Fix File::Copy::copy with pipes on GNU/kFreeBSD DEBPKG:fixes/anon-tmpfile-dir - http://bugs.debian.org/528544 [perl #66452] Honor TMPDIR when open()ing an anonymous temporary file DEBPKG:fixes/abstract-sockets - http://bugs.debian.org/329291 [89904c0] Add support for Abstract namespace sockets. DEBPKG:fixes/hurd_cppsymbols - http://bugs.debian.org/544307 [eeb92b7] Add gcc predefined macros to $Config{cppsymbols} on GNU/Hurd. DEBPKG:fixes/autodie-flock - http://bugs.debian.org/543731 Allow for flock returning EAGAIN instead of EWOULDBLOCK on linux/parisc DEBPKG:fixes/archive-tar-instance-error - http://bugs.debian.org/539355 [rt.cpan.org #48879] Separate Archive::Tar instance error strings from each other DEBPKG:fixes/positive-gpos - http://bugs.debian.org/545234 [perl #69056] [c584a96] Fix \\G crash on first match DEBPKG:debian/devel-ppport-ia64-optim - http://bugs.debian.org/548943 Work around an ICE on ia64 DEBPKG:fixes/trie-logic-match - http://bugs.debian.org/552291 [perl #69973] [0abd0d7] Fix a DoS in Unicode processing [CVE-2009-3626] DEBPKG:fixes/hppa-thread-eagain - http://bugs.debian.org/554218 make the threads-shared test suite more robust, fixing failures on hppa DEBPKG:fixes/crash-on-undefined-destroy - http://bugs.debian.org/564074 [perl #71952] [1f15e67] Fix a NULL pointer dereference when looking for a DESTROY method DEBPKG:fixes/tainted-errno - http://bugs.debian.org/574129 [perl #61976] [be1cf43] fix an errno stringification bug in taint mode DEBPKG:fixes/safe-upgrade - http://bugs.debian.org/582978 Upgrade Safe.pm to 2.25, fixing CVE-2010-1974 DEBPKG:fixes/tell-crash - http://bugs.debian.org/578577 [f4817f3] Fix a tell() crash on bad arguments. DEBPKG:fixes/format-write-crash - http://bugs.debian.org/579537 [perl #22977] [421f30e] Fix a crash in format/write DEBPKG:fixes/arm-alignment - http://bugs.debian.org/289884 [f1c7503] Prevent gcc from optimizing the alignment test away on armel DEBPKG:fixes/fcgi-test - Fix a failure in CGI/t/fast.t when FCGI is installed DEBPKG:fixes/hurd-ccflags - http://bugs.debian.org/587901 Make hints/gnu.sh append to $ccflags rather than overriding them DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts DEBPKG:fixes/lc-numeric-docs - http://bugs.debian.org/379329 [perl #78452] [903eb63] LC_NUMERIC documentation fixes DEBPKG:fixes/lc-numeric-sprintf - http://bugs.debian.org/601549 [perl #78632] [b3fd614] Fix sprintf not to ignore LC_NUMERIC with constants DEBPKG:fixes/concat-stack-corruption - http://bugs.debian.org/596105 [perl #78674] [e3393f5] Fix stack pointer corruption in pp_concat() with 'use encoding' DEBPKG:fixes/cgi-multiline-header - http://bugs.debian.org/606995 [CVE-2010-2761 CVE-2010-4410 CVE-2010-4411] CGI.pm MIME boundary and multiline header vulnerabilities DEBPKG:fixes/casing-taint-cve-2011-1487 - http://bugs.debian.org/622817 [perl #87336] fix unwanted taint laundering in lc(), uc() et al. DEBPKG:fixes/safe-reval-rdo-cve-2010-1447 - [PATCH] Wrap by default coderefs returned by rdo and reval DEBPKG:patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.10.1-17squeeze2 in patchlevel.h @INC for perl 5.10.1: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl . Environment for perl 5.10.1: HOME=/home/jerry LANG=en_GB.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/local/bin:/usr/bin:/bin:/usr/games:/home/jerry/bin PERL_BADLANG (unset) SHELL=/bin/bash ```

p5pRT commented 12 years ago

From @jkeenan

On Tue Sep 13 08:17:35 2011\, jerry@nightwatch.org.uk wrote:

    "03abcde" =~ /^$\.\.$$\(??\{sprintf"\.\{%d\}"\,hex\("0x$1"$\}\)\)/;
    print "$\`&#8203;:$&&#8203;:$' \-> \[$1\] \[$2\]\\n"
Run the same code with 'use bigint;':
    use bigint;
    "03abcde" =~ /^$\.\.$$\(??\{sprintf"\.\{%d\}"\,hex\("0x$1"$\}\)\)/;
    print "$\`&#8203;:$&&#8203;:$' \-> \[$1\] \[$2\]\\n"
and the RE doesn't match and $' is filled with garbage\, which would appear to be the memory of the process.

I get a somewhat different result.

### "03abcde" =~ /^(..)((??{sprintf".{%d}"\,hex("0x$1")}))/; print "$`:$&:$' -> [$1] [$2]\n";

use bigint; "03abcde" =~ /^(..)((??{sprintf".{%d}"\,hex("0x$1")}))/; print "$`:$&:$' -> [$1] [$2]\n"; ###

... produces:

### :03abc:de -> [03] [abc] Use of uninitialized value $2 in concatenation (.) or string at 99026.pl line 13. :03abc:de -> [03] [] ###

... i.e.\, no garbage\, but no good results either. (Perl 5.14.2/Linux i386)

Thank you very much. Jim Keenan

p5pRT commented 12 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 12 years ago

From @nwc10

Running the test code under valgrind gives:

:03abc:de -> [03] [abc] ==7658== Invalid read of size 4 ==7658== at 0x82C10FC: S_regcppush (regexec.c:369) ==7658== by 0x82DA88F: S_regmatch (regexec.c:4391) ==7658== by 0x82D174D: S_regtry (regexec.c:2683) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B1EF7: Perl_pp_match (pp_hot.c:1355) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x8094BDE: S_run_body (perl.c:2383) ==7658== by 0x8093E39: perl_run (perl.c:2301) ==7658== by 0x80603EC: main (perlmain.c:120) ==7658== Address 0x4277974 is 20 bytes inside a block of size 28 free'd ==7658== at 0x4024046: realloc (vg_replace_malloc.c:525) ==7658== by 0x815B25D: Perl_safesysrealloc (util.c:193) ==7658== by 0x82D162A: S_regtry (regexec.c:2653) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B839C: Perl_pp_subst (pp_hot.c:2151) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x82D9D3F: S_regmatch (regexec.c:4275) ==7658== by 0x82D174D: S_regtry (regexec.c:2683) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B1EF7: Perl_pp_match (pp_hot.c:1355) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x8094BDE: S_run_body (perl.c:2383) ==7658== ==7658== Invalid write of size 4 ==7658== at 0x82C15F5: S_regcppop (regexec.c:430) ==7658== by 0x82DF03A: S_regmatch (regexec.c:5492) ==7658== by 0x82D174D: S_regtry (regexec.c:2683) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B1EF7: Perl_pp_match (pp_hot.c:1355) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x8094BDE: S_run_body (perl.c:2383) ==7658== by 0x8093E39: perl_run (perl.c:2301) ==7658== by 0x80603EC: main (perlmain.c:120) ==7658== Address 0x4277970 is 16 bytes inside a block of size 28 free'd ==7658== at 0x4024046: realloc (vg_replace_malloc.c:525) ==7658== by 0x815B25D: Perl_safesysrealloc (util.c:193) ==7658== by 0x82D162A: S_regtry (regexec.c:2653) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B839C: Perl_pp_subst (pp_hot.c:2151) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x82D9D3F: S_regmatch (regexec.c:4275) ==7658== by 0x82D174D: S_regtry (regexec.c:2683) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B1EF7: Perl_pp_match (pp_hot.c:1355) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x8094BDE: S_run_body (perl.c:2383) ==7658== ==7658== Invalid read of size 4 ==7658== at 0x82DB29E: S_regmatch (regexec.c:4490) ==7658== by 0x82D174D: S_regtry (regexec.c:2683) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B1EF7: Perl_pp_match (pp_hot.c:1355) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x8094BDE: S_run_body (perl.c:2383) ==7658== by 0x8093E39: perl_run (perl.c:2301) ==7658== by 0x80603EC: main (perlmain.c:120) ==7658== Address 0x4277974 is 20 bytes inside a block of size 28 free'd ==7658== at 0x4024046: realloc (vg_replace_malloc.c:525) ==7658== by 0x815B25D: Perl_safesysrealloc (util.c:193) ==7658== by 0x82D162A: S_regtry (regexec.c:2653) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B839C: Perl_pp_subst (pp_hot.c:2151) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x82D9D3F: S_regmatch (regexec.c:4275) ==7658== by 0x82D174D: S_regtry (regexec.c:2683) ==7658== by 0x82CE7E4: Perl_regexec_flags (regexec.c:2173) ==7658== by 0x81B1EF7: Perl_pp_match (pp_hot.c:1355) ==7658== by 0x815A603: Perl_runops_debug (dump.c:2118) ==7658== by 0x8094BDE: S_run_body (perl.c:2383) ==7658== :03abc:de -> [03] []

Nicholas Clark

p5pRT commented 12 years ago

From jerry@nightwatch.org.uk

I get a somewhat different result.

### "03abcde" =~ /^(..)((??{sprintf".{%d}"\,hex("0x$1")}))/; print "$`:$&:$' -> [$1] [$2]\n";

use bigint; "03abcde" =~ /^(..)((??{sprintf".{%d}"\,hex("0x$1")}))/; print "$`:$&:$' -> [$1] [$2]\n"; ###

... produces:

### :03abc:de -> [03] [abc] Use of uninitialized value $2 in concatenation (.) or string at 99026.pl line 13. :03abc:de -> [03] [] ###

... i.e.\, no garbage\, but no good results either. (Perl 5.14.2/Linux i386)

Ah\, sorry that wasn't very clear regarding the code snippets. If you run them as separate pieces of code\, i.e.:

"03abcde" =~ /^(..)((??{sprintf".{%d}"\,hex("0x$1")}))/; print "$`:$&:$' -> [$1] [$2]\n";

and then with the "use bigint;"

use bigint; "03abcde" =~ /^(..)((??{sprintf".{%d}"\,hex("0x$1")}))/; print "$`:$&:$' -> [$1] [$2]\n";

That way you'll see the difference. I've just tested\, and if run separately I get the garbage for the second chunk\, but together I get valid output just like you had (apart from the second RE didn't match when it should have).

Jerry.

p5pRT commented 12 years ago

From @jkeenan

On Tue Dec 06 15:03:02 2011\, jerry@nightwatch.org.uk wrote:

Ah\, sorry that wasn't very clear regarding the code snippets. If you run them as separate pieces of code\, i.e.:
"03abcde" =~ /^$\.\.$$\(??\{sprintf"\.\{%d\}"\,hex\("0x$1"$\}\)\)/;
print "$\`&#8203;:$&&#8203;:$' \-> \[$1\] \[$2\]\\n";
and then with the "use bigint;"
use bigint;
"03abcde" =~ /^$\.\.$$\(??\{sprintf"\.\{%d\}"\,hex\("0x$1"$\}\)\)/;
print "$\`&#8203;:$&&#8203;:$' \-> \[$1\] \[$2\]\\n";
That way you'll see the difference. I've just tested\, and if run separately I get the garbage for the second chunk\, but together I get valid output just like you had (apart from the second RE didn't match when it should have).

But I find that even if I separate the chunks into separate files\, I get the same results I previously posted (which\, to make a correction\, were from Darwin; what follows is from Linux):

##### $ diff 99026.pl bigint_99026.pl 7a8

use bigint;

$ perl 99026.pl ;perl bigint_99026.pl :03abc:de -> [03] [abc]

Use of uninitialized value $2 in concatenation (.) or string at bigint_99026.pl line 10. :03abc:de -> [03] []

#####

p5pRT commented 12 years ago

From @cpansprout

This has been fixed by the commits leading up to eb58a7e122.

p5pRT commented 12 years ago

@cpansprout - Status changed from 'open' to 'resolved'

Perl / perl5