fold_array overflow in regmatch

Migrated from rt.perl.org#115332 (status was 'resolved')

Searchable as RT115332$

From @rurban

This is a bug report for perl from rurban@cpanel.net\, generated with the help of perlbug 1.39 running under perl 5.14.2.

Compile blead with asan as decribed in perlhacks.pod

cd ext/DynaLoader gdb --args ./miniperl -I../../lib Makefile.PL /INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none PERL_CORE=1 \ LIBPERL_A=libperl.a LINKTYPE=static CCCDLFLAGS=

(gdb) r [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Processing hints file hints/linux.pl

Breakpoint 1\, 0x000000000117b050 in __asan_report_error () (gdb) up #1 0x000000000117b047 in __asan_report_load1 () (gdb) up #2 0x0000000000f8f1f8 in S_regmatch (reginfo=\\, #startpos=\\, prog=\) at regexec.c:4130 4130 if (UCHARAT(s) != nextchr && (gdb) p nextchr $2 = -66368 (gdb) l 4125 locinput = e; 4126 break; 4127 } 4128
4129 /* Neither the target nor the pattern are utf8 */ 4130 if (UCHARAT(s) != nextchr && 4131 UCHARAT(s) != fold_array[nextchr]) 4132 { 4133 sayNO; 4134 }

Flags: category=core severity=high

This perlbug was built using Perl 5.17.3 - Mon Jul 30 16:28:27 CDT 2012 It is being executed now by Perl 5.14.2 - Wed Oct 26 17:33:43 CDT 2011.

Site configuration information for perl 5.14.2:

Configured by rurban at Wed Oct 26 17:33:43 CDT 2011.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:
Platform: osname=linux\, osvers=3.0.0-1-amd64\, archname=x86_64-linux uname='linux reini 3.0.0-1-amd64 #1 smp sun jul 24 02:24:44 utc 2011 x86_64 gnulinux ' config_args='-de -Dmksymlinks -Duseshrplib -Dusedevel -Doptimize=-Os -Accflags=-msse4.2 -Accflags=-march=corei7 -Dcf_email=rurban@cpanel.net -Dperladmin=rurban@cpanel.net -Dstartperl=#!/usr/local/bin/perl5.14.2-nt -Dperlpath=/usr/local/bin/perl5.14.2-nt' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='ccache cc'\, ccflags ='-msse4.2 -march=corei7 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-Os'\, cppflags='-msse4.2 -march=corei7 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.6.1'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='cc'\, ldflags =' -Os -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/lib/x86_64-linux-gnu /lib64 /usr/lib64 libs=-lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/local/lib/perl5/5.14.2/x86_64-linux/CORE' cccdlflags='-fPIC'\, lddlflags='-shared -Os -L/usr/local/lib -fstack-protector'

Locally applied patches:

@INC for perl 5.14.2: /usr/local/lib/perl5/site_perl/5.14.2/x86_64-linux /usr/local/lib/perl5/site_perl/5.14.2 /usr/local/lib/perl5/5.14.2/x86_64-linux /usr/local/lib/perl5/5.14.2 .

Environment for perl 5.14.2: HOME=/home/rurban LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/rurban/bin:/home/rurban/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games PERL_BADLANG (unset) SHELL=/bin/bash

From @khwilliamson

On 10/17/2012 07:42 PM\, rurban@cpanel.net (via RT) wrote:

# New Ticket Created by rurban@cpanel.net # Please include the string: [perl #115332] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=115332 >

This is a bug report for perl from rurban@cpanel.net\, generated with the help of perlbug 1.39 running under perl 5.14.2.

----------------------------------------------------------------- Compile blead with asan as decribed in perlhacks.pod

cd ext/DynaLoader gdb --args ./miniperl -I../../lib Makefile.PL /INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none PERL_CORE=1 \ LIBPERL_A=libperl.a LINKTYPE=static CCCDLFLAGS=

(gdb) r [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Processing hints file hints/linux.pl

Breakpoint 1\, 0x000000000117b050 in __asan_report_error () (gdb) up #1 0x000000000117b047 in __asan_report_load1 () (gdb) up #2 0x0000000000f8f1f8 in S_regmatch (reginfo=\\, #startpos=\\, prog=\) at regexec.c:4130 4130 if (UCHARAT(s) != nextchr && (gdb) p nextchr $2 = -66368 (gdb) l 4125 locinput = e; 4126 break; 4127 } 4128
4129 /* Neither the target nor the pattern are utf8 */ 4130 if (UCHARAT(s) != nextchr && 4131 UCHARAT(s) != fold_array[nextchr]) 4132 { 4133 sayNO; 4134 }

----------------------------------------------------------------- --- Flags: category=core severity=high --- This perlbug was built using Perl 5.17.3 - Mon Jul 30 16:28:27 CDT 2012 It is being executed now by Perl 5.14.2 - Wed Oct 26 17:33:43 CDT 2011.

Site configuration information for perl 5.14.2:

Configured by rurban at Wed Oct 26 17:33:43 CDT 2011.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

Platform: osname=linux\, osvers=3.0.0-1-amd64\, archname=x86_64-linux uname='linux reini 3.0.0-1-amd64 #1 smp sun jul 24 02:24:44 utc 2011 x86_64 gnulinux ' config_args='-de -Dmksymlinks -Duseshrplib -Dusedevel -Doptimize=-Os -Accflags=-msse4.2 -Accflags=-march=corei7 -Dcf_email=rurban@cpanel.net -Dperladmin=rurban@cpanel.net -Dstartperl=#!/usr/local/bin/perl5.14.2-nt -Dperlpath=/usr/local/bin/perl5.14.2-nt' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='ccache cc'\, ccflags ='-msse4.2 -march=corei7 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-Os'\, cppflags='-msse4.2 -march=corei7 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.6.1'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='cc'\, ldflags =' -Os -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/lib/x86_64-linux-gnu /lib64 /usr/lib64 libs=-lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/local/lib/perl5/5.14.2/x86_64-linux/CORE' cccdlflags='-fPIC'\, lddlflags='-shared -Os -L/usr/local/lib -fstack-protector'

Locally applied patches:

--- @INC for perl 5.14.2: /usr/local/lib/perl5/site_perl/5.14.2/x86_64-linux /usr/local/lib/perl5/site_perl/5.14.2 /usr/local/lib/perl5/5.14.2/x86_64-linux /usr/local/lib/perl5/5.14.2 .

--- Environment for perl 5.14.2: HOME=/home/rurban LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/rurban/bin:/home/rurban/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games PERL_BADLANG (unset) SHELL=/bin/bash

The line numbers in the gdb session are off by about 80 from current blead\, and the bug report was made on a 5.14 system\, so all the version information therein is useless.

There were a number of recent changes to this file that may have exacerbated or fixed this problem. There is no perlhacks.pod\, and asan is not a string in perlhacktips\, which I believe is what you meant\, after asking on irc. valgrind does not show a problem with current blead.

Independent of this bug report\, I today realized there are a number of bugs (all of a similar nature) in regrepeat() in regexec.c Some\, I introduced just yesterday; some I introduced in probably 5.12\, and some have been there since before I came along. (For the most part\, I just assumed that the pre-existing code was valid\, and re-used its flawed paradigm.) I have fixes\, but now need to figure out tests for them.

But my guess is that you are using a blead earlier than Dave Mitchell's recent commits to stop reading beyond the end of the buffer\, and that those commits may very well fix your problem. It would be helpful if you were to test on the most recent blead\, and use perlbug from that blead.

The RT System itself - Status changed from 'new' to 'open'

From @rurban

On Wed\, Oct 17\, 2012 at 11:11 PM\, Karl Williamson \public@khwilliamson\.com wrote:

On 10/17/2012 07:42 PM\, rurban@cpanel.net (via RT) wrote:

----------------------------------------------------------------- Compile blead with asan as decribed in perlhacks.pod

perlhacktips it is.

cd ext/DynaLoader gdb --args ./miniperl -I../../lib Makefile.PL /INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none PERL_CORE=1 \ LIBPERL_A=libperl.a LINKTYPE=static CCCDLFLAGS=

(gdb) r [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Processing hints file hints/linux.pl

Breakpoint 1\, 0x000000000117b050 in __asan_report_error () (gdb) up #1 0x000000000117b047 in __asan_report_load1 () (gdb) up #2 0x0000000000f8f1f8 in S_regmatch (reginfo=\\, #startpos=\\, prog=\) at regexec.c:4130 4130 if (UCHARAT(s) != nextchr &&

The new line number as of commit 084b56d is regexec.c:4208 The bug is the same.

(gdb) p nextchr $2 = -66368 (gdb) l 4125 locinput = e; 4126 break; 4127 } 4128 4129 /* Neither the target nor the pattern are utf8 */ 4130 if (UCHARAT(s) != nextchr && 4131 UCHARAT(s) != fold_array[nextchr]) 4132 { 4133 sayNO; 4134 }

----------------------------------------------------------------- --- Flags: category=core severity=high --- This perlbug was built using Perl 5.17.3 - Mon Jul 30 16:28:27 CDT 2012 It is being executed now by Perl 5.14.2 - Wed Oct 26 17:33:43 CDT 2011.

Site configuration information for perl 5.14.2:

Configured by rurban at Wed Oct 26 17:33:43 CDT 2011.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

Platform: osname=linux\, osvers=3.0.0-1-amd64\, archname=x86_64-linux uname='linux reini 3.0.0-1-amd64 #1 smp sun jul 24 02:24:44 utc 2011 x86_64 gnulinux ' config_args='-de -Dmksymlinks -Duseshrplib -Dusedevel -Doptimize=-Os -Accflags=-msse4.2 -Accflags=-march=corei7 -Dcf_email=rurban@cpanel.net -Dperladmin=rurban@cpanel.net -Dstartperl=#!/usr/local/bin/perl5.14.2-nt -Dperlpath=/usr/local/bin/perl5.14.2-nt' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='ccache cc'\, ccflags ='-msse4.2 -march=corei7 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-Os'\, cppflags='-msse4.2 -march=corei7 -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.6.1'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='cc'\, ldflags =' -Os -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/lib/x86_64-linux-gnu /lib64 /usr/lib64 libs=-lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/local/lib/perl5/5.14.2/x86_64-linux/CORE' cccdlflags='-fPIC'\, lddlflags='-shared -Os -L/usr/local/lib -fstack-protector'

Locally applied patches:

--- @INC for perl 5.14.2: /usr/local/lib/perl5/site_perl/5.14.2/x86_64-linux /usr/local/lib/perl5/site_perl/5.14.2 /usr/local/lib/perl5/5.14.2/x86_64-linux /usr/local/lib/perl5/5.14.2 .

--- Environment for perl 5.14.2: HOME=/home/rurban LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset)

PATH=/home/rurban/bin:/home/rurban/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games PERL_BADLANG (unset) SHELL=/bin/bash

The line numbers in the gdb session are off by about 80 from current blead\, and the bug report was made on a 5.14 system\, so all the version information therein is useless.

With a perl I cannot use perlbug. I only got a miniperl and I did not care enough to manually construct a perlbug report just for convenience. Please try to reproduce and fix the bug instead. It cannot be that I'm the only one for over a year who tries to find bugs like this with asan.

There were a number of recent changes to this file that may have exacerbated or fixed this problem. There is no perlhacks.pod\, and asan is not a string in perlhacktips\, which I believe is what you meant\, after asking on irc. valgrind does not show a problem with current blead.

Because if you read perlhacktips you'll see that valgrind is not able to detect those kind of buffer overflows with globals.

There were a number of recent changes which caused this bug. I was able build with asan until recently.

Independent of this bug report\, I today realized there are a number of bugs (all of a similar nature) in regrepeat() in regexec.c Some\, I introduced just yesterday; some I introduced in probably 5.12\, and some have been there since before I came along. (For the most part\, I just assumed that the pre-existing code was valid\, and re-used its flawed paradigm.) I have fixes\, but now need to figure out tests for them.

But my guess is that you are using a blead earlier than Dave Mitchell's recent commits to stop reading beyond the end of the buffer\, and that those commits may very well fix your problem.

On the contrary. My guess is that dave's branch with 7016d6ebb4af introduced this regression.

It would be helpful if you were to test on the most recent blead\,

Bug confirmed with def6ed2 and 084b56d. Tested against the commit before dave's nextchr hacks\, 37d7e3 from Wed Sep 26 08:33:20 2012\, and this worked fine. So Dave's improved_charclass_generator branch introduced this regression\, even if he recognized that nextchr might go \< 0 in some places. Not in this one.

and use perlbug from that blead.

This is not possible. You need a perl to be able to use perlbug. If you try to reproduce this bug you'll see what I am talking about.

And I strongly recommend to those developers who hack in C to use asan regularly. -- Reini Urban http://cpanel.net/ http://www.perl-compiler.org/

From @craigberry

On Thu\, Oct 18\, 2012 at 8:32 AM\, Reini Urban \rurban@x\-ray\.at wrote:

On Wed\, Oct 17\, 2012 at 11:11 PM\, Karl Williamson \public@khwilliamson\.com wrote:

and use perlbug from that blead.

This is not possible. You need a perl to be able to use perlbug.

The precise version\, configuration\, and platform information are what's important\, not whether you use perlbug to get it. If perlbug is not working because you can't get as far as the perl executable but you can build miniperl\, just do:

% make miniperl % make lib/Config.pm % ./miniperl -Ilib -V

and copy and paste the output of that last command into a message sent to perlbug AT perl DOT org using your regular mailer.

From @doughera88

On Thu\, 18 Oct 2012\, Craig A. Berry wrote:

On Thu\, Oct 18\, 2012 at 8:32 AM\, Reini Urban \rurban@x\-ray\.at wrote:

On Wed\, Oct 17\, 2012 at 11:11 PM\, Karl Williamson \public@khwilliamson\.com wrote:

and use perlbug from that blead.

This is not possible. You need a perl to be able to use perlbug.

The precise version\, configuration\, and platform information are what's important\, not whether you use perlbug to get it. If perlbug is not working because you can't get as far as the perl executable but you can build miniperl\, just do:

% make miniperl % make lib/Config.pm % ./miniperl -Ilib -V

and copy and paste the output of that last command into a message sent to perlbug AT perl DOT org using your regular mailer.

And if that doesn't work\, then just run the ./myconfig script and include the output in your mail message\, as suggested in the INSTALL file. This information is often tremendously helpful for someone else to use to try to reproduce your findings.

-- Andy Dougherty doughera@lafayette.edu

From @iabyn

On Thu\, Oct 18\, 2012 at 08:32:49AM -0500\, Reini Urban wrote:

On the contrary. My guess is that dave's branch with 7016d6ebb4af introduced this regression.

Quite possibly.

So Dave's improved_charclass_generator branch introduced this regression\, even if he recognized that nextchr might go \< 0 in some places. Not in this one.

Um\, my branch was never called improved_charclass_generator?

After that branch\, nextchr was supposed to never go \< 0\, apart from the special value NEXTCHR_EOS\, -10 (and never above 255 either).

And I strongly recommend to those developers who hack in C to use asan regularly.

My Fedora 17 system only comes with clang 3.0\, and I've been waiting for Fedora to move to 3.1.

Anyway\, I can *partially* reproduce this issue\, in that I can see nextchr being NEXTCHR_EOS\, -10\, when the code does fold_array[nextchr]\, which is definitely a logic flaw; but I'm not seeing any wild negative values (such as -66368) appearing.

To reproduce the -10 issue:

$ gdb --args ./miniperl -e'"a" =~ /a(b|c)?/i' (gdb) b regexec.c:4209 (gdb) r Breakpoint 1\, S_regmatch (reginfo=0x7fffffffda00\, startpos=0x9de460 "a"\, prog=0x9de6c0) at regexec.c:4209 4209 UCHARAT(s) != fold_array[nextchr]) Missing separate debuginfos\, use: debuginfo-install glibc-2.15-57.fc17.x86_64 nss-softokn-freebl-3.13.5-1.fc17.x86_64 (gdb) p nextchr $1 = 0xfffffff6 (gdb) l 4204 break; 4205 } 4206
4207 /* Neither the target nor the pattern are utf8 */ 4208 if (UCHARAT(s) != nextchr && 4209 UCHARAT(s) != fold_array[nextchr]) 4210 { 4211 sayNO; 4212 } 4213 if (PL_regeol - locinput \< ln)

-- This is a great day for France! -- Nixon at Charles De Gaulle's funeral

From @iabyn

On Thu\, Oct 18\, 2012 at 04:34:39PM +0100\, Dave Mitchell wrote:

To reproduce the -10 issue:

I forgot to mention\, I actually used a condition on that breakpoint:

Initially:

cond 1 nextchr \< 0 || nextchr > 255

which found the -10 issue; I then changed it to

cond 1 (nextchr \< 0 && nextchr != -10) || nextchr > 255

running with

-I../../lib Makefile.PL INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none PERL_CORE=1 LIBPERL_A=libperl.a LINKTYPE=static CCCDLFLAGS=

and it failed to spot anything on my build.

-- Fire extinguisher (n) a device for holding open fire doors.

From @rurban

On 10/18/2012 10:35 AM\, Dave Mitchell via RT wrote:

On Thu\, Oct 18\, 2012 at 08:32:49AM -0500\, Reini Urban wrote:

On the contrary. My guess is that dave's branch with 7016d6ebb4af introduced this regression.

Quite possibly.

So Dave's improved_charclass_generator branch introduced this regression\, even if he recognized that nextchr might go \< 0 in some places. Not in this one.

Um\, my branch was never called improved_charclass_generator?

Sorry. That's what my git-prompt told me\, but it's obviously a wrong name.

After that branch\, nextchr was supposed to never go \< 0\, apart from the special value NEXTCHR_EOS\, -10 (and never above 255 either).

Yes\, I saw your ASSERT.

And I strongly recommend to those developers who hack in C to use asan regularly.

My Fedora 17 system only comes with clang 3.0\, and I've been waiting for Fedora to move to 3.1.

I'm compiling my own clang since a year\, based on these instructions: http://code.google.com/p/address-sanitizer/wiki/HowToBuild

Anyway\, I can *partially* reproduce this issue\, in that I can see nextchr being NEXTCHR_EOS\, -10\, when the code does fold_array[nextchr]\, which is definitely a logic flaw; but I'm not seeing any wild negative values (such as -66368) appearing.

Interesting.

To reproduce the -10 issue:

$ gdb --args ./miniperl -e'"a" =~ /a(b|c)?/i' (gdb) b regexec.c:4209 (gdb) r Breakpoint 1\, S_regmatch (reginfo=0x7fffffffda00\, startpos=0x9de460 "a"\, prog=0x9de6c0) at regexec.c:4209 4209 UCHARAT(s) != fold_array[nextchr]) Missing separate debuginfos\, use: debuginfo-install glibc-2.15-57.fc17.x86_64 nss-softokn-freebl-3.13.5-1.fc17.x86_64 (gdb) p nextchr $1 = 0xfffffff6 (gdb) l 4204 break; 4205 } 4206
4207 /* Neither the target nor the pattern are utf8 */ 4208 if (UCHARAT(s) != nextchr && 4209 UCHARAT(s) != fold_array[nextchr]) 4210 { 4211 sayNO; 4212 } 4213 if (PL_regeol - locinput \< ln)

The regexec logic is over my head\, sorry. You are alone with Yves and Karl on this one :) I'm only doing the simple stuff. -- Reini

Working towards a true Modern Perl. Slim\, functional\, unbloated\, compile-time optimizable

From @iabyn

On Thu\, Oct 18\, 2012 at 10:55:56AM -0500\, Reini Urban wrote:

The regexec logic is over my head\, sorry. You are alone with Yves and Karl on this one :) I'm only doing the simple stuff.

If you have the ability to reproduce this on blead\, is it possible for you to run miniperl with -Dr to generate voluminous rgexex debugging\, then to post to the list the output associated just with the last match attempted\, e.g.

$ perl -Dr -e'"abc" =~ /\wb./'

....

Guessing start of match in sv for REx "\wb." against "abc" Found anchored substr "b" at offset 1... start_shift: 1 check_at: 1 s: 0 endpos: 1 checked_upto: 0 Does not contradict STCLASS... Guessed: match at offset 0 Matching REx "\wb." against "abc" 0 \<> \ | 1:ALNUM(2) 1 \ \ | 2:EXACT \(4) 2 \ \ | 4:REG_ANY(5) 3 \ \<> | 5:END(0)

Ideally capturing all of the above\, or if its very long\, to post at least the beginning part and the last few lines-worth of the ops it executes?

Thanks.

-- The crew of the Enterprise encounter an alien life form which is surprisingly neither humanoid nor made from pure energy. -- Things That Never Happen in "Star Trek" #22

From @demerphq

On Thursday\, 18 October 2012\, Dave Mitchell wrote:

On Thu\, Oct 18\, 2012 at 08:32:49AM -0500\, Reini Urban wrote:

On the contrary. My guess is that dave's branch with 7016d6ebb4af introduced this regression.

Quite possibly.

So Dave's improved_charclass_generator branch introduced this regression\, even if he recognized that nextchr might go \< 0 in some places. Not in this one.

Um\, my branch was never called improved_charclass_generator?

That was my branch to add suffix grouping to regcharclass\, which I don't think is relevant here.

Yves

-- perl -Mre=debug -e "/just|another|perl|hacker/"

From @iabyn

On Thu\, Oct 18\, 2012 at 04:34:39PM +0100\, Dave Mitchell wrote:

Anyway\, I can *partially* reproduce this issue\, in that I can see nextchr being NEXTCHR_EOS\, -10\, when the code does fold_array[nextchr]\, which is definitely a logic flaw;

Now fixed with 1443c94c5785506d57ff756925baa65702a6cf98.

but I'm not seeing any wild negative values (such as -66368) appearing.

I've also added some assertions on the correct value of nextchr with e6ca698ca4309632ab09826ad47492d2934e10bd\,

but in the absence of someone else reproducing this and/or providing further info\, I can't take the 'wild' value issue any further\, since I can't reproduce it.

-- Atheism is a religion like not collecting stamps is a hobby

From @rurban

On 10/19/2012 05:26 AM\, Dave Mitchell via RT wrote:

On Thu\, Oct 18\, 2012 at 04:34:39PM +0100\, Dave Mitchell wrote:

Anyway\, I can *partially* reproduce this issue\, in that I can see nextchr being NEXTCHR_EOS\, -10\, when the code does fold_array[nextchr]\, which is definitely a logic flaw;

Now fixed with 1443c94c5785506d57ff756925baa65702a6cf98.

but I'm not seeing any wild negative values (such as -66368) appearing.

I've also added some assertions on the correct value of nextchr with e6ca698ca4309632ab09826ad47492d2934e10bd\,

but in the absence of someone else reproducing this and/or providing further info\, I can't take the 'wild' value issue any further\, since I can't reproduce it.

I can confirm that with asan I can now successfully build DynaLoader.

6b54ddc5f039cda5c3fd0fa36516955199bdb957

Summary of my perl5 (revision 5 version 17 subversion 6) configuration:

Platform: osname=linux\, osvers=3.2.0-2-amd64\, archname=x86_64-linux uname='linux reini 3.2.0-2-amd64 #1 smp mon may 21 17:45:41 utc 2012 x86_64 gnulinux ' config_args='-de -Dusedevel -Uversiononly -Dinstallman1dir=none -Dinstallman3dir=none -Dinstallsiteman1dir=none -Dinstallsiteman3dir=none -DEBUGGING -Doptimize=-g3 -Uuseithreads -Dcc=clang -Dld=clang -Accflags=-faddress-sanitizer -Aldflags=-faddress-sanitizer -Alddlflags=-shared -faddress-sanitizer -Duseshrplib -Dcf_email=rurban@cpanel.net -Dperladmin=rurban@cpanel.net -Duseshrplib -Accflags=-Wno-unused-value' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='clang'\, ccflags ='-faddress-sanitizer -Wno-unused-value -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-g3'\, cppflags='-faddress-sanitizer -Wno-unused-value -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include' ccversion=''\, gccversion='4.2.1 Compatible Clang 3.2 (trunk)'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='clang'\, ldflags =' -faddress-sanitizer -L/usr/local/lib' libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@6b54ddc5/CORE' cccdlflags='-fPIC'\, lddlflags=' -shared -faddress-sanitizer -L/usr/local/lib '

The next error is a heap-overflow at a read of size 1 at t/base/lex.t in S_scan_heredoc() => new ticket -- Reini

Working towards a true Modern Perl. Slim\, functional\, unbloated\, compile-time optimizable

@iabyn - Status changed from 'open' to 'resolved'

Perl / perl5