/i unicode regex causes segfault in uninitialized utf8::SWASHINIT opaddr

[GoogleCodeExporter]

$>perlcc  -O3 -e ' "\x{101}a" =~ qr/\x{100}/i && print "ok\n" '; ./a.out

segmentation fault  ./a.out

Original issue reported on code.google.com by todd.e.rinaldo on 9 May 2014 at 9:35

[GoogleCodeExporter]

There seems to be some play here:

perlcc  -O3 -e '"\x{100}a" =~ qr/\x{100}/i && print "ok\n"'; ./a.out

segfault...

Original comment by todd.e.rinaldo on 9 May 2014 at 9:35

[GoogleCodeExporter]

NOTE: This is from t/re/pat_advanced.t in the core unit tests.

SECTION: my $message = "Folding matches and Unicode";

Original comment by todd.e.rinaldo on 9 May 2014 at 9:36

[GoogleCodeExporter]

It crashes in swash_init, where the op_ppaddr of the start nextstate op of the 
compiled utf8::SWASHNEW is 0xb5.
because the initialization of the op_ppaddr happens later, in line 38635
with
        for( i = 0; i < 813; ++i ) {
            cop_list[i].op_ppaddr = PL_ppaddr[PTR2IV(cop_list[i].op_ppaddr)];
        }

We need to postpone Perl_pregcomp to init2

(gdb) p *PL_op
$4 = {op_next = 0xaf3be0 <op_list+160>, op_sibling = 0xb1cd40 <binop_list>, 
op_ppaddr = 0xb5, op_targ = 0, op_type = 181, op_opt = 0, 
  op_latefree = 1, op_latefreed = 0, op_attached = 0, op_spare = 0, op_flags = 1 '\001', op_private = 0 '\000'}

#1  0x00000000005b9c31 in Perl_runops_debug () at dump.c:2266
#2  0x00000000004f5cae in Perl_call_sv (sv=0xbeb078, flags=2) at perl.c:2648
#3  0x000000000078b027 in Perl_swash_init (pkg=0x8bb0d4 "utf8", name=0x8bb4ae 
"ToFold", listsv=0xba51f0 <PL_sv_undef>, minbits=4, none=0)
    at utf8.c:2099
#4  0x0000000000788580 in Perl_to_utf8_case (p=0x7fffffff9130 "Ā\276", 
ustrp=0x7fffffff9130 "Ā\276", lenp=0x7fffffff8f58, 
    swashp=0xba5660 <PL_utf8_tofold>, normal=0x8bb4ae "ToFold", special=0x8bb49d "utf8::ToSpecFold") at utf8.c:1855
#5  0x0000000000788b72 in Perl__to_utf8_fold_flags (p=0x7fffffff9130 "Ā\276", 
ustrp=0x7fffffff9130 "Ā\276", lenp=0x7fffffff8f58, 
    flags=1 '\001') at utf8.c:2041
#6  0x0000000000787741 in Perl__to_uni_fold_flags (c=256, p=0x7fffffff9130 
"Ā\276", lenp=0x7fffffff8f58, flags=1 '\001') at utf8.c:1349
#7  0x0000000000597182 in S_regatom (pRExC_state=0x7fffffff9c50, 
flagp=0x7fffffff91bc, depth=4) at regcomp.c:8875
#8  0x0000000000590859 in S_regpiece (pRExC_state=0x7fffffff9c50, 
flagp=0x7fffffff92a0, depth=3) at regcomp.c:7437
#9  0x00000000005901f3 in S_regbranch (pRExC_state=0x7fffffff9c50, 
flagp=0x7fffffff93c8, first=1, depth=2) at regcomp.c:7379
#10 0x000000000058eae6 in S_reg (pRExC_state=0x7fffffff9c50, paren=0, 
flagp=0x7fffffff9794, depth=1) at regcomp.c:7190
#11 0x00000000005814d9 in Perl_re_compile (pattern=0xbeb0c0, orig_pm_flags=4) 
at regcomp.c:4663
#12 0x00000000005802af in Perl_pregcomp (pattern=0xbeb0c0, flags=4) at 
regcomp.c:4514
#13 0x0000000000473b89 in perl_init_aaab ()
#14 0x00000000004c31c5 in perl_init ()
#15 0x00000000004c55b9 in main ()

Original comment by reini.urban on 12 May 2014 at 4:18

• Changed state: Started

[GoogleCodeExporter]

So we're not going to be able to pre-compile regexes because of this? They have 
to happen at init for now on?

Original comment by todd.e.rinaldo on 12 May 2014 at 4:22

[GoogleCodeExporter]

Postponing all pregcomp calls to init2 fixed this issue, but caused more 
trouble.

Fixed it by adding a new init0 section which will be called by perl_init before 
perl_init_aaaa. Smoking branch rx-i330...

Original comment by reini.urban on 12 May 2014 at 5:05

[GoogleCodeExporter]

Original comment by reini.urban on 12 May 2014 at 5:06

• Changed title: /i regex causes segfault in uninitialized utf8::SWASHINIT opaddr

[GoogleCodeExporter]

Fixed in branch rx-init0-i330. pm14 and core tests all pass.

commit 59f33ec15085094113a43424a7ccba39cd6d4eed
Author: Reini Urban <rurban@cpanel.net>
Date:   Mon May 12 12:26:13 2014 -0500

    C 1.45_13: initialize init0 for fixup_ppaddr before init

    add a new init0 section, and initialize it with perl_init.
    Needed to set the op_ppaddr when a init function calls an already
    compiled function, such as CALLREGCOMP() calling utf8::SWASHINIT
    when doing a m//i.
    We cannot postpone CALLREGCOMP() to init2, so we need to init the op_ppaddr
    before when using -fppaddr/-O1.

Original comment by reini.urban on 12 May 2014 at 5:51

• Changed title: /i unicode regex causes segfault in uninitialized utf8::SWASHINIT opaddr
• Changed state: Fixed
• Added labels: Milestone-NextRelease

[GoogleCodeExporter]

Still not fixed for 5.18, where the re-assignment to PL_utf8_tofold fails when 
free'ing the statically saved PL_utf8_tofold swash (ToCf - casefolding swash).

Initializing it dynamically as done with commit 
2e6fe7965720eef3136ba615f3011d405edb20ef
Author: Reini Urban <rurban@cpanel.net>
Date:   Tue May 13 18:21:17 2014 -0500

    C: >=5.18 init utf8_tofold swash dynamically to avoid free on static errors

    We need this swash with utf8 m//i, but since the &PL_utf8_tofold is empty
    we need to init it somehow, to avoid free static pv2... errors (the old swash).
    This does not save the folding tables into the binary.
    We really need to init utf8_tofold statically (TODO)

failed in a variety of modules: Pod::Simple Test::NoWarnings Text::Balanced 
Path::Class Test::Pod MooseX::Types

The better fix is to look for the ToCf key in utf8::SWASHNEW->save and set 
PL_utf8_tofold to this static swash.

Original comment by reini.urban on 14 May 2014 at 2:06

• Changed state: Started

[GoogleCodeExporter]

Fixed with 

commit 4871f4573204e97731ce46d5367709969a93bb73
Author: Reini Urban <rurban@cpanel.net>
Date:   Wed May 14 09:07:59 2014 -0500

    C: fix #330 with 5.18. set the PL_utf8_tofold ToCf swash statically

    dynamically did not work as the existing swash got re-assigned, and the free
    of the static keys failed.
    Look now for the ToCf key in utf8::SWASHNEW->save and set PL_utf8_tofold to
    this static swash.
    Fixes issue 330 on 5.18, fixes the d676db56cf8d regressions.

and tested ok with 5.18 and 5.20

Original comment by reini.urban on 14 May 2014 at 2:43

• Changed state: Fixed

[GoogleCodeExporter]

This issue was closed by revision 4871f4573204.

Original comment by reini.urban on 14 May 2014 at 4:28