segfault on perl 5.8.4

[p5pRT]

Migrated from rt.perl.org#29437 (status was 'resolved')

Searchable as RT29437$

[p5pRT]

From gotar@polanet.pl

Hi!

Here's the sample code​:

#v+ #!/usr/bin/perl

use open OUT => '​:locale';

open F\,'/etc/shadow'; warn "$!\n"; #v-

If one runs it from unprivileged account he gets "Permission denied" UNLESS he had set locale producing error message with national letters\, e.g. "Brak dostępu" in polish (eogonek between 't' and 'p')​:

LC_ALL=pl_PL ./perl-test zsh​: segmentation fault LC_ALL=pl_PL ./perl-test

-- Tom Pala \gotar@​pld\-linux\.org http​://vfmg.sourceforge.net/   http​://www.pld-linux.org/Members/gotar/

[p5pRT]

From radek42@gmail.com

(Probably) related​:

$ perl -Mencoding=latin2 -Mopen=​:locale -wle 'print "\x{ea}"' "\x{a2ad}" does not map to iso-8859-2 at -e line 1. panic​: sv_setpvn called with negative strlen at -e line 1. "\x{a2aa}" does not map to iso-8859-2. \x{a2aa} $ perl -Mopen=​:locale -Mencoding=latin2 -wle 'print "\x{ea}"' ę

"\x{ea}" eq "\N{LATIN SMALL LETTER E WITH OGONEK}" in the iso-8859-2 charset. Using the "\N{}" notation gives exactly the same output in the examples above.

My setup​:

$ locale LANG=pl_PL.ISO-8859-2 LC_CTYPE="pl_PL.ISO-8859-2" LC_NUMERIC="pl_PL.ISO-8859-2" LC_TIME="pl_PL.ISO-8859-2" LC_COLLATE="pl_PL.ISO-8859-2" LC_MONETARY="pl_PL.ISO-8859-2" LC_MESSAGES="pl_PL.ISO-8859-2" LC_PAPER="pl_PL.ISO-8859-2" LC_NAME="pl_PL.ISO-8859-2" LC_ADDRESS="pl_PL.ISO-8859-2" LC_TELEPHONE="pl_PL.ISO-8859-2" LC_MEASUREMENT="pl_PL.ISO-8859-2" LC_IDENTIFICATION="pl_PL.ISO-8859-2" LC_ALL=

$ perl -V Summary of my perl5 (revision 5 version 8 subversion 4) configuration​:   Platform​:   osname=linux\, osvers=2.6.5\, archname=athlon-pld-linux-thread-multi   uname='linux bongo 2.6.5 #1 sun apr 4 18​:56​:17 utc 2004 i686 amd_athlon(tm)_xp_1700+ unknown pld linux '   config_args='-des -Dcc=athlon-pld-linux-gcc -Darchname=athlon-pld-linux -Dcccdlflags=-fPIC -Dccdlflags=-rdynamic -Doptimize=-O2 -march=athlon -Duseshrplib -Dd_dosuid -Dman1dir=/usr/share/man/man1 -Dman1ext=1 -Dman3dir=/usr/share/man/man3 -Dman3ext=3perl -Dvendorman1=/usr/share/man/man1 -Dvendorman1ext=1p -Dvendorman3=/usr/share/man/man3 -Dvendorman3ext=3pm -Dsiteman1=/usr/local/share/man/man1 -Dsiteman1ext=1p -Dsiteman3=/usr/local/share/man/man3p -Dsiteman3ext=3pm -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dlibpth=/usr/lib /lib -Dprivlib=/usr/share/perl5/5.8.4 -Darchlib=/usr/lib/perl5/5.8.4/athlon-pld-linux-thread-multi -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib/perl5/5.8.0/athlon-pld-linux-thread-multi -Dvendorlib=/usr/share/perl5/vendor_perl -Dvendorarch=/usr/lib/perl5/vendor_perl/5.8.0/athlon-pld-linux-thread-multi -Ui_db -Ui_dbm -Di_gdbm -Ui_ndbm -Dlibswanted=dl m c crypt gdbm -Dusethreads -Duselargefiles'   hint=recommended\, useposix=true\, d_sigaction=define   usethreads=define use5005threads=undef useithreads=define usemultiplicity=define   useperlio=define d_sfio=undef uselargefiles=define usesocks=undef   use64bitint=undef use64bitall=undef uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='athlon-pld-linux-gcc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,   optimize='-O2 -march=athlon'\,   cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include'   ccversion=''\, gccversion='3.3.3 (PLD Linux)'\, gccosandvers=''   intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12   ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=4\, prototype=define   Linker and Libraries​:   ld='athlon-pld-linux-gcc'\, ldflags =''   libpth=/usr/lib /lib   libs=-ldl -lm -lpthread -lc -lcrypt -lgdbm   perllibs=-ldl -lm -lpthread -lc -lcrypt   libc=/lib/libc-2.3.3.so\, so=so\, useshrplib=true\, libperl=libperl.so   gnulibc_version='2.3.3'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-rdynamic -Wl\,-rpath\,/usr/lib/perl5/5.8.4/athlon-pld-linux-thread-multi/CORE'   cccdlflags='-fPIC'\, lddlflags='-shared'

Characteristics of this binary (from libperl)​:   Compile-time options​: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL_IMPLICIT_CONTEXT   Built under linux   Compiled at Apr 26 2004 19​:54​:11   @​INC​:   /usr/local/lib/perl5/5.8.0/athlon-pld-linux-thread-multi   /usr/local/share/perl5   /usr/lib/perl5/vendor_perl/5.8.0/athlon-pld-linux-thread-multi   /usr/share/perl5/vendor_perl   /usr/lib/perl5/5.8.4/athlon-pld-linux-thread-multi   /usr/share/perl5/5.8.4   .

[p5pRT]

From @iabyn

On Sat\, May 08\, 2004 at 10​:26​:03AM -0000\, Tomasz Pala wrote​:

#!/usr/bin/perl

use open OUT => '​:locale';

open F\,'/etc/shadow'; warn "$!\n"; #v-

If one runs it from unprivileged account he gets "Permission denied" UNLESS he had set locale producing error message with national letters\, e.g. "Brak dost??pu" in polish (eogonek between 't' and 'p')​:

LC_ALL=pl_PL ./perl-test zsh​: segmentation fault LC_ALL=pl_PL ./perl-test

I couldn't reproduce this with the code above\, but I could with the following (using the default locale)​:

use open OUT => '​:locale'; print STDERR "\x{100}\n";

The printing calls encode() which does (in the encode_method() C function)​:

Perl_warner (my_perl=0x81a9988\, err=0x2c\, pat=0x4002ebc0 "\"\\x{%04lx}\" does not map to %s")

which then merrily recurses until the stack blows up. SAlso happens on Bleed. I have no particular ideas as to how this should be fixed. Ideas\, anyone?

The error message expands to​:

  "\x{0100}" does not map to iso-8859-1

-- Thank God I'm an atheist.....

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @nwc10

On Sat\, May 08\, 2004 at 10​:26​:03AM -0000\, Tomasz Pala wrote​:

# New Ticket Created by Tomasz Pala # Please include the string​: [perl #29437] # in the subject line of all future correspondence about this issue. # \<URL​: http​://rt.perl.org​:80/rt3/Ticket/Display.html?id=29437 >

Hi!

Here's the sample code​:

#v+ #!/usr/bin/perl

use open OUT => '​:locale';

open F\,'/etc/shadow'; warn "$!\n"; #v-

If one runs it from unprivileged account he gets "Permission denied" UNLESS he had set locale producing error message with national letters\, e.g. "Brak dostępu" in polish (eogonek between 't' and 'p')​:

LC_ALL=pl_PL ./perl-test zsh​: segmentation fault LC_ALL=pl_PL ./perl-test

Hmm. Interesting. I can recreate this. Here's a section of the stack trace​:

#1147 0x080ba90c in Perl_warner (err=44\,   pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s") at util.c​:1346 #1148 0x400bd5c0 in encode_method (enc=0x400f89a8\, dir=0x400f6c00\,   src=0x817d230\, check=258\, offset=0x0\, term=0x0\, retcode=0x0)   at Encode.xs​:154 #1149 0x400bed6a in XS_Encode__XS_encode (cv=0x8202af0) at Encode.xs​:472 #1150 0x080d753c in Perl_pp_entersub () at pp_hot.c​:2798 #1151 0x080b7370 in Perl_runops_debug () at dump.c​:1564 #1152 0x08063ce6 in S_call_body (myop=0xbf818c10\, is_eval=0) at perl.c​:2160 #1153 0x080635b7 in Perl_call_sv (sv=0x966084c\, flags=64) at perl.c​:2039 #1154 0x08063349 in Perl_call_method (methname=0x400cd52d "encode"\, flags=0)   at perl.c​:1972 #1155 0x400cbdf0 in PerlIOEncode_flush (f=0x818592c) at encoding.xs​:419 #1156 0x0814792d in Perl_PerlIO_flush (f=0x818592c) at perlio.c​:1599 #1157 0x0814a997 in PerlIOBuf_write (f=0x818592c\, vbuf=0x966bc80\, count=63)   at perlio.c​:3727 #1158 0x400cca7f in PerlIOEncode_write (f=0x818592c\, vbuf=0x966bc80\, count=63)   at encoding.xs​:582 #1159 0x081477d0 in Perl_PerlIO_write (f=0x818592c\, vbuf=0x966bc80\, count=63)   at perlio.c​:1576 #1160 0x080b96ba in Perl_write_to_stderr (   message=0x966bc80 "\"\\x{ac35}\" does not map to iso-8859-15 at perl-test line 6.\n"\, msglen=63) at util.c​:1021 #1161 0x080bb3f6 in Perl_vwarner (err=44\,   pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s"\, args=0xbf818e34)   at util.c​:1432 #1162 0x080ba90c in Perl_warner (err=44\,   pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s") at util.c​:1346

I'm not sure what the true cause is\, let alone the correct fix\, but it appears that there is an infinite loop within Encode trying to report the error\, and the SEGV occurs when memory (probably the C stack) is exhausted.

Nicholas Clark

[p5pRT]

From mjtg@cam.ac.uk

Nicholas Clark \nick@&#8203;ccl4\.org wrote

#1147 0x080ba90c in Perl_warner (err=44\, pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s") at util.c​:1346   ... #1155 0x400cbdf0 in PerlIOEncode_flush (f=0x818592c) at encoding.xs​:419   ... #1162 0x080ba90c in Perl_warner (err=44\, pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s") at util.c​:1346

Speaking in total ignorance of the code (and far too lazy to look at it)\, let me guess​:

while (1) {   Encode is generating a warning because it finds in its buffer a character   it can't convert.

  The perlIO routines do a flush\, calling on Encode to flush its buffers.

  Encode attempts to process the failing character. }

If so\, the fix is presumably to get Encode to step past the character and/or replace it with something harmless *before* generating the warning.

Mike Guy

[p5pRT]

From @Hugmeir

On Mon May 10 08​:18​:54 2004\, mjtg@​cam.ac.uk wrote​:

Nicholas Clark \nick@&#8203;ccl4\.org wrote

#1147 0x080ba90c in Perl_warner (err=44\, pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s") at util.c​:1346 ... #1155 0x400cbdf0 in PerlIOEncode_flush (f=0x818592c) at encoding.xs​:419 ... #1162 0x080ba90c in Perl_warner (err=44\, pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s") at util.c​:1346

Speaking in total ignorance of the code (and far too lazy to look at it)\, let me guess​:

while (1) { Encode is generating a warning because it finds in its buffer a character it can't convert.

The perlIO routines do a flush\, calling on Encode to flush its

buffers.

Encode attempts to process the failing character\.

}

If so\, the fix is presumably to get Encode to step past the character and/or replace it with something harmless *before* generating the warning.

Mike Guy

I can't reproduce any of the bugs in this ticket with newer Perls; so it seems that Encode was fixed. I vote to close this.

[p5pRT]

@cpansprout - Status changed from 'open' to 'resolved'

[p5pRT]

From @nwc10

On Sun\, Apr 29\, 2012 at 10​:44​:24AM -0700\, Brian Fraser via RT wrote​:

On Mon May 10 08​:18​:54 2004\, mjtg@​cam.ac.uk wrote​:

Nicholas Clark \nick@&#8203;ccl4\.org wrote

#1147 0x080ba90c in Perl_warner (err=44\, pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s") at util.c​:1346 ... #1155 0x400cbdf0 in PerlIOEncode_flush (f=0x818592c) at encoding.xs​:419 ... #1162 0x080ba90c in Perl_warner (err=44\, pat=0x400c1a00 "\"\\x{%04lx}\" does not map to %s") at util.c​:1346

Speaking in total ignorance of the code (and far too lazy to look at it)\, let me guess​:

while (1) { Encode is generating a warning because it finds in its buffer a character it can't convert.

The perlIO routines do a flush\, calling on Encode to flush its

buffers.

Encode attempts to process the failing character\.

}

If so\, the fix is presumably to get Encode to step past the character and/or replace it with something harmless *before* generating the warning.

Mike Guy

I can't reproduce any of the bugs in this ticket with newer Perls; so it seems that Encode was fixed. I vote to close this.

I ran this​:

LC_ALL=pl_PL.ISO8859-2 ../perl/Porting/bisect.pl --expect-fail --start perl-5.8.0 -e 'use open OUT => "​:locale"; print STDERR "\x{100}\n";'

It said this​:

commit 74f6c1ca58b1c40741f55591ab97a77b6751f510 Author​: Slaven Rezic \slaven@&#8203;rezic\.de Date​: Tue Feb 6 00​:04​:07 2007 +0100

  Re​: [perl #41442] segfault (dead loop) with Encoding\, use open :locale\, print STDERR   Message-ID​: \87veiggt2g\.fsf@&#8203;biokovo\.herceg\.de

  p4raw-id​: //depot/perl@​30213

:100644 100644 760c921584ee6ced54e6f51ea2657881178689d3 41874528b76a354ab83c6c332b6e317e60f0fe16 M MANIFEST :040000 040000 fca5fa13398978ebf037e2770ed4dea2ca10f705 2d1b3562c74933e1446f1b09bcc10cedad48bc55 M ext bisect run success That took 1970 seconds

Seems that 41442 and 29437 are duplicates.

I guess that if there is something to learn here\, it's that once 29437 was diagnosed a bit better\, the title of it should have been changed to reflect the cause. Which would have made the title closer to that of 41442's title.

Nicholas Clark