perl5 segfaults (perl5 versions gentoo(5.20.0, 5.22.0), guix (5.22.1)) without message

p5pRT commented 8 years ago

Migrated from rt.perl.org#128740 (status was 'resolved')

Searchable as RT128740$

p5pRT commented 8 years ago

From ng0@we.make.ritual.n0.is

Hello\,

I am packaging the Net::PSYC application suite for Guix and Gentoo\, on both systems I run into segfaults with the binary "psycion". The program in question can be fetched on Gentoo this way: layman -a youbroketheinternet; emerge --ask dev-perl/Net-PSYC

On Guix you have to look at the perl-Curses and perl-net-psyc patches which have not been merged yet\, which is why I left the report out and focus on Gentoo.

The source on CPAN (https://metacpan.org/pod/Net::PSYC) is outdated\, it is selfhosted these days: http://perl.psyc.eu \, we used the latest git checkout for this debugging (git://git.psyced.org/git/perlpsyc or git://cheettyiapsyciew.onion/perlpsyc - you want to compare with the website for eventual typos I made).

Output below was captured on the Gentoo developing system\, first with torsocks-1.2-r2\, later with an updated release candidate version.

The developer runs 5.22.0 of perl5 on Gentoo\, I run 5.20.0 on Gentoo and on Guix 5.22.1. torsocks versions differ\, ssl used differs (openssl\, libressl)\, but the segfault happens on every system.

My experience with perl5 is limited to packaging for Guix and Gentoo and using it\, not developing for it. The message of the developer was that perl5 should never segfault or coredump and provide an error message if it does - which in this case it doesn't. My thoughts on the message at the end is that this could mean anything or nothing - I get the torsocks error in daily use with other applications occasionally\, but it never affected functionality.

As the Gentoo ebuild is not yet finished and we do not use the updated Makefile of the application: IO::Socket:SSL in the latest version was used\, same for Curses perl module which are both the minimum for psycion.

The main issue is with "torify psycion" with an defined URI of psyc://loupsycedyglgamf.onion/~username \, for psyced.org we realize that the application needs an update as psyced is very strict about the types of secure connections (ciphers etc).

ng0@shikahr ~ $ gdb --silent --args perl /usr/bin/psycion Reading symbols from perl...Reading symbols from /usr/lib64/debug//usr/bin/perl.debug...done. done. (gdb) run Starting program: /usr/bin/perl /usr/bin/psycion [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Neither you have a password in ~/.psyc/auth nor did you specify it with -w.

psyc://loupsycedyglgamf.onion/~ng0> Program received signal SIGSEGV\, Segmentation fault. S_space_join_names_mortal (array=0x0) at pp_sys.c:4654 4654 if (array && *array) { (gdb) bt #0 S_space_join_names_mortal (array=0x0) at pp_sys.c:4654 #1 0x00007ffff788346f in Perl_pp_ghostent () at pp_sys.c:4739 #2 0x00007ffff77c1024 in Perl_runops_debug () at dump.c:2427 #3 0x00007ffff7743925 in S_run_body (oldscope=1) at perl.c:2456 #4 perl_run (my_perl=\) at perl.c:2372 #5 0x0000000000400d9b in main (argc=2\, argv=0x7fffffffe148\, env=0x7fffffffe160) at perlmain.c:114

This is perl 5\, version 20\, subversion 2 (v5.20.2) built for x86_64-linux-debug (with 27 registered patches\, see perl -V for more detail)

Perl may be copied only under the terms of either the Artistic License or the GNU General Public License\, which may be found in the Perl 5 source kit.

Complete documentation for Perl\, including FAQ lists\, should be found on this system using "man perl" or "perldoc perl". If you have access to the Internet\, point your browser at http://www.perl.org/\, the Perl Home Page.

Summary of my perl5 (revision 5 version 20 subversion 2) configuration:

Platform: osname=linux\, osvers=4.4.6-gentoo\, archname=x86_64-linux-debug uname='linux shikahr 4.4.6-gentoo #1 smp wed jul 20 18:09:08 utc 2016 x86_64 intel(r) core(tm)2 cpu t5600 @ 1.83ghz genuineintel gnulinux ' config_args='-des -Duseshrplib -Darchname=x86_64-linux-debug -Dcc=x86_64-pc-linux-gnu-gcc -Doptimize=-O2 -pipe -march=native -ggdb -g -Dldflags=-Wl\,-O1 -Wl\,--as-needed -Dprefix=/usr -Dinstallprefix=/usr -Dsiteprefix=/usr/local -Dvendorprefix=/usr -Dscriptdir=/usr/bin -Dprivlib=/usr/lib64/perl5/5.20.2 -Darchlib=/usr/lib64/perl5/5.20.2/x86_64-linux-debug -Dsitelib=/usr/local/lib64/perl5/5.20.2 -Dsitearch=/usr/local/lib64/perl5/5.20.2/x86_64-linux-debug -Dvendorlib=/usr/lib64/perl5/vendor_perl/5.20.2 -Dvendorarch=/usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-debug -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dvendorman1dir=/usr/share/man/man1 -Dvendorman3dir=/usr/share/man/man3 -Dman1ext=1 -Dman3ext=3pm -Dlibperl=libperl.so.5.20.2 -Dlocincpth=/usr/include -Dglibpth=/lib64 /usr/lib64 -Duselargefiles -Dd_semctl_semun -Dcf_by=Gentoo -Dmyhostname=localhost -Dperladmin=root@localhost -Dinstallusrbinperl=n -Ud_csh -Uusenm -Di_ndbm -Di_gdbm -Di_db -DDEBUGGING -Dinc_version_list=5.20.0/x86_64-linux-debug 5.20.0 5.20.1/x86_64-linux-debug 5.20.1 -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Dnoextensions=ODBM_File' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='x86_64-pc-linux-gnu-gcc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-O2 -pipe -march=native -ggdb -g'\, cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe' ccversion=''\, gccversion='4.9.3'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='x86_64-pc-linux-gnu-gcc'\, ldflags ='-Wl\,-O1 -Wl\,--as-needed' libpth=/usr/local/lib64 /lib64 /usr/lib64 /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/include-fixed /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib libs=-lnsl -lnm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lnsl -lnm -ldl -lm -lcrypt -lutil -lc libc=libc-2.22.so\, so=so\, useshrplib=true\, libperl=libperl.so.5.20.2 gnulibc_version='2.22' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -O2 -pipe -march=native -ggdb -g -Wl\,-O1 -Wl\,--as-needed'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF Locally applied patches: gentoo/hints_hpux - Fix hpux hints gentoo/aix_soname - aix gcc detection and shared library soname support gentoo/EUMM-RUNPATH - https://bugs.gentoo.org/105054 cpan/ExtUtils-MakeMaker: drop $PORTAGE_TMPDIR from LD_RUN_PATH gentoo/config_over - Remove -rpath and append LDFLAGS to lddlflags gentoo/opensolaris_headers - [PATCH] Add headers for opensolaris gentoo/patchlevel - List packaged patches for perl-5.20.2(#1) in patchlevel.h gentoo/cpanplus_definstalldirs - Configure CPANPLUS to use the site directories by default. gentoo/cleanup-paths - [PATCH] Cleanup PATH and shrpenv gentoo/enc2xs - Tweak enc2xs to follow symlinks and ignore missing @INC directories. gentoo/enc2xs_checksums - gentoo/darwin-cc-ld - https://bugs.gentoo.org/297751 [PATCH] darwin: Use $CC to link gentoo/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN. gentoo/interix - [PATCH] Fix interix hints gentoo/create_libperl_soname - https://bugs.gentoo.org/286840 [PATCH] Set libperl soname gentoo/mod_paths - Add /etc/perl to @INC gentoo/EUMM_delete_packlist - gentoo/drop_fstack_protector - https://bugs.gentoo.org/348557 [PATCH] Don't force -fstack-protector on everyone gentoo/usr_local - [PATCH] Remove /usr/local paths gentoo/D-SHA-CFLAGS - https://bugs.gentoo.org/506818 [PATCH] Do not set custom CFLAGS in cpan/Digest-SHA gentoo/io_socket_ip_tests - debian/cpan-missing-site-dirs - Fix CPAN::FirstTime defaults with nonexisting site dirs if a parent is writable debian/regcomp-mips-optim - Downgrade the optimization of regcomp.c on mips and mipsel due to a gcc-4.9 bug debian/perldoc-less-R - Tell the 'less' pager to allow terminal escape sequences debian/makemaker-pasthru - Pass LD settings through to subdirectories fixes/net_smtp_docs - [rt.cpan.org #36038] Document the Net::SMTP 'Port' option fixes/memoize_storable_nstore - [rt.cpan.org #77790] Memoize::Storable: respect 'nstore' option not respected fixes/document_makemaker_ccflags - [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags} Built under linux Compiled at Jul 25 2016 22:02:50 @INC: /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-debug /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-debug /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-debug /usr/lib64/perl5/5.20.2 .

Upgraded to torsocks-2.2.0-rc1: ng0@shikahr ~ $ . torsocks on; gdb --silent --args perl /usr/bin/psycion Reading symbols from perl...Reading symbols from /usr/lib64/debug//usr/bin/perl.debug...done. done. (gdb) run Starting program: /usr/bin/perl /usr/bin/psycion [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Neither you have a password in ~/.psyc/auth nor did you specify it with -w.

psyc://loupsycedyglgamf.onion/~ng0> Program received signal SIGSEGV\, Segmentation fault. S_space_join_names_mortal (array=0x0) at pp_sys.c:4654 4654 if (array && *array) { (gdb) bt #0 S_space_join_names_mortal (array=0x0) at pp_sys.c:4654 #1 0x00007ffff788046f in Perl_pp_ghostent () at pp_sys.c:4739 #2 0x00007ffff77be024 in Perl_runops_debug () at dump.c:2427 #3 0x00007ffff7740925 in S_run_body (oldscope=1) at perl.c:2456 #4 perl_run (my_perl=\) at perl.c:2372 #5 0x0000000000400d9b in main (argc=2\, argv=0x7fffffffe128\, env=0x7fffffffe140) at perlmain.c:114

(gdb) quit A debugging session is active.

Inferior 1 [process 28129] will be killed.

Quit anyway? (y or n) y 1469533497 WARNING torsocks[28127]: [syscall] Unsupported syscall number 200. Denying the call (in tsocks_syscall() at syscall.c:465)

thanks\, -- ♥Ⓐ ng0 Current Keys: https://we.make.ritual.n0.is/ng0.txt For non-prism friendly talk find me on http://www.psyced.org

p5pRT commented 8 years ago

From zefram@fysh.org

ng0 wrote:

                                                                                   S\_space\_join\_names\_mortal \(array=0x0\) at pp\_sys\.c&#8203;:4654

4654 if (array && *array) {

The code in the body of this function is prepared for the array argument to be null\, as it is\, and on its own would handle that without difficulty. But in embed.fnc the parameter is declared "NN"\, so the assertions at the top of the function assert that it's not null. In this build\, obviously that doesn't result in checking the assertion and declaring it failed; instead\, the compiler has used the assertion to optimise out the explicit check for the argument being null. The explicit check has subsequently been removed from the source\, by commit 3dc78631ef8 in 5.21.10\, on the strength of the "NN" declaration.

The null arises from the h_aliases element of struct hostent. None of the documentation that I can find admits the possibility of this being null. (It's supposedly a pointer to a null-terminated array.) This would explain the "NN" declaration. However\, since this bug report shows that it can actually be null in the wild on real libcs\, it seems that we should reevaluate that.

I suggest that we should reinstate the "array &&" check on that line\, and remove the "NN" declaration from embed.fnc. (This would also cause the assertion macro to vanish.)

Note for reporter: if it were not this straightforward\, we would probably have rejected the bug report because of the heavy code dependencies\, especially the use of XS modules. XS modules can easily make perl crash\, and the general statement that perl should never crash doesn't apply if the crash can be attributed to them. So we would have asked you to reduce your test case to something using only the perl core\, or at least not using any XS modules\, with a view to blaming the XS modules that you're using if you couldn't so reduce it. Please bear this in mind for future bug reports. But thanks for including so much information in this report; that's what made my diagnosis possible.

-zefram

p5pRT commented 8 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 8 years ago

From @cpansprout

On Tue Jul 26 07:05:30 2016\, zefram@fysh.org wrote:

I suggest that we should reinstate the "array &&" check on that line\, and remove the "NN" declaration from embed.fnc. (This would also cause the assertion macro to vanish.)

Please review the attached patch. I am not familiar with this sort of thing\, so I would like to make sure the commit message makes sense.

--

Father Chrysostomos

p5pRT commented 8 years ago

From @cpansprout

On Sun Jul 31 19:25:50 2016\, sprout wrote:

On Tue Jul 26 07:05:30 2016\, zefram@fysh.org wrote:

I suggest that we should reinstate the "array &&" check on that line\, and remove the "NN" declaration from embed.fnc. (This would also cause the assertion macro to vanish.)

Please review the attached patch. I am not familiar with this sort of thing\, so I would like to make sure the commit message makes sense.

Now I’m doing it! Here is the attachment.

--

Father Chrysostomos

p5pRT commented 8 years ago

From @cpansprout

From 65acdc751c6469b9ddf45214ce74e1547d90f183 Mon Sep 17 00:00:00 2001 From: Father Chrysostomos \sprout@cpan\.org Date: Sun\, 31 Jul 2016 19:21:02 -0700

[perl #128740] Check for null in pp_ghostent et al.

Specifically in the S_space_join_names_mortal static function that several pp functions call. On some platforms (such as Gentoo Linux)\, hent->h_aliases (where hent is a struct hostent *) may be null after a gethostent call.

Inline Patch

```diff diff --git a/embed.fnc b/embed.fnc index c0b7a3e..2021b3e 100644 --- a/embed.fnc +++ b/embed.fnc @@ -2132,7 +2132,7 @@ s |OP* |doform |NN CV *cv|NN GV *gv|NULLOK OP *retop # if !defined(HAS_MKDIR) || !defined(HAS_RMDIR) sR |int |dooneliner |NN const char *cmd|NN const char *filename # endif -s |SV * |space_join_names_mortal|NN char *const *array +s |SV * |space_join_names_mortal|NULLOK char *const *array #endif p |OP * |tied_method|NN SV *methname|NN SV **sp \ |NN SV *const sv|NN const MAGIC *const mg \ diff --git a/pp_sys.c b/pp_sys.c index 3bf2673..d16a0e5 100644 --- a/pp_sys.c +++ b/pp_sys.c @@ -4934,9 +4934,7 @@ S_space_join_names_mortal(pTHX_ char *const *array) { SV *target; - PERL_ARGS_ASSERT_SPACE_JOIN_NAMES_MORTAL; - - if (*array) { + if (array && *array) { target = newSVpvs_flags("", SVs_TEMP); while (1) { sv_catpv(target, *array); diff --git a/proto.h b/proto.h index a06b6d0..da11ced 100644 --- a/proto.h +++ b/proto.h @@ -4859,8 +4859,6 @@ STATIC OP* S_doform(pTHX_ CV *cv, GV *gv, OP *retop); #define PERL_ARGS_ASSERT_DOFORM \ assert(cv); assert(gv) STATIC SV * S_space_join_names_mortal(pTHX_ char *const *array); -#define PERL_ARGS_ASSERT_SPACE_JOIN_NAMES_MORTAL \ - assert(array) #endif #if defined(PERL_IN_REGCOMP_C) STATIC SV* S__make_exactf_invlist(pTHX_ RExC_state_t *pRExC_state, regnode *node) ```

p5pRT commented 8 years ago

From @tonycoz

On Tue Jul 26 07:05:30 2016\, zefram@fysh.org wrote:

The null arises from the h_aliases element of struct hostent. None of the documentation that I can find admits the possibility of this being null. (It's supposedly a pointer to a null-terminated array.) This would explain the "NN" declaration. However\, since this bug report shows that it can actually be null in the wild on real libcs\, it seems that we should reevaluate that.

I suspect it's not a real libc\, but torsocks:

https://gitweb.torproject.org/torsocks.git/tree/src/lib/gethostbyname.c#n92

torify is a command that LD_PRELOADs libtorsocks\, which replaces gethostbyname() etc.

Tony

p5pRT commented 8 years ago

From zefram@fysh.org

Father Chrysostomos via RT wrote:

Please review the attached patch.

Looks good to me.

-zefram

p5pRT commented 8 years ago

From zefram@fysh.org

Tony Cook via RT wrote:

I suspect it's not a real libc\, but torsocks:

Ah\, yes. Should probably be reported as a bug in torsocks\, then. (In addition to our change to liberally accept null.)

-zefram

p5pRT commented 8 years ago

From @cpansprout

On Sun Jul 31 23:46:48 2016\, zefram@fysh.org wrote:

Father Chrysostomos via RT wrote:

Please review the attached patch.

Looks good to me.

Thank you. Now applied as d35c1b5. I propose we backport this to the maint branches\, but probably not till after the imminent releases.

--

Father Chrysostomos

p5pRT commented 8 years ago

@cpansprout - Status changed from 'open' to 'pending release'

p5pRT commented 8 years ago

From ng0@we.make.ritual.n0.is

Zefram via RT \perlbug\-followup@perl\.org writes:

Tony Cook via RT wrote:

I suspect it's not a real libc\, but torsocks:

Ah\, yes. Should probably be reported as a bug in torsocks\, then. (In addition to our change to liberally accept null.)

-zefram

Thank you all for your work on fixing this bug.

I will a get in contact with torsocks developers to address the bug on their side too. -- ♥Ⓐ ng0 Current Keys: https://we.make.ritual.n0.is/ng0.txt For non-prism friendly talk find me on http://www.psyced.org

p5pRT commented 7 years ago

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0\, this and 210 other issues have been resolved.

Perl 5.26.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists\, feel free to reopen this ticket.

p5pRT commented 7 years ago

@khwilliamson - Status changed from 'pending release' to 'resolved'

Perl / perl5