Tests spew unreferenced scalar warnings

p5pRT commented 9 years ago

Migrated from rt.perl.org#124181 (status was 'resolved')

Searchable as RT124181$

p5pRT commented 9 years ago

From @cpansprout

I just noticed this:

ext/XS-Typemap/t/Typemap ...................................... Attempt to free unreferenced scalar: SV 0x7fd68ba42750 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68b885f88 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68ba3fb98 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68ba42600 during global destruction. ok

We need to fix it (or at least look into it) before 5.22.

$ perl5.21.11 -V Summary of my perl5 (revision 5 version 21 subversion 11) configuration: Snapshot of: 9baf31049f1a90bbba573fadb8c62278a52202db Platform: osname=darwin\, osvers=12.5.0\, archname=darwin-2level uname='darwin pint.local 12.5.0 darwin kernel version 12.5.0: sun sep 29 13:33:47 pdt 2013; root:xnu-2050.48.12~1release_x86_64 x86_64 ' config_args='-de -Dusedevel -DDEBUGGING' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-fno-common -DPERL_DARWIN -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'\, optimize='-O3 -g'\, cppflags='-fno-common -DPERL_DARWIN -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.27)'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'\, ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/bin/../lib/clang/4.2/lib /usr/lib libs=-lpthread -ldbm -ldl -lm -lutil -lc perllibs=-lpthread -ldl -lm -lutil -lc libc=\, so=dylib\, useshrplib=false\, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=bundle\, d_dlsymun=undef\, ccdlflags=' ' cccdlflags=' '\, lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Built under darwin Compiled at Mar 25 2015 08:55:47 @INC: /usr/local/lib/perl5/site_perl/5.21.11/darwin-2level /usr/local/lib/perl5/site_perl/5.21.11 /usr/local/lib/perl5/5.21.11/darwin-2level /usr/local/lib/perl5/5.21.11 /usr/local/lib/perl5/site_perl .

--

Father Chrysostomos

p5pRT commented 9 years ago

From @wolfsage

On Wed\, Mar 25\, 2015 at 12:22 PM\, Father Chrysostomos \perlbug\-followup@perl\.org wrote:

I just noticed this:

ext/XS-Typemap/t/Typemap ...................................... Attempt to free unreferenced scalar: SV 0x7fd68ba42750 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68b885f88 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68ba3fb98 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68ba42600 during global destruction. ok

Hrm\, I can't reproduce. I tried your configure args and also ./Configure -de -Dusedevel -DDEBUGGING -Doptimize='-O3 -g' (since I get -O2 by default) and I get no such messages from any test.

./perl -Ilib -V Summary of my perl5 (revision 5 version 21 subversion 11) configuration: Commit id: d8c1af4e0d4774b60bd59dc0039c7e101f292a4a Platform: osname=linux\, osvers=3.13.0-46-generic\, archname=x86_64-linux uname='linux dory 3.13.0-46-generic #79-ubuntu smp tue mar 10 20:06:50 utc 2015 x86_64 x86_64 x86_64 gnulinux ' config_args='-de -Dusedevel -DDEBUGGING -Doptimize=-O3 -g' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-O3 -g'\, cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.8.2'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='cc'\, ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -O3 -g -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Mar 25 2015 14:59:18 @INC: lib /usr/local/lib/perl5/site_perl/5.21.11/x86_64-linux /usr/local/lib/perl5/site_perl/5.21.11 /usr/local/lib/perl5/5.21.11/x86_64-linux /usr/local/lib/perl5/5.21.11 .

-- Matthew Horsfall (alh)

p5pRT commented 9 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 9 years ago

From @jkeenan

On Wed Mar 25 12:09:28 2015\, alh wrote:

On Wed\, Mar 25\, 2015 at 12:22 PM\, Father Chrysostomos \perlbug\-followup@perl\.org wrote:

I just noticed this:

ext/XS-Typemap/t/Typemap ...................................... Attempt to free unreferenced scalar: SV 0x7fd68ba42750 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68b885f88 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68ba3fb98 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68ba42600 during global destruction. ok

Hrm\, I can't reproduce. I tried your configure args and also ./Configure -de -Dusedevel -DDEBUGGING -Doptimize='-O3 -g' (since I get -O2 by default) and I get no such messages from any test.

I tried both with and without -DDEBUGGING in an environment similar to alh's (Linux x86_64). I could not reproduce this warning\, either.

-- James E Keenan (jkeenan@cpan.org)

p5pRT commented 9 years ago

From @cpansprout

On Wed Mar 25 18:47:06 2015\, jkeenan wrote:

On Wed Mar 25 12:09:28 2015\, alh wrote:

Hrm\, I can't reproduce. I tried your configure args and also ./Configure -de -Dusedevel -DDEBUGGING -Doptimize='-O3 -g' (since I get -O2 by default) and I get no such messages from any test.

I tried both with and without -DDEBUGGING in an environment similar to alh's (Linux x86_64). I could not reproduce this warning\, either.

That puts the onus on me to debug it\, then. :-( I can reproduce this with multiple compilers and multiple configurations. Maybe it is Mac-specific.

--

Father Chrysostomos

p5pRT commented 9 years ago

From @tonycoz

On Wed Mar 25 22:11:21 2015\, sprout wrote:

On Wed Mar 25 18:47:06 2015\, jkeenan wrote:

On Wed Mar 25 12:09:28 2015\, alh wrote:

Hrm\, I can't reproduce. I tried your configure args and also ./Configure -de -Dusedevel -DDEBUGGING -Doptimize='-O3 -g' (since I get -O2 by default) and I get no such messages from any test.

I tried both with and without -DDEBUGGING in an environment similar to alh's (Linux x86_64). I could not reproduce this warning\, either.

That puts the onus on me to debug it\, then. :-( I can reproduce this with multiple compilers and multiple configurations. Maybe it is Mac- specific.

I just tried to reproduce this on my Macs and didn't see the problem.

My macs are running OS X 10.10.2 (uname -r 14.1.0) and 10.9.5 (13.4.0)\, from your -V output it looks like you're still running 10.8.

Tony

p5pRT commented 9 years ago

From @bulk88

On Wed Mar 25 22:11:21 2015\, sprout wrote:

That puts the onus on me to debug it\, then. :-( I can reproduce this with multiple compilers and multiple configurations. Maybe it is Mac- specific.

DEBUG_LEAKING_SCALARS+conditional breakpoints in SvREFCNT_dec and newSV (or equivelent)\, set on the absoolute ptr addr of the SV being in var sv. Atleast for me on Win32 Perl\, SV*s are identical from run to run assuming all inputs/PP/XS code is the same.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

From @jkeenan

On Wed Mar 25 09:22:14 2015\, sprout wrote:

I just noticed this:

ext/XS-Typemap/t/Typemap ...................................... Attempt to free unreferenced scalar: SV 0x7fd68ba42750 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68b885f88 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68ba3fb98 during global destruction. Attempt to free unreferenced scalar: SV 0x7fd68ba42600 during global destruction. ok

We need to fix it (or at least look into it) before 5.22.

FWIW\, I was unable to reproduce these warnings on an older Darwin.

-- James E Keenan (jkeenan@cpan.org)

p5pRT commented 9 years ago

From @jkeenan

Summary of my perl5 (revision 5 version 21 subversion 11) configuration: Commit id: 497a6d903cbb311376e7be948d0c6267f2378901 Platform: osname=darwin\, osvers=8.11.0\, archname=darwin-2level uname='darwin macintosh-9.local 8.11.0 darwin kernel version 8.11.0: wed oct 10 18:26:00 pdt 2007; root:xnu-792.24.17~1release_ppc power macintosh powerpc ' config_args='-des -Dusedevel -DDEBUGGING' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=undef\, use64bitall=undef\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-fno-common -DPERL_DARWIN -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -I/opt/local/include -D_FORTIFY_SOURCE=2'\, optimize='-O3 -g'\, cppflags='-fno-common -DPERL_DARWIN -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -I/opt/local/include' ccversion=''\, gccversion='4.0.1 (Apple Computer\, Inc. build 5250)'\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=4321\, doublekind=4 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=6 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'\, ldflags =' -L/usr/local/lib -L/opt/local/lib' libpth=/usr/local/lib /usr/lib /opt/local/lib libs=-lpthread -ldbm -ldl -lm -lc perllibs=-lpthread -ldl -lm -lc libc=\, so=dylib\, useshrplib=false\, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=bundle\, d_dlsymun=undef\, ccdlflags=' ' cccdlflags=' '\, lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Built under darwin Compiled at Apr 6 2015 19:18:11 %ENV: PERLBREW_BASHRC_VERSION="0.59" PERLBREW_HOME="/Users/jimk/.perlbrew" PERLBREW_ROOT="/Users/jimk/perl5/perlbrew" PERL_WORKDIR="gitwork/perl" @INC: lib /usr/local/lib/perl5/site_perl/5.21.11/darwin-2level /usr/local/lib/perl5/site_perl/5.21.11 /usr/local/lib/perl5/5.21.11/darwin-2level /usr/local/lib/perl5/5.21.11 /usr/local/lib/perl5/site_perl .

p5pRT commented 9 years ago

From @hvds

I'm able to reproduce this. Here are cut-down versions of the 4 cases:

% PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")' Attempt to free unreferenced scalar: SV 0x1d1b6a8 during global destruction. % PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_INOUT($fh); close $fh' Attempt to free unreferenced scalar: SV 0x2844760 during global destruction. % PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "\<"\, \$buf; XS::Typemap::T_IN($fh); close $fh' Attempt to free unreferenced scalar: SV 0x2748628 during global destruction. % PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_OUT($fh); close $fh' Attempt to free unreferenced scalar: SV 0xa1c7a8 during global destruction. %

% ./perl -Ilib -V Summary of my perl5 (revision 5 version 22 subversion 0) configuration: Commit id: 416c06fc852c00a948ade2fd39e7bf1b9f00ca9d Platform: osname=linux\, osvers=3.13.0-37-generic\, archname=x86_64-linux uname='linux shad2 3.13.0-37-generic #64-ubuntu smp mon sep 22 21:28:38 utc 2014 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dcc=gcc -Dprefix=/opt/blead-d -Doptimize=-g -O6 -DDEBUGGING -Dusedevel -Uversiononly' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='gcc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-g -O6'\, cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.8.2'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='gcc'\, ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -g -O6 -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Locally applied patches: RC0 Built under linux Compiled at May 9 2015 09:35:23 %ENV: PERL_TEST_MEMORY="8" @INC: lib /opt/blead-d/lib/perl5/site_perl/5.22.0/x86_64-linux /opt/blead-d/lib/perl5/site_perl/5.22.0 /opt/blead-d/lib/perl5/5.22.0/x86_64-linux /opt/blead-d/lib/perl5/5.22.0 /opt/blead-d/lib/perl5/site_perl/5.21.9 /opt/blead-d/lib/perl5/site_perl . %

Not sure I know how to diagnose this\, but here's a stack trace for the complaint on the first example:

PERL_DESTRUCT_LEVEL=2 gdb ./perl GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7 Copyright (C) 2014 Free Software Foundation\, Inc. License GPLv3+: GNU GPL version 3 or later \<http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY\, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions\, please see: \<http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: \<http://www.gnu.org/software/gdb/documentation/>. For help\, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./perl...done. (gdb) break sv.c:7069 Breakpoint 1 at 0x528811: file sv.c\, line 7069. (gdb) run -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")' Starting program: /src/package/lang/perl/gitperl/perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")' [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1\, Perl_sv_free2 (sv=sv@entry=0x9815b0\, rc=rc@entry=0) at sv.c:7069 7069 Perl_warner(aTHX_ packWARN(WARN_INTERNAL)\, (gdb) where #0 Perl_sv_free2 (sv=sv@entry=0x9815b0\, rc=rc@entry=0) at sv.c:7069 #1 0x0000000000527d9a in S_SvREFCNT_dec (sv=0x9815b0) at inline.h:166 #2 Perl_sv_free (sv=0x9815b0) at sv.c:7002 #3 Perl_sv_clear (orig_sv=orig_sv@entry=0x9536b0) at sv.c:6846 #4 0x00000000005286e1 in Perl_sv_free2 (sv=sv@entry=0x9536b0\, rc=\) at sv.c:7033 #5 0x000000000045c20e in S_SvREFCNT_dec (sv=0x9536b0) at inline.h:166 #6 Perl_gp_free (gv=gv@entry=0x953698) at gv.c:2547 #7 0x00000000005280ba in Perl_sv_clear (orig_sv=orig_sv@entry=0x953680) at sv.c:6684 #8 0x00000000005286e1 in Perl_sv_free2 (sv=sv@entry=0x953680\, rc=\) at sv.c:7033 #9 0x000000000045c20e in S_SvREFCNT_dec (sv=0x953680) at inline.h:166 #10 Perl_gp_free (gv=gv@entry=0x953668) at gv.c:2547 #11 0x00000000005280ba in Perl_sv_clear (orig_sv=orig_sv@entry=0x935fe0) at sv.c:6684 #12 0x00000000005286e1 in Perl_sv_free2 (sv=sv@entry=0x935fe0\, rc=\) at sv.c:7033 #13 0x0000000000456715 in S_SvREFCNT_dec (sv=0x935fe0) at inline.h:166 #14 perl_destruct (my_perl=\) at perl.c:1089 #15 0x0000000000420ab4 in main (argc=3\, argv=0x7fffffffe5c8\, env=0x7fffffffe5e8) at perlmain.c:127 (gdb) p *sv $1 = {sv_any = 0x9362b0\, sv_refcnt = 0\, sv_flags = 255\, sv_u = {svu_pv = 0x0\, svu_iv = 0\, svu_uv = 0\, svu_nv = 0\, svu_rv = 0x0\, svu_rx = 0x0\, svu_array = 0x0\, svu_hash = 0x0\, svu_gp = 0x0\, svu_fp = 0x0}} (gdb) p /x *(SV*)0x9536b0 $2 = {sv_any = 0x93bf00\, sv_refcnt = 0x0\, sv_flags = 0x3200000c\, sv_u = { svu_pv = 0x96dc90\, svu_iv = 0x96dc90\, svu_uv = 0x96dc90\, svu_nv = 0x0\, svu_rv = 0x96dc90\, svu_rx = 0x96dc90\, svu_array = 0x96dc90\, svu_hash = 0x96dc90\, svu_gp = 0x96dc90\, svu_fp = 0x96dc90}} (gdb) p *(XPVHV*)0x93bf00 $3 = {xmg_stash = 0x0\, xmg_u = {xmg_magic = 0x0\, xmg_hash_index = 0}\, xhv_keys = 29\, xhv_max = 63}

I'm not sure how to determine what's actually being freed here\, suggestions welcome.

p5pRT commented 9 years ago

From @jkeenan

On Sat May 09 04:53:53 2015\, hv wrote:

I'm able to reproduce this. Here are cut-down versions of the 4 cases:

% PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")' Attempt to free unreferenced scalar: SV 0x1d1b6a8 during global destruction. % PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_INOUT($fh); close $fh' Attempt to free unreferenced scalar: SV 0x2844760 during global destruction. % PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "\<"\, \$buf; XS::Typemap::T_IN($fh); close $fh' Attempt to free unreferenced scalar: SV 0x2748628 during global destruction. % PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_OUT($fh); close $fh' Attempt to free unreferenced scalar: SV 0xa1c7a8 during global destruction. %

% ./perl -Ilib -V Summary of my perl5 (revision 5 version 22 subversion 0) configuration: Commit id: 416c06fc852c00a948ade2fd39e7bf1b9f00ca9d Platform: osname=linux\, osvers=3.13.0-37-generic\, archname=x86_64-linux uname='linux shad2 3.13.0-37-generic #64-ubuntu smp mon sep 22 21:28:38 utc 2014 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dcc=gcc -Dprefix=/opt/blead-d -Doptimize=-g -O6 -DDEBUGGING -Dusedevel -Uversiononly' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='gcc'\, ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-g -O6'\, cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack- protector -I/usr/local/include' ccversion=''\, gccversion='4.8.2'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='gcc'\, ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.8/include- fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -g -O6 -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Locally applied patches: RC0 Built under linux Compiled at May 9 2015 09:35:23 %ENV: PERL_TEST_MEMORY="8" @INC: lib /opt/blead-d/lib/perl5/site_perl/5.22.0/x86_64-linux /opt/blead-d/lib/perl5/site_perl/5.22.0 /opt/blead-d/lib/perl5/5.22.0/x86_64-linux /opt/blead-d/lib/perl5/5.22.0 /opt/blead-d/lib/perl5/site_perl/5.21.9 /opt/blead-d/lib/perl5/site_perl . %

Not sure I know how to diagnose this\, but here's a stack trace for the complaint on the first example:

PERL_DESTRUCT_LEVEL=2 gdb ./perl GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7 Copyright (C) 2014 Free Software Foundation\, Inc. License GPLv3+: GNU GPL version 3 or later \<http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY\, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions\, please see: \<http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: \<http://www.gnu.org/software/gdb/documentation/>. For help\, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./perl...done. (gdb) break sv.c:7069 Breakpoint 1 at 0x528811: file sv.c\, line 7069. (gdb) run -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")' Starting program: /src/package/lang/perl/gitperl/perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")' [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux- gnu/libthread_db.so.1".

Breakpoint 1\, Perl_sv_free2 (sv=sv@entry=0x9815b0\, rc=rc@entry=0) at sv.c:7069 7069 Perl_warner(aTHX_ packWARN(WARN_INTERNAL)\, (gdb) where #0 Perl_sv_free2 (sv=sv@entry=0x9815b0\, rc=rc@entry=0) at sv.c:7069 #1 0x0000000000527d9a in S_SvREFCNT_dec (sv=0x9815b0) at inline.h:166 #2 Perl_sv_free (sv=0x9815b0) at sv.c:7002 #3 Perl_sv_clear (orig_sv=orig_sv@entry=0x9536b0) at sv.c:6846 #4 0x00000000005286e1 in Perl_sv_free2 (sv=sv@entry=0x9536b0\, rc=\) at sv.c:7033 #5 0x000000000045c20e in S_SvREFCNT_dec (sv=0x9536b0) at inline.h:166 #6 Perl_gp_free (gv=gv@entry=0x953698) at gv.c:2547 #7 0x00000000005280ba in Perl_sv_clear (orig_sv=orig_sv@entry=0x953680) at sv.c:6684 #8 0x00000000005286e1 in Perl_sv_free2 (sv=sv@entry=0x953680\, rc=\) at sv.c:7033 #9 0x000000000045c20e in S_SvREFCNT_dec (sv=0x953680) at inline.h:166 #10 Perl_gp_free (gv=gv@entry=0x953668) at gv.c:2547 #11 0x00000000005280ba in Perl_sv_clear (orig_sv=orig_sv@entry=0x935fe0) at sv.c:6684 #12 0x00000000005286e1 in Perl_sv_free2 (sv=sv@entry=0x935fe0\, rc=\) at sv.c:7033 #13 0x0000000000456715 in S_SvREFCNT_dec (sv=0x935fe0) at inline.h:166 #14 perl_destruct (my_perl=\) at perl.c:1089 #15 0x0000000000420ab4 in main (argc=3\, argv=0x7fffffffe5c8\, env=0x7fffffffe5e8) at perlmain.c:127 (gdb) p *sv $1 = {sv_any = 0x9362b0\, sv_refcnt = 0\, sv_flags = 255\, sv_u = {svu_pv = 0x0\, svu_iv = 0\, svu_uv = 0\, svu_nv = 0\, svu_rv = 0x0\, svu_rx = 0x0\, svu_array = 0x0\, svu_hash = 0x0\, svu_gp = 0x0\, svu_fp = 0x0}} (gdb) p /x *(SV*)0x9536b0 $2 = {sv_any = 0x93bf00\, sv_refcnt = 0x0\, sv_flags = 0x3200000c\, sv_u = { svu_pv = 0x96dc90\, svu_iv = 0x96dc90\, svu_uv = 0x96dc90\, svu_nv = 0x0\, svu_rv = 0x96dc90\, svu_rx = 0x96dc90\, svu_array = 0x96dc90\, svu_hash = 0x96dc90\, svu_gp = 0x96dc90\, svu_fp = 0x96dc90}} (gdb) p *(XPVHV*)0x93bf00 $3 = {xmg_stash = 0x0\, xmg_u = {xmg_magic = 0x0\, xmg_hash_index = 0}\, xhv_keys = 29\, xhv_max = 63}

I'm not sure how to determine what's actually being freed here\, suggestions welcome.

With the configuration attached\, I could not reproduce your findings:

##### [perl] 33 $ PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")' [perl] 34 $ PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_INOUT($fh); close $fh' [perl] 35 $ PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "\<"\, \$buf; XS::Typemap::T_IN($fh); close $fh' [perl] 36 $ PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_OUT($fh); close $fh' #####

No error output in any of the 4 cases. Am I doing something wrong? Is my configuration significantly different from yours?

Thank you very much.

-- James E Keenan (jkeenan@cpan.org)

p5pRT commented 9 years ago

From @jkeenan

Summary of my perl5 (revision 5 version 22 subversion 0) configuration: Commit id: 416c06fc852c00a948ade2fd39e7bf1b9f00ca9d Platform: osname=linux\, osvers=3.13.0-52-generic\, archname=x86_64-linux uname='linux zareason 3.13.0-52-generic #86-ubuntu smp mon may 4 04:32:59 utc 2015 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dusedevel -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-O2'\, cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.8.2'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='cc'\, ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64 libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Locally applied patches: RC0 Built under linux Compiled at May 9 2015 07:27:49 %ENV: PERL5LIB="/home/jkeenan/perl5/lib/perl5" PERLBREW_BASHRC_VERSION="0.67" PERLBREW_HOME="/home/jkeenan/.perlbrew" PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/man" PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin:/home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/bin" PERLBREW_PERL="perl-5.20.1" PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew" PERLBREW_VERSION="0.67" PERL_LOCAL_LIB_ROOT="/home/jkeenan/perl5" PERL_MB_OPT="--install_base "/home/jkeenan/perl5"" PERL_MM_OPT="INSTALL_BASE=/home/jkeenan/perl5" PERL_WORKDIR="gitwork/perl" @INC: lib /home/jkeenan/perl5/lib/perl5/x86_64-linux /home/jkeenan/perl5/lib/perl5 /home/jkeenan/testing/blead/lib/perl5/site_perl/5.22.0/x86_64-linux /home/jkeenan/testing/blead/lib/perl5/site_perl/5.22.0 /home/jkeenan/testing/blead/lib/perl5/5.22.0/x86_64-linux /home/jkeenan/testing/blead/lib/perl5/5.22.0 .

p5pRT commented 9 years ago

From perl@profvince.com

With the configuration attached\, I could not reproduce your findings:

##### [perl] 33 $ PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")' [perl] 34 $ PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_INOUT($fh); close $fh' [perl] 35 $ PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "\<"\, \$buf; XS::Typemap::T_IN($fh); close $fh' [perl] 36 $ PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_OUT($fh); close $fh' #####

No error output in any of the 4 cases. Am I doing something wrong? Is my configuration significantly different from yours?

Thank you very much.

You don't have DEBUGGING enabled\, for starters.

Vincent

p5pRT commented 9 years ago

From @jkeenan

On Sat May 09 07:01:15 2015\, perl@profvince.com wrote:

With the configuration attached\, I could not reproduce your findings:

No error output in any of the 4 cases. Am I doing something wrong? Is my configuration significantly different from yours?

Thank you very much.

You don't have DEBUGGING enabled\, for starters.

Yes\, confirmed.

##### $ !33;!34;!35;!36 PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } XS::Typemap::T_STDIO_open("stdio.tmp")';PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_INOUT($fh); close $fh';PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "\<"\, \$buf; XS::Typemap::T_IN($fh); close $fh';PERL_DESTRUCT_LEVEL=2 ./perl -e '@INC = "lib"; { package XS::Typemap; require XSLoader; XSLoader::load() } $buf = ""; open $fh\, "+\<"\, \$buf; XS::Typemap::T_OUT($fh); close $fh' Attempt to free unreferenced scalar: SV 0x1702fa8 during global destruction. Attempt to free unreferenced scalar: SV 0x1249a30 during global destruction. Attempt to free unreferenced scalar: SV 0x22e5270 during global destruction. Attempt to free unreferenced scalar: SV 0xea4148 during global destruction. ##### -- James E Keenan (jkeenan@cpan.org)

p5pRT commented 9 years ago

From @bulk88

Confirmed on Win32 with DEBUGGING.

C:\p521\srcpara>perl -Ilib -e " { package XS::Typemap; require XSLoader; XSLoade r::load() } XS::Typemap::T_STDIO_open(\"stdio.tmp\")" Attempt to free unreferenced scalar: SV 0x1ca24dc\, Perl interpreter: 0x1c44dec d uring global destruction.

C:\p521\srcpara>perl -V Summary of my perl5 (revision 5 version 21 subversion 12) configuration: Derived from: 5d370191488b4e919956a033fa450813df23ee36 Platform: osname=MSWin32\, osvers=5.2\, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended\, useposix=true\, d_sigaction=undef useithreads=define\, usemultiplicity=define use64bitint=undef\, use64bitall=undef\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cl'\, ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DDEBUGGING -GL -DWIN32 -D_CO NSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL _TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS'\, optimize='-O1 -MD -Zi -DDEBUGGING -GL'\, cppflags='-DWIN32' ccversion='15.00.30729.01'\, gccversion=''\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234\, doublekind=3

d_longlong=undef\, longlongsize=8\, d_longdbl=define\, longdblsize=8\, longdblki nd=0 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='__int64'\, lseeksi ze=8 alignbytes=8\, prototype=define Linker and Libraries: ld='link'\, ldflags ='-nologo -nodefaultlib -debug -libpath:"c:\per l\lib\CORE" -machine:x86 "/manifestdependency:type='Win32' name='Mic rosoft.Windows.Common-Controls' version='6.0.0.0' processorArchitecture='*' publ icKeyToken='6595b64144ccf1df' language='*'"' libpth=\lib libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.li b advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.l ib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib

perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg3 2.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_ 32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt .lib libc=msvcrt.lib\, so=dll\, useshrplib=true\, libperl=perl521.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs\, dlext=dll\, d_dlsymun=undef\, ccdlflags=' ' cccdlflags=' '\, lddlflags='-dll -nologo -nodefaultlib -debug -libpath:"c:\perl\lib\CORE" -machine:x86 "/manifestdependency:type=' Win32' name='Microsoft.Windows.Common-Controls' version='6.0.0.0' processorArchi tecture='*' publicKeyToken='6595b64144ccf1df' language='*'"'

Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES HAVE_INTERP_INTERN MULTIPLICITY PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_TRACK_MEMPOOL USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Locally applied patches: uncommitted-changes Built under MSWin32 Compiled at May 9 2015 12:47:14 %ENV: PERL_DESTRUCT_LEVEL="2" @INC: C:/p521/srcpara/lib .

C:\p521\srcpara>

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

From @bulk88

I am preparing a patch for this ticket.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

From @bulk88

On Sat May 09 15:55:43 2015\, bulk88 wrote:

I am preparing a patch for this ticket.

stack traces of the SV * that caused "Attempt to free unreferenced scalar"

created at

+ cop_file 0x008f6f34 "-e" char * cop_line 1 unsigned long

perl521.dll!S_new_SV(interpreter * my_perl=0x003853cc\, const char * file=0x282bd4ac\, int line=5640\, const char * func=0x282beb44) Line 348 C perl521.dll!Perl_newSV(interpreter * my_perl=0x003853cc\, const unsigned int len=0) Line 5640 + 0x18 C perl521.dll!Perl_hv_common(interpreter * my_perl=0x003853cc\, hv * hv=0x0090d9e4\, sv * keysv=0x00000000\, const char * key=0x0098b4b9\, unsigned int klen=6\, int flags=0\, int action=48\, sv * val=0x00000000\, unsigned long hash=70165563) Line 775 + 0x22 C perl521.dll!Perl_hv_common_key_len(interpreter * my_perl=0x003853cc\, hv * hv=0x0090d9e4\, const char * key=0x0098b4b9\, long klen_i32=6\, const int action=48\, sv * val=0x00000000\, const unsigned long hash=0) Line 333 + 0x27 C perl521.dll!Perl_gv_fetchpvn_flags(interpreter * my_perl=0x003853cc\, const char * nambeg=0x0098b4ac\, unsigned int full_len=19\, long flags=1\, svtype sv_type=SVt_PVGV) Line 2265 + 0x3d C perl521.dll!Perl_gv_fetchpv(interpreter * my_perl=0x003853cc\, const char * nambeg=0x0098b4ac\, long add=1\, svtype sv_type=SVt_PVGV) Line 1513 + 0x22 C perl521.dll!Perl_newGVgen_flags(interpreter * my_perl=0x003853cc\, const char * pack=0x1000ca30\, unsigned long flags=0) Line 2466 + 0x5f pack is 0x1000ca30 "XS::Typemap" Typemap.dll!XS_XS__Typemap_T_STDIO_open(interpreter * my_perl=0x003853cc\, cv * cv=0x00922bcc) Line 1698 + 0x11 C perl521.dll!Perl_pp_entersub(interpreter * my_perl=0x003853cc) Line 3270 + 0x10 C perl521.dll!Perl_runops_debug(interpreter * my_perl=0x003853cc) Line 2234 + 0xd C perl521.dll!S_run_body(interpreter * my_perl=0x003853cc\, long oldscope=1) Line 2448 + 0xd C perl521.dll!perl_run(interpreter * my_perl=0x003853cc) Line 2374 C perl521.dll!RunPerl(int argc=4\, char * * argv=0x00384eb8\, char * * env=0x00382830) Line 258 + 0x9 C++ perl.exe!main(int argc=4\, char * * argv=0x00384eb8\, char * * env=0x00383298) Line 39 + 0x12 C perl.exe!mainCRTStartup() Line 398 + 0xe C kernel32.dll!_BaseProcessStart@4() + 0x23

first dec event\, but the SV on temp stack isn;t the one that gets double freed\, the doublefreed one "is inside" the tmps stack one\, I can't dump the SVs since p5p rejected this patch https://rt-archive.perl.org/perl5/Ticket/Display.html?id=121932 and I am on a virgin blead without my normal 30 patches applied\, curcop is "-e" line 0

perl521.dll!Perl_sv_clear(interpreter * my_perl=0x003853cc\, sv * const orig_sv=0x0093edfc) Line 6849 C perl521.dll!Perl_sv_free2(interpreter * my_perl=0x003853cc\, sv * const sv=0x0093edfc\, const unsigned long rc=0x00000001) Line 7033 + 0xd C perl521.dll!S_SvREFCNT_dec_NN(interpreter * my_perl=0x003853cc\, sv * sv=0x0093edfc) Line 177 + 0x11 C perl521.dll!Perl_free_tmps(interpreter * my_perl=0x003853cc) Line 178 + 0xd C perl521.dll!perl_run(interpreter * my_perl=0x003853cc) Line 2376 + 0x17 C perl521.dll!RunPerl(int argc=0x00000004\, char * * argv=0x00384eb8\, char * * env=0x00382830) Line 258 + 0x9 C++ perl.exe!main(int argc=0x00000004\, char * * argv=0x00384eb8\, char * * env=0x00383298) Line 39 + 0x12 C perl.exe!mainCRTStartup() Line 398 + 0xe C kernel32.dll!_BaseProcessStart@4() + 0x23

/* unrolled SvREFCNT_dec and sv_free2 follows: */

if (!sv) continue; if (!SvREFCNT(sv)) { sv_free(sv); continue; } if (--(SvREFCNT(sv)))\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<\<line 6849 continue; #ifdef DEBUGGING if (SvTEMP(sv)) { Perl_ck_warner_d(aTHX_ packWARN(WARN_DEBUGGING)\, "Attempt to free temp prematurely: SV 0x%"UVxf pTHX__FORMAT\, PTR2UV(sv) pTHX__VALUE); continue; } #endif

2nd dec event\, now with "Attempt to free unreferenced scalar"

perl521.dll!Perl_sv_free2(interpreter * my_perl=0x003853cc\, sv * const sv=0x0094ad24\, const unsigned long rc=0x00000000) Line 7071 C perl521.dll!S_SvREFCNT_dec(interpreter * my_perl=0x003853cc\, sv * sv=0x0094ad24) Line 166 + 0x11 C perl521.dll!Perl_sv_free(interpreter * my_perl=0x003853cc\, sv * const sv=0x0094ad24) Line 7002 + 0xd C perl521.dll!Perl_sv_clear(interpreter * my_perl=0x003853cc\, sv * const orig_sv=0x0090d9e4) Line 6846 + 0xd C perl521.dll!Perl_sv_free2(interpreter * my_perl=0x003853cc\, sv * const sv=0x0090d9e4\, const unsigned long rc=0x00000001) Line 7033 + 0xd C perl521.dll!S_SvREFCNT_dec(interpreter * my_perl=0x003853cc\, sv * sv=0x0090d9e4) Line 166 + 0x11 C perl521.dll!Perl_gp_free(interpreter * my_perl=0x003853cc\, gv * gv=0x0090d9c4) Line 2547 + 0xd C perl521.dll!Perl_sv_clear(interpreter * my_perl=0x003853cc\, sv * const orig_sv=0x0090d9a4) Line 6684 + 0xd C perl521.dll!Perl_sv_free2(interpreter * my_perl=0x003853cc\, sv * const sv=0x0090d9a4\, const unsigned long rc=0x00000001) Line 7033 + 0xd C perl521.dll!S_SvREFCNT_dec(interpreter * my_perl=0x003853cc\, sv * sv=0x0090d9a4) Line 166 + 0x11 C perl521.dll!Perl_gp_free(interpreter * my_perl=0x003853cc\, gv * gv=0x0090d984) Line 2547 + 0xd C perl521.dll!Perl_sv_clear(interpreter * my_perl=0x003853cc\, sv * const orig_sv=0x003890ec) Line 6684 + 0xd C perl521.dll!Perl_sv_free2(interpreter * my_perl=0x003853cc\, sv * const sv=0x003890ec\, const unsigned long rc=0x00000001) Line 7033 + 0xd C perl521.dll!S_SvREFCNT_dec(interpreter * my_perl=0x003853cc\, sv * sv=0x003890ec) Line 166 + 0x11 C perl521.dll!perl_destruct(interpreter * my_perl=0x003853cc) Line 1089 + 0xd C perl521.dll!RunPerl(int argc=0x00000004\, char * * argv=0x00384eb8\, char * * env=0x00382830) Line 262 + 0x9 C++ perl.exe!main(int argc=0x00000004\, char * * argv=0x00384eb8\, char * * env=0x00383298) Line 39 + 0x12 C perl.exe!mainCRTStartup() Line 398 + 0xe C kernel32.dll!_BaseProcessStart@4() + 0x23

PERL_DESTRUCT_LEVEL = 2 didn't cause a crash for me on non-DEBUGGING for me\, just "Attempt to free unreferenced" warnings\, but there seems to be no way to capture or make fatal that to harness\, those warnings that come from perl_destruct\, but I did figure out another way to trigger the crash. I split it into 2 patches to be sure that Typemap.t fails reliably\, perhaps some other people should smoke it with just the first patch to make sure it fails (and it randomly SEGVed for me on non-DEBUGGING) on platforms other than Win32. Maybe the 2 patches should be squashed into one for bisectablity or something.

harness doesn't listen to STDERR\, and a warning doesn't normally trigger a non-zero exit\, and use warnings fatal doesn't seem to work inside perl_destruct (is that a bug?). Maybe those "panic" style internal warnings should be croaks/fatal\, not warnings. Emitting a warning about guaranteed "memory corruption" and continuing execution even though the Perl VM is corrupt is a bit silly to me. A year or 2 ago khw did something that was causing unreferenced warnings on the George Win32 smoker but still passing for months until I spotted it one day while happening to watch a "make test" on my machine. The issue is nobody reviews the 1 MB smoke logs by eye and harness doesn't care about STDERR.

Not only were the typemaps causing memory corruption\, there were some other bugs and shoddy code with the typemap entries I fixed such as the typemap entries only working for RETVAL and not an outgoing @_ arg\, using raw decimal numbers as flags instead of named constants flags\, and duplicate hash lookups.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

From @bulk88

0001-add-test-that-fails-for-124181-to-Typemap.t.patch

```diff From 594d4089be5c8783f6fa1f53d315e4a701145d0a Mon Sep 17 00:00:00 2001 From: Daniel Dragan Date: Sun, 10 May 2015 11:36:05 -0400 Subject: [PATCH 1/2] add test that fails for #124181 to Typemap.t These tests will either fail with harness, and randomly SEGV for me, which is intentional since they are testing memory corruption. --- ext/XS-Typemap/Typemap.pm | 4 ++-- ext/XS-Typemap/Typemap.xs | 9 +++++++++ ext/XS-Typemap/t/Typemap.t | 20 ++++++++++++++++++-- 3 files changed, 29 insertions(+), 4 deletions(-) diff --git a/ext/XS-Typemap/Typemap.pm b/ext/XS-Typemap/Typemap.pm index de3319b..a1ae021 100644 --- a/ext/XS-Typemap/Typemap.pm +++ b/ext/XS-Typemap/Typemap.pm @@ -36,7 +36,7 @@ require XSLoader; use vars qw/ $VERSION @EXPORT /; -$VERSION = '0.13'; +$VERSION = '0.14'; @EXPORT = (qw/ T_SV @@ -76,7 +76,7 @@ $VERSION = '0.13'; T_OPAQUEPTR_IN T_OPAQUEPTR_OUT T_OPAQUEPTR_OUT_short T_OPAQUEPTR_IN_struct T_OPAQUEPTR_OUT_struct T_ARRAY - T_STDIO_open T_STDIO_close T_STDIO_print + T_STDIO_open T_STDIO_open_ret_in_arg T_STDIO_close T_STDIO_print T_PACKED_in T_PACKED_out T_PACKEDARRAY_in T_PACKEDARRAY_out T_INOUT T_IN T_OUT diff --git a/ext/XS-Typemap/Typemap.xs b/ext/XS-Typemap/Typemap.xs index 3fa0e74ab..8314cc2 100644 --- a/ext/XS-Typemap/Typemap.xs +++ b/ext/XS-Typemap/Typemap.xs @@ -906,6 +906,15 @@ T_STDIO_open( file ) OUTPUT: RETVAL +void +T_STDIO_open_ret_in_arg( file, io) + const char * file + FILE * io = NO_INIT + CODE: + io = xsfopen( file ); + OUTPUT: + io + SysRet T_STDIO_close( f ) PerlIO * f diff --git a/ext/XS-Typemap/t/Typemap.t b/ext/XS-Typemap/t/Typemap.t index 27b4086..49ac479 100644 --- a/ext/XS-Typemap/t/Typemap.t +++ b/ext/XS-Typemap/t/Typemap.t @@ -6,10 +6,11 @@ BEGIN { } } -use Test::More tests => 152; +use Test::More tests => 156; use strict; -use warnings; +#catch WARN_INTERNAL type errors, and anything else unexpected +use warnings FATAL => 'all'; use XS::Typemap; pass(); @@ -213,6 +214,7 @@ is( T_PV("a string"), "a string"); is( T_PV(52), 52); ok !defined T_PV_null, 'RETVAL = NULL returns undef for char*'; { + use warnings NONFATAL => 'all'; my $uninit; local $SIG{__WARN__} = sub { ++$uninit if shift =~ /uninit/ }; () = ''.T_PV_null; @@ -393,6 +395,16 @@ if (defined $fh) { } } +$fh = "FOO"; +T_STDIO_open_ret_in_arg( $testfile, $fh); +ok( $fh ne "FOO", 'return io in arg open succeeds'); +ok( print($fh "first line\n"), 'can print to return io in arg'); +ok( close($fh), 'can close return io in arg'); +$fh = "FOO"; +#now with a bad file name to make sure $fh is written to on failure +T_STDIO_open_ret_in_arg( "", $fh); +ok( !defined$fh, 'return io in arg open failed successfully'); + # T_INOUT note("T_INOUT"); SCOPE: { @@ -439,6 +451,10 @@ SCOPE: { ok(!close $fh2); } +# Perl RT #124181 SEGV due to double free in typemap +# "Attempt to free unreferenced scalar" +%{*{main::XS::}{HASH}} = (); + sub is_approx { my ($l, $r, $n) = @_; if (not defined $l or not defined $r) { -- 1.7.9.msysgit.0 ```

p5pRT commented 9 years ago

From @bulk88

0002-fix-124181-double-free-refcnt-problems-in-IO-types-i.patch

```diff From a5ccb571fc79c106e20af7e9bf690ab98748c7de Mon Sep 17 00:00:00 2001 From: Daniel Dragan Date: Sat, 9 May 2015 23:46:11 -0400 Subject: [PATCH 2/2] fix #124181 double free/refcnt problems in IO types in typemap commit 50e5165b96 "stop T_IN/OUT/INOUT/STDIO typemaps leaking" changed newRV to newRV_noinc, but the GV * returned by newGVgen() is owned by the package tree, like the SV * returned by get_sv(). Now when the RV to GV is freed on mortal stack, the GV * in the package tree is freed, and now there is a freed GV * in the package tree, if you turn on "PERL_DESTRUCT_LEVEL=2" (and perhaps DEBUGGING is needed too), the package tree is destroyed SV * by SV *, and perl will eventually warn with "Attempt to free unreferenced scalar" which a very bad panic type warning. Also fix the problem, that if this OUTPUT: type is being used for an incoming arg, not the outgoing RETVAL arg, you can't assign a new SV* ontop of the old one, that only works for perl stack return args, so replace "$arg = &PL_sv_undef;" with "sv_setsv($arg, &PL_sv_undef);" if its not RETVAL, this way OUTPUT on incoming args also works if it goes down the error path. For efficiency, in a RETVAL siutation, let the undef original SV* in $arg which is typically obtained from sv_newmortal() by xsubpp pass through if we error out. Also for efficiency, if it is RETVAL (which is more common) dont do the sv_setsv/SvREFCNT_dec_NN stuff (2 function calls), just mortalize (1 function call) the ex-temp RV and arrange for the RV to wind up on perl stack. Also, the GV * already knows what HV * stash it belongs to, so avoid the stash lookup done by gv_stashpv() and just use GvSTASH which are simple pointer derefs. --- lib/ExtUtils/typemap | 48 ++++++++++++++++++++++++------------------------ 1 files changed, 24 insertions(+), 24 deletions(-) diff --git a/lib/ExtUtils/typemap b/lib/ExtUtils/typemap index 831baad..5f61527 100644 --- a/lib/ExtUtils/typemap +++ b/lib/ExtUtils/typemap @@ -389,47 +389,47 @@ T_STDIO GV *gv = newGVgen("$Package"); PerlIO *fp = PerlIO_importFILE($var,0); if ( fp && do_open(gv, "+<&", 3, FALSE, 0, 0, fp) ) { - SV *rv = newRV_noinc((SV*)gv); - rv = sv_bless(rv, gv_stashpv("$Package",1)); - sv_setsv($arg, rv); - SvREFCNT_dec_NN(rv); - } + SV *rv = newRV_inc((SV*)gv); + rv = sv_bless(rv, GvSTASH(gv)); + ${"$var" eq "RETVAL" ? \"$arg = sv_2mortal(rv);" + : \"sv_setsv($arg, rv);\n\t\tSvREFCNT_dec_NN(rv);"} + }${"$var" ne "RETVAL" ? \" else - $arg = &PL_sv_undef; + sv_setsv($arg, &PL_sv_undef);\n" : \""} } T_IN { GV *gv = newGVgen("$Package"); if ( do_open(gv, "<&", 2, FALSE, 0, 0, $var) ) { - SV *rv = newRV_noinc((SV*)gv); - rv = sv_bless(rv, gv_stashpv("$Package",1)); - sv_setsv($arg, rv); - SvREFCNT_dec_NN(rv); - } + SV *rv = newRV_inc((SV*)gv); + rv = sv_bless(rv, GvSTASH(gv)); + ${"$var" eq "RETVAL" ? \"$arg = sv_2mortal(rv);" + : \"sv_setsv($arg, rv);\n\t\tSvREFCNT_dec_NN(rv);"} + }${"$var" ne "RETVAL" ? \" else - $arg = &PL_sv_undef; + sv_setsv($arg, &PL_sv_undef);\n" : \""} } T_INOUT { GV *gv = newGVgen("$Package"); if ( do_open(gv, "+<&", 3, FALSE, 0, 0, $var) ) { - SV *rv = newRV_noinc((SV*)gv); - rv = sv_bless(rv, gv_stashpv("$Package",1)); - sv_setsv($arg, rv); - SvREFCNT_dec_NN(rv); - } + SV *rv = newRV_inc((SV*)gv); + rv = sv_bless(rv, GvSTASH(gv)); + ${"$var" eq "RETVAL" ? \"$arg = sv_2mortal(rv);" + : \"sv_setsv($arg, rv);\n\t\tSvREFCNT_dec_NN(rv);"} + }${"$var" ne "RETVAL" ? \" else - $arg = &PL_sv_undef; + sv_setsv($arg, &PL_sv_undef);\n" : \""} } T_OUT { GV *gv = newGVgen("$Package"); if ( do_open(gv, "+>&", 3, FALSE, 0, 0, $var) ) { - SV *rv = newRV_noinc((SV*)gv); - rv = sv_bless(rv, gv_stashpv("$Package",1)); - sv_setsv($arg, rv); - SvREFCNT_dec_NN(rv); - } + SV *rv = newRV_inc((SV*)gv); + rv = sv_bless(rv, GvSTASH(gv)); + ${"$var" eq "RETVAL" ? \"$arg = sv_2mortal(rv);" + : \"sv_setsv($arg, rv);\n\t\tSvREFCNT_dec_NN(rv);"} + }${"$var" ne "RETVAL" ? \" else - $arg = &PL_sv_undef; + sv_setsv($arg, &PL_sv_undef);\n" : \""} } -- 1.7.9.msysgit.0 ```

p5pRT commented 9 years ago

From @rjbs

On Sat May 09 15:55:43 2015\, bulk88 wrote:

I am preparing a patch for this ticket.

Could we get a review here? I'd like to know how we feel about getting this into RC1\, which I'd like to get out the door in the next few days.

-- rjbs

p5pRT commented 9 years ago

From @iabyn

On Fri\, May 15\, 2015 at 02:49:33PM -0700\, Ricardo SIGNES via RT wrote:

On Sat May 09 15:55:43 2015\, bulk88 wrote:

I am preparing a patch for this ticket.

Could we get a review here? I'd like to know how we feel about getting this into RC1\, which I'd like to get out the door in the next few days.

I've just looked at it. I don't like it at all\, from the perspective that\, this close to RC1\, the one commit is doing several things: fixing the premature free that is the issue of this ticket\, but also fixing/changing/optimising a whole bunch of other stuff too\, which I certainly don't understand without looking at it a whole lot more closely.

In more detail:

Since perl 5.000 for T_IN\, T_OUT\, T_INOUT\, and since 5.8.0 for T_STDIO\, these typemaps have leaked an RV and GV per invocation.

My commit 50e5165b9 of Dec 2014 fixed those leaks\, but was over-enthusiastic\, in that it "fixed" the GV leak twice\, so now the GV may be prematurely freed.

Daniel's suggested change fixes that (changing the newRV_noinc back to newRV_inc)\, but does a bunch of extra stuff related to those typemaps.

I recommend that at this very late stage\, we revert my 50e5165b9 commit and release 5.22 with known leaks in those typemaps (leaks that nobody noticed for around 20 years)\, then readdress this issue post 5.22.

I've submitted the revert for smoking: smoke-me/davem/rt124181.

Failing that\, we should do just the minimal fix of s/newRV_noinc/newRV_inc/\, and readdress the remaining issues post-5.22.

-- Spock (or Data) is fired from his high-ranking position for not being able to understand the most basic nuances of about one in three sentences that anyone says to him. -- Things That Never Happen in "Star Trek" #19

p5pRT commented 9 years ago

From @khwilliamson

On 05/17/2015 01:00 PM\, Dave Mitchell wrote:

recommend that at this very late stage\, we revert my 50e5165b9 commit and release 5.22 with known leaks in those typemaps (leaks that nobody noticed for around 20 years)\, then readdress this issue post 5.22.

Karl Williamson pushed to smoke-me/khw-revert50e5165b9 (v5.21.11-144-g8c93413): created

p5pRT commented 9 years ago

From @iabyn

On Mon\, May 18\, 2015 at 08:42:57AM -0600\, Karl Williamson wrote:

On 05/17/2015 01:00 PM\, Dave Mitchell wrote:

recommend that at this very late stage\, we revert my 50e5165b9 commit and release 5.22 with known leaks in those typemaps (leaks that nobody noticed for around 20 years)\, then readdress this issue post 5.22.

Karl Williamson pushed to smoke-me/khw-revert50e5165b9 (v5.21.11-144-g8c93413): created

Later on in my email I said:

I've submitted the revert for smoking: smoke-me/davem/rt124181.

I'm assuming your smoke is an accidental duplicate of that???

-- Wesley Crusher gets beaten up by his classmates for being a smarmy git\, and consequently has a go at making some friends of his own age for a change. -- Things That Never Happen in "Star Trek" #18

p5pRT commented 9 years ago

From @khwilliamson

On 05/18/2015 08:49 AM\, Dave Mitchell wrote:

On Mon\, May 18\, 2015 at 08:42:57AM -0600\, Karl Williamson wrote:

On 05/17/2015 01:00 PM\, Dave Mitchell wrote:

recommend that at this very late stage\, we revert my 50e5165b9 commit and release 5.22 with known leaks in those typemaps (leaks that nobody noticed for around 20 years)\, then readdress this issue post 5.22.

Karl Williamson pushed to smoke-me/khw-revert50e5165b9 (v5.21.11-144-g8c93413): created

Later on in my email I said:
 I've submitted the revert for smoking&#8203;: smoke\-me/davem/rt124181\.
I'm assuming your smoke is an accidental duplicate of that???

Yes. Sorry.

p5pRT commented 9 years ago

From @iabyn

On Mon\, May 18\, 2015 at 09:34:41AM -0600\, Karl Williamson wrote:

Yes. Sorry.

No worries :-)

My branch has smoked ok. I'll merge it sometime later this evening (GMT+1) unless there's any further feedback.

-- Never do today what you can put off till tomorrow.

p5pRT commented 9 years ago

From @iabyn

On Mon\, May 18\, 2015 at 06:20:51PM +0100\, Dave Mitchell wrote:

On Mon\, May 18\, 2015 at 09:34:41AM -0600\, Karl Williamson wrote:

Yes. Sorry.

No worries :-)

My branch has smoked ok. I'll merge it sometime later this evening (GMT+1) unless there's any further feedback.

Now reverted with bae466e878e8602a507f96eab3bac37f2066b4fb.

-- That he said that that that that is is is debatable\, is debatable.

p5pRT commented 9 years ago

From @bulk88

Updated patches for blead/5.23.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

From @bulk88

0001-add-test-that-fails-for-124181-to-Typemap.t.patch

```diff From 1c8b0e23ba3b68cc69354e585bdfb7ec7518b6fe Mon Sep 17 00:00:00 2001 From: Daniel Dragan Date: Sun, 10 May 2015 11:36:05 -0400 Subject: [PATCH 1/2] add test that fails for #124181 to Typemap.t These tests will either fail with harness, and randomly SEGV for me, which is intentional since they are testing memory corruption. --- ext/XS-Typemap/Typemap.pm | 4 ++-- ext/XS-Typemap/Typemap.xs | 9 +++++++++ ext/XS-Typemap/t/Typemap.t | 20 ++++++++++++++++++-- 3 files changed, 29 insertions(+), 4 deletions(-) diff --git a/ext/XS-Typemap/Typemap.pm b/ext/XS-Typemap/Typemap.pm index de3319b..a1ae021 100644 --- a/ext/XS-Typemap/Typemap.pm +++ b/ext/XS-Typemap/Typemap.pm @@ -36,7 +36,7 @@ require XSLoader; use vars qw/ $VERSION @EXPORT /; -$VERSION = '0.13'; +$VERSION = '0.14'; @EXPORT = (qw/ T_SV @@ -76,7 +76,7 @@ $VERSION = '0.13'; T_OPAQUEPTR_IN T_OPAQUEPTR_OUT T_OPAQUEPTR_OUT_short T_OPAQUEPTR_IN_struct T_OPAQUEPTR_OUT_struct T_ARRAY - T_STDIO_open T_STDIO_close T_STDIO_print + T_STDIO_open T_STDIO_open_ret_in_arg T_STDIO_close T_STDIO_print T_PACKED_in T_PACKED_out T_PACKEDARRAY_in T_PACKEDARRAY_out T_INOUT T_IN T_OUT diff --git a/ext/XS-Typemap/Typemap.xs b/ext/XS-Typemap/Typemap.xs index 3fa0e74ab..8314cc2 100644 --- a/ext/XS-Typemap/Typemap.xs +++ b/ext/XS-Typemap/Typemap.xs @@ -906,6 +906,15 @@ T_STDIO_open( file ) OUTPUT: RETVAL +void +T_STDIO_open_ret_in_arg( file, io) + const char * file + FILE * io = NO_INIT + CODE: + io = xsfopen( file ); + OUTPUT: + io + SysRet T_STDIO_close( f ) PerlIO * f diff --git a/ext/XS-Typemap/t/Typemap.t b/ext/XS-Typemap/t/Typemap.t index 27b4086..49ac479 100644 --- a/ext/XS-Typemap/t/Typemap.t +++ b/ext/XS-Typemap/t/Typemap.t @@ -6,10 +6,11 @@ BEGIN { } } -use Test::More tests => 152; +use Test::More tests => 156; use strict; -use warnings; +#catch WARN_INTERNAL type errors, and anything else unexpected +use warnings FATAL => 'all'; use XS::Typemap; pass(); @@ -213,6 +214,7 @@ is( T_PV("a string"), "a string"); is( T_PV(52), 52); ok !defined T_PV_null, 'RETVAL = NULL returns undef for char*'; { + use warnings NONFATAL => 'all'; my $uninit; local $SIG{__WARN__} = sub { ++$uninit if shift =~ /uninit/ }; () = ''.T_PV_null; @@ -393,6 +395,16 @@ if (defined $fh) { } } +$fh = "FOO"; +T_STDIO_open_ret_in_arg( $testfile, $fh); +ok( $fh ne "FOO", 'return io in arg open succeeds'); +ok( print($fh "first line\n"), 'can print to return io in arg'); +ok( close($fh), 'can close return io in arg'); +$fh = "FOO"; +#now with a bad file name to make sure $fh is written to on failure +T_STDIO_open_ret_in_arg( "", $fh); +ok( !defined$fh, 'return io in arg open failed successfully'); + # T_INOUT note("T_INOUT"); SCOPE: { @@ -439,6 +451,10 @@ SCOPE: { ok(!close $fh2); } +# Perl RT #124181 SEGV due to double free in typemap +# "Attempt to free unreferenced scalar" +%{*{main::XS::}{HASH}} = (); + sub is_approx { my ($l, $r, $n) = @_; if (not defined $l or not defined $r) { -- 1.7.9.msysgit.0 ```

p5pRT commented 9 years ago

From @bulk88

0002-fix-124181-double-free-refcnt-problems-in-IO-types-i.patch

```diff From 7afd92de70add9ddcab27b094be848c432e3ec32 Mon Sep 17 00:00:00 2001 From: Daniel Dragan Date: Wed, 24 Jun 2015 15:48:12 -0400 Subject: [PATCH 2/2] fix #124181 double free/refcnt problems in IO types in typemap commit 50e5165b96 "stop T_IN/OUT/INOUT/STDIO typemaps leaking" changed newRV to newRV_noinc, but the GV * returned by newGVgen() is owned by the package tree, like the SV * returned by get_sv(). Now when the RV to GV is freed on mortal stack, the GV * in the package tree is freed, and now there is a freed GV * in the package tree, if you turn on "PERL_DESTRUCT_LEVEL=2" (and perhaps DEBUGGING is needed too), the package tree is destroyed SV * by SV *, and perl will eventually warn with "Attempt to free unreferenced scalar" which a very bad panic type warning. commit 50e5165b96 was reverted in commit bae466e878 "Revert "stop T_IN/OUT/INOUT/STDIO typemaps leaking" for 5.22's release to stop the panic, but reintroduced the SV/RV leak. So fix the RV leak (the val passed as source arg of sv_setsv) by freeing it after the copying. In a very unlikely scenario, the RV could still leak if sv_setsv dies. Also fix the problem, that if this OUTPUT: type is being used for an incoming arg, not the outgoing RETVAL arg, you can't assign a new SV* ontop of the old one, that only works for perl stack return args, so replace "$arg = &PL_sv_undef;" with "sv_setsv($arg, &PL_sv_undef);" if its not RETVAL, this way OUTPUT on incoming args also works if it goes down the error path. For efficiency, in a RETVAL siutation, let the undef original SV* in $arg which is typically obtained from sv_newmortal() by xsubpp pass through if we error out. Also for efficiency, if it is RETVAL (which is more common) dont do the sv_setsv/SvREFCNT_dec_NN stuff (2 function calls), just mortalize (1 function call) the ex-temp RV and arrange for the RV to wind up on perl stack. Also, the GV * already knows what HV * stash it belongs to, so avoid the stash lookup done by gv_stashpv() and just use GvSTASH which are simple pointer derefs. --- lib/ExtUtils/typemap | 40 ++++++++++++++++++++++++++++------------ pod/perldelta.pod | 7 +++++++ 2 files changed, 35 insertions(+), 12 deletions(-) diff --git a/lib/ExtUtils/typemap b/lib/ExtUtils/typemap index 0b09641..5f61527 100644 --- a/lib/ExtUtils/typemap +++ b/lib/ExtUtils/typemap @@ -388,32 +388,48 @@ T_STDIO { GV *gv = newGVgen("$Package"); PerlIO *fp = PerlIO_importFILE($var,0); - if ( fp && do_open(gv, "+<&", 3, FALSE, 0, 0, fp) ) - sv_setsv($arg, sv_bless(newRV((SV*)gv), gv_stashpv("$Package",1))); + if ( fp && do_open(gv, "+<&", 3, FALSE, 0, 0, fp) ) { + SV *rv = newRV_inc((SV*)gv); + rv = sv_bless(rv, GvSTASH(gv)); + ${"$var" eq "RETVAL" ? \"$arg = sv_2mortal(rv);" + : \"sv_setsv($arg, rv);\n\t\tSvREFCNT_dec_NN(rv);"} + }${"$var" ne "RETVAL" ? \" else - $arg = &PL_sv_undef; + sv_setsv($arg, &PL_sv_undef);\n" : \""} } T_IN { GV *gv = newGVgen("$Package"); - if ( do_open(gv, "<&", 2, FALSE, 0, 0, $var) ) - sv_setsv($arg, sv_bless(newRV((SV*)gv), gv_stashpv("$Package",1))); + if ( do_open(gv, "<&", 2, FALSE, 0, 0, $var) ) { + SV *rv = newRV_inc((SV*)gv); + rv = sv_bless(rv, GvSTASH(gv)); + ${"$var" eq "RETVAL" ? \"$arg = sv_2mortal(rv);" + : \"sv_setsv($arg, rv);\n\t\tSvREFCNT_dec_NN(rv);"} + }${"$var" ne "RETVAL" ? \" else - $arg = &PL_sv_undef; + sv_setsv($arg, &PL_sv_undef);\n" : \""} } T_INOUT { GV *gv = newGVgen("$Package"); - if ( do_open(gv, "+<&", 3, FALSE, 0, 0, $var) ) - sv_setsv($arg, sv_bless(newRV((SV*)gv), gv_stashpv("$Package",1))); + if ( do_open(gv, "+<&", 3, FALSE, 0, 0, $var) ) { + SV *rv = newRV_inc((SV*)gv); + rv = sv_bless(rv, GvSTASH(gv)); + ${"$var" eq "RETVAL" ? \"$arg = sv_2mortal(rv);" + : \"sv_setsv($arg, rv);\n\t\tSvREFCNT_dec_NN(rv);"} + }${"$var" ne "RETVAL" ? \" else - $arg = &PL_sv_undef; + sv_setsv($arg, &PL_sv_undef);\n" : \""} } T_OUT { GV *gv = newGVgen("$Package"); - if ( do_open(gv, "+>&", 3, FALSE, 0, 0, $var) ) - sv_setsv($arg, sv_bless(newRV((SV*)gv), gv_stashpv("$Package",1))); + if ( do_open(gv, "+>&", 3, FALSE, 0, 0, $var) ) { + SV *rv = newRV_inc((SV*)gv); + rv = sv_bless(rv, GvSTASH(gv)); + ${"$var" eq "RETVAL" ? \"$arg = sv_2mortal(rv);" + : \"sv_setsv($arg, rv);\n\t\tSvREFCNT_dec_NN(rv);"} + }${"$var" ne "RETVAL" ? \" else - $arg = &PL_sv_undef; + sv_setsv($arg, &PL_sv_undef);\n" : \""} } diff --git a/pod/perldelta.pod b/pod/perldelta.pod index 58ece4a..2a94b60 100644 --- a/pod/perldelta.pod +++ b/pod/perldelta.pod @@ -236,6 +236,13 @@ caused sub-threads in threaded -DPERL_TRACE_OPS builds to spew exceedingly large op-counts at destruct. These counts would print %x as "ABABABAB", clearly a mem-poison value. +=item * + +A leak in the XS typemap caused one scalar to be leaked each time a C +or a C was Ced or imported to Perl, since perl 5.000. These +particular typemap entries are thought to be extremely rarely used by XS +modules. [perl #124181] + =back =head1 Acknowledgements -- 1.7.9.msysgit.0 ```

p5pRT commented 9 years ago

From @bulk88

On Wed Jun 24 12:50:22 2015\, bulk88 wrote:

Updated patches for blead/5.23.

Bump.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

From @tonycoz

On Wed Jun 24 12:50:22 2015\, bulk88 wrote:

Updated patches for blead/5.23.

Thanks\, appled as c1b8440fca7358a5c52763ce726d40026870519c and 7ed1d857c7b7016b9bde564c6802a4721d903d95.

If you can put the perldelta changes in a separate patch (or just as prose in the ticket) it will simplify applying these changes\, since perldelta sees a lot of churn.

Thanks\, Tony

p5pRT commented 9 years ago

@tonycoz - Status changed from 'open' to 'resolved'

Perl / perl5