search

Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.98k stars 559 forks source link

"panic: free from wrong pool" while freeing the env after fork on darwin #14955

Closed p5pRT closed 7 years ago

p5pRT commented 9 years ago

Migrated from rt.perl.org#126240 (status was 'resolved')

Searchable as RT126240$

p5pRT commented 9 years ago

From perl@profvince.com

Created by perl@profvince.com

fork() causes a panic at destruction time with a debugging threaded blead on Mac OS X Yosemite :

$ ./perl -e '$pid = fork and waitpid $pid\, 0' panic​: free from wrong pool\, 0!=7f8828801200 during global destruction.

which also causes t/op/magic.t to fail.

valgrind says :

$ valgrind ./perl -e '$pid = fork and waitpid $pid\, 0' ==9987== Memcheck\, a memory error detector ==9987== Copyright (C) 2002-2015\, and GNU GPL'd\, by Julian Seward et al. ==9987== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==9987== Command​: ./perl -e $pid\ =\ fork\ and\ waitpid\ $pid\,\ 0 ==9987== ==9988== Invalid read of size 8 ==9988== at 0x1001E76FC​: Perl_safesysfree (util.c​:348) ==9988== by 0x100065E23​: perl_destruct (perl.c​:836) ==9988== by 0x100000AF1​: main (perlmain.c​:127) ==9988== Address 0x100e5c038 is 24 bytes before a block of size 19 alloc'd ==9988== at 0x100627EBB​: malloc (in /usr/local/Cellar/valgrind/3.11.0/lib/valgrind/vgpreload_memcheck-amd64-darwin.so) ==9988== by 0x10081CAD5​: __setenv (in /usr/lib/system/libsystem_c.dylib) ==9988== by 0x10081CE59​: setenv (in /usr/lib/system/libsystem_c.dylib) ==9988== by 0x100A7B4A9​: xpc_atfork_child (in /usr/lib/system/libxpc.dylib) ==9988== by 0x100630B9E​: libSystem_atfork_child (in /usr/lib/libSystem.B.dylib) ==9988== by 0x1007C8F07​: fork (in /usr/lib/system/libsystem_c.dylib) ==9988== by 0x1001F234C​: Perl_my_fork (util.c​:2705) ==9988== by 0x10042636C​: Perl_pp_fork (pp_sys.c​:4189) ==9988== by 0x1001E41BB​: Perl_runops_debug (dump.c​:2224) ==9988== by 0x10006F9CA​: S_run_body (perl.c​:2456) ==9988== by 0x10006E89A​: perl_run (perl.c​:2379) ==9988== by 0x100000A61​: main (perlmain.c​:116) ==9988== ==9988== Invalid read of size 8 ==9988== at 0x1001E7714​: Perl_safesysfree (util.c​:350) ==9988== by 0x100065E23​: perl_destruct (perl.c​:836) ==9988== by 0x100000AF1​: main (perlmain.c​:127) ==9988== Address 0x100e5c038 is 24 bytes before a block of size 19 alloc'd ==9988== at 0x100627EBB​: malloc (in /usr/local/Cellar/valgrind/3.11.0/lib/valgrind/vgpreload_memcheck-amd64-darwin.so) ==9988== by 0x10081CAD5​: __setenv (in /usr/lib/system/libsystem_c.dylib) ==9988== by 0x10081CE59​: setenv (in /usr/lib/system/libsystem_c.dylib) ==9988== by 0x100A7B4A9​: xpc_atfork_child (in /usr/lib/system/libxpc.dylib) ==9988== by 0x100630B9E​: libSystem_atfork_child (in /usr/lib/libSystem.B.dylib) ==9988== by 0x1007C8F07​: fork (in /usr/lib/system/libsystem_c.dylib) ==9988== by 0x1001F234C​: Perl_my_fork (util.c​:2705) ==9988== by 0x10042636C​: Perl_pp_fork (pp_sys.c​:4189) ==9988== by 0x1001E41BB​: Perl_runops_debug (dump.c​:2224) ==9988== by 0x10006F9CA​: S_run_body (perl.c​:2456) ==9988== by 0x10006E89A​: perl_run (perl.c​:2379) ==9988== by 0x100000A61​: main (perlmain.c​:116) ==9988== panic​: free from wrong pool\, 0!=100e27150 during global destruction. ==9988== ==9988== HEAP SUMMARY​: ==9988== in use at exit​: 186\,142 bytes in 1\,021 blocks ==9988== total heap usage​: 1\,209 allocs\, 188 frees\, 212\,558 bytes allocated ==9988== ==9988== LEAK SUMMARY​: ==9988== definitely lost​: 231 bytes in 1 blocks ==9988== indirectly lost​: 0 bytes in 0 blocks ==9988== possibly lost​: 0 bytes in 0 blocks ==9988== still reachable​: 150\,107 bytes in 591 blocks ==9988== suppressed​: 35\,804 bytes in 429 blocks ==9988== Rerun with --leak-check=full to see details of leaked memory ==9988== ==9988== For counts of detected and suppressed errors\, rerun with​: -v ==9988== ERROR SUMMARY​: 2 errors from 2 contexts (suppressed​: 0 from 0) ==9987== ==9987== HEAP SUMMARY​: ==9987== in use at exit​: 34\,920 bytes in 427 blocks ==9987== total heap usage​: 1\,199 allocs\, 772 frees\, 202\,849 bytes allocated ==9987== ==9987== LEAK SUMMARY​: ==9987== definitely lost​: 231 bytes in 1 blocks ==9987== indirectly lost​: 0 bytes in 0 blocks ==9987== possibly lost​: 0 bytes in 0 blocks ==9987== still reachable​: 0 bytes in 0 blocks ==9987== suppressed​: 34\,689 bytes in 426 blocks ==9987== Rerun with --leak-check=full to see details of leaked memory ==9987== ==9987== For counts of detected and suppressed errors\, rerun with​: -v ==9987== ERROR SUMMARY​: 0 errors from 0 contexts (suppressed​: 0 from 0)

Perl Info ``` Flags: category=core severity=medium Site configuration information for perl 5.23.4: Configured by vincent at Thu Oct 1 13:59:15 BRT 2015. Summary of my perl5 (revision 5 version 23 subversion 4) configuration: Commit id: 4b89cb47a04366ffd503d523f2fcf2375cec0a8e Platform: osname=darwin, osvers=14.5.0, archname=darwin-thread-multi-2level uname='darwin dhcp-147-65-161-189.impa.br 14.5.0 darwin kernel version 14.5.0: wed jul 29 02:26:53 pdt 2015; root:xnu-2782.40.9~1release_x86_64 x86_64 ' config_args='-des -Dusedevel -Doptimize=-O0 -g3 -DDEBUGGING -Duseithreads' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-common -DPERL_DARWIN -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include', optimize='-O0 -g3', cppflags='-fno-common -DPERL_DARWIN -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='', gccversion='4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/7.0.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc perllibs=-lpthread -ldl -lm -lutil -lc libc=, so=dylib, useshrplib=false, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong' @INC for perl 5.23.4: /usr/local/lib/perl5/site_perl/5.23.4/darwin-thread-multi-2level /usr/local/lib/perl5/site_perl/5.23.4 /usr/local/lib/perl5/5.23.4/darwin-thread-multi-2level /usr/local/lib/perl5/5.23.4 /usr/local/lib/perl5/site_perl . /tmp/perl/lib Environment for perl 5.23.4: DYLD_LIBRARY_PATH=/tmp/perl HOME=/Users/vincent LANG=C LANGUAGE (unset) LC_ALL=C LD_LIBRARY_PATH=/usr/local/lib:/usr/lib LOGDIR (unset) PATH=/Users/vincent/bin:/Users/vincent/perl/builds/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin PERL5LIB= PERLDOC_PAGER=less -R PERL_BADLANG (unset) SHELL=/bin/bash ```
p5pRT commented 9 years ago

From @bulk88

On Thu Oct 01 10​:20​:48 2015\, perl@​profvince.com wrote​:

This is a bug report for perl from perl@​profvince.com\, generated with the help of perlbug 1.40 running under perl 5.23.4.

----------------------------------------------------------------- [Please describe your issue here]

fork() causes a panic at destruction time with a debugging threaded blead on Mac OS X Yosemite :

$ ./perl -e '$pid = fork and waitpid $pid\, 0' panic​: free from wrong pool\, 0!=7f8828801200 during global destruction.

which also causes t/op/magic.t to fail.

valgrind says :

$ valgrind ./perl -e '$pid = fork and waitpid $pid\, 0' ==9987== Memcheck\, a memory error detector ==9987== Copyright (C) 2002-2015\, and GNU GPL'd\, by Julian Seward et al. ==9987== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==9987== Command​: ./perl -e $pid\ =\ fork\ and\ waitpid\ $pid\,\ 0 ==9987== ==9988== Invalid read of size 8 ==9988== at 0x1001E76FC​: Perl_safesysfree (util.c​:348) ==9988== by 0x100065E23​: perl_destruct (perl.c​:836)

What is exactly is perl.c​:836 ? What C code is at line 836 on your perl?

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 9 years ago

From perl@profvince.com

What is exactly is perl.c​:836 ? What C code is at line 836 on your perl?

gitweb is kind enough to answer your question :

http​://perl5.git.perl.org/perl.git/blob/4b89cb47a04366ffd503d523f2fcf2375cec0a8e​:/perl.c#l836

p5pRT commented 9 years ago

From @tonycoz

On Thu Oct 01 10​:20​:48 2015\, perl@​profvince.com wrote​:

fork() causes a panic at destruction time with a debugging threaded blead on Mac OS X Yosemite :

$ ./perl -e '$pid = fork and waitpid $pid\, 0' panic​: free from wrong pool\, 0!=7f8828801200 during global destruction.

I'm not seeing this in blead​:

neptune​:perl tony$ uname -a Darwin neptune.local 14.5.0 Darwin Kernel Version 14.5.0​: Tue Sep 1 21​:23​:09 PDT 2015; root​:xnu-2782.50.1~1/RELEASE_X86_64 x86_64 neptune​:perl tony$ ./perl -v

This is perl 5\, version 23\, subversion 4 (v5.23.4 (v5.23.3-83-gf7294c1)) built for darwin-thread-multi-2level

... neptune​:perl tony$ ./perl -e '$pid = fork and waitpid $pid\, 0' neptune​:perl tony$ ./perl -Ilib -V​:config_args config_args='-des -Dusedevel -Doptimize=-O0 -g3 -DDEBUGGING -Duseithreads'; neptune​:perl tony$

Tony

p5pRT commented 9 years ago

From perl@profvince.com

Le 09/11/2015 21​:23\, Tony Cook via RT a écrit :

On Thu Oct 01 10​:20​:48 2015\, perl@​profvince.com wrote​:

fork() causes a panic at destruction time with a debugging threaded blead on Mac OS X Yosemite :

$ ./perl -e '$pid = fork and waitpid $pid\, 0' panic​: free from wrong pool\, 0!=7f8828801200 during global destruction.

I'm not seeing this in blead​:

neptune​:perl tony$ uname -a Darwin neptune.local 14.5.0 Darwin Kernel Version 14.5.0​: Tue Sep 1 21​:23​:09 PDT 2015; root​:xnu-2782.50.1~1/RELEASE_X86_64 x86_64 neptune​:perl tony$ ./perl -v

This is perl 5\, version 23\, subversion 4 (v5.23.4 (v5.23.3-83-gf7294c1)) built for darwin-thread-multi-2level

... neptune​:perl tony$ ./perl -e '$pid = fork and waitpid $pid\, 0' neptune​:perl tony$ ./perl -Ilib -V​:config_args config_args='-des -Dusedevel -Doptimize=-O0 -g3 -DDEBUGGING -Duseithreads'; neptune​:perl tony$

Well I still do :

$ ./perl -Ilib -e'$pid = fork and waitpid $pid\, 0' panic​: free from wrong pool\, 0!=7f9ae1001200 during global destruction.

$ uname -a Darwin vega.local 14.5.0 Darwin Kernel Version 14.5.0​: Tue Sep 1 21​:23​:09 PDT 2015; root​:xnu-2782.50.1~1/RELEASE_X86_64 x86_64

$ ./perl -Ilib -V Summary of my perl5 (revision 5 version 23 subversion 5) configuration​:   Commit id​: a74a9a49e542832922f217362cd5de93fee73335   Platform​:   osname=darwin\, osvers=14.5.0\, archname=darwin-thread-multi-2level   uname='darwin vega.local 14.5.0 darwin kernel version 14.5.0​: tue sep 1 21​:23​:09 pdt 2015; root​:xnu-2782.50.1~1release_x86_64 x86_64 '   config_args='-des -Dusedevel -Doptimize=-O0 -g3 -DDEBUGGING -Duseithreads'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=define\, usemultiplicity=define   use64bitint=define\, use64bitall=define\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.10 -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'\,   optimize='-O0 -g3'\,   cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.10 -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'   ccversion=''\, gccversion='4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)'\, gccosandvers=''   intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3   ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='cc'\, ldflags =' -mmacosx-version-min=10.10 -fstack-protector-strong -L/usr/local/lib'   libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/7.0.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib   libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc   perllibs=-lpthread -ldl -lm -lutil -lc   libc=\, so=dylib\, useshrplib=false\, libperl=libperl.a   gnulibc_version=''   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=bundle\, d_dlsymun=undef\, ccdlflags=' '   cccdlflags=' '\, lddlflags=' -mmacosx-version-min=10.10 -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:   Compile-time options​: DEBUGGING HAS_TIMES MULTIPLICITY PERLIO_LAYERS   PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV   PERL_HASH_FUNC_ONE_AT_A_TIME_HARD   PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP   PERL_PRESERVE_IVUV PERL_TRACK_MEMPOOL PERL_USE_DEVEL   USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS   USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE   USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME   USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API   Built under darwin   Compiled at Nov 9 2015 21​:35​:14   %ENV​:   PERLDOC_PAGER="less -R"   @​INC​:   lib   /usr/local/lib/perl5/site_perl/5.23.5/darwin-thread-multi-2level   /usr/local/lib/perl5/site_perl/5.23.5   /usr/local/lib/perl5/5.23.5/darwin-thread-multi-2level   /usr/local/lib/perl5/5.23.5   /usr/local/lib/perl5/site_perl   .

p5pRT commented 8 years ago

From @tonycoz

On Mon Nov 09 15​:42​:47 2015\, perl@​profvince.com wrote​:

Le 09/11/2015 21​:23\, Tony Cook via RT a écrit :

On Thu Oct 01 10​:20​:48 2015\, perl@​profvince.com wrote​:

fork() causes a panic at destruction time with a debugging threaded blead on Mac OS X Yosemite :

$ ./perl -e '$pid = fork and waitpid $pid\, 0' panic​: free from wrong pool\, 0!=7f8828801200 during global destruction.

I'm not seeing this in blead​:

Well I still do :

$ ./perl -Ilib -e'$pid = fork and waitpid $pid\, 0' panic​: free from wrong pool\, 0!=7f9ae1001200 during global destruction.

I've now reproduced this.

It doesn't happen within a ssh shell\, but it does in Terminal.app.

Tony

p5pRT commented 8 years ago

From @tonycoz

On Mon Dec 07 20​:46​:09 2015\, tonyc wrote​:

I've now reproduced this.

It doesn't happen within a ssh shell\, but it does in Terminal.app.

What I suspect is happening is the xpc_atfork_child function is calling setenv() and libc is rearranging the environment\, invalidating our environ pointers (allocated with malloc() instead of safesysmalloc()).

Building with -DPERL_USE_SAFE_PUTENV prevents the errors.

Tony

p5pRT commented 8 years ago

From perl@profvince.com

Le 08/12/2015 03​:54\, Tony Cook via RT a écrit :

On Mon Dec 07 20​:46​:09 2015\, tonyc wrote​:

I've now reproduced this.

It doesn't happen within a ssh shell\, but it does in Terminal.app.

What I suspect is happening is the xpc_atfork_child function is calling setenv() and libc is rearranging the environment\, invalidating our environ pointers (allocated with malloc() instead of safesysmalloc()).

Building with -DPERL_USE_SAFE_PUTENV prevents the errors.

Tony

This makes sense and matches what I've been seeing intermittently from valgrind. Does this mean that this option should always be enabled on darwin then? What are the trade-offs?

Vincent

p5pRT commented 8 years ago

From @tonycoz

On Tue Dec 08 03​:34​:17 2015\, perl@​profvince.com wrote​:

Le 08/12/2015 03​:54\, Tony Cook via RT a écrit :

Building with -DPERL_USE_SAFE_PUTENV prevents the errors.

Tony

This makes sense and matches what I've been seeing intermittently from valgrind. Does this mean that this option should always be enabled on darwin then? What are the trade-offs?

I suspect that we should be using -DPERL_USE_SAFE_PUTENV on any system we can\, since it prevents conflicts with external libraries that use setenv()/unsetenv()\, which was the problem here.

From looking at the hints\, riscos\, sufficiently modern solaris and cygwin enable it.

Please try the attached patch.

Tony

p5pRT commented 8 years ago

From @tonycoz

0001-perl-126240-use-DPERL_USE_SAFE_PUTENV-where-possible.patch ```diff From 1bde2c639b40d1021477ad7b48c14805b32c7b60 Mon Sep 17 00:00:00 2001 From: Tony Cook Date: Wed, 9 Dec 2015 10:20:46 +1100 Subject: [perl #126240] use -DPERL_USE_SAFE_PUTENV where possible on OS X On threaded builds on OS X, libSystem registers atfork handlers that call setenv(), which internally modifies members of environ[], setting them to malloc()ed blocks. In some cases Perl_my_setenv() reallocates environ[] using safesysmalloc(), which under debugging builds adds a tracking header, and if perl_destruct() sees that environ[] has been reallocated, frees it with safesysfree(). When these combine, perl attempts to free the malloc()ed block with safesysfree(), which attempts to access the tracking header, causing an invalid access in tools like valgrind, or a "free from wrong pool" error, since the header contains unrelated data. Avoid this mess by letting libc manage environ[] if unsetenv() is available. --- hints/darwin.sh | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/hints/darwin.sh b/hints/darwin.sh index ee20760..2af6ae7 100644 --- a/hints/darwin.sh +++ b/hints/darwin.sh @@ -490,3 +490,13 @@ esac # makefile in the same place. Since Darwin uses GNU make, this dodges # the problem. firstmakefile=GNUmakefile; + +# Parts of the system call setenv(), in particular in an atfork handler. +# This causes problems when the child tries to clean up environ[], so +# let libc manage environ[]. +cat >> config.over <<'EOOVER' +if test "$d_unsetenv" = "$define" -a \ + `expr "$ccflags" : '.*-DPERL_USE_SAFE_PUTENV'` -eq 0; then + ccflags="$ccflags -DPERL_USE_SAFE_PUTENV" +fi +EOOVER -- 2.4.9 (Apple Git-60) ```
p5pRT commented 8 years ago

From @jkeenan

On Tue Dec 08 15​:38​:01 2015\, tonyc wrote​:

On Tue Dec 08 03​:34​:17 2015\, perl@​profvince.com wrote​:

Le 08/12/2015 03​:54\, Tony Cook via RT a écrit :

Building with -DPERL_USE_SAFE_PUTENV prevents the errors.

Tony

This makes sense and matches what I've been seeing intermittently from valgrind. Does this mean that this option should always be enabled on darwin then? What are the trade-offs?

I suspect that we should be using -DPERL_USE_SAFE_PUTENV on any system we can\, since it prevents conflicts with external libraries that use setenv()/unsetenv()\, which was the problem here.

From looking at the hints\, riscos\, sufficiently modern solaris and cygwin enable it.

Please try the attached patch.

Tony

Smoking in branch​: smoke-me/jkeenan/126240-darwin-use-safe-putenv

-- James E Keenan (jkeenan@​cpan.org)

p5pRT commented 8 years ago

From @rjbs

This patch cleared up my tests failures.

-- rjbs

p5pRT commented 8 years ago

From @jkeenan

On Tue Dec 08 16​:22​:42 2015\, jkeenan wrote​:

On Tue Dec 08 15​:38​:01 2015\, tonyc wrote​:

On Tue Dec 08 03​:34​:17 2015\, perl@​profvince.com wrote​:

Le 08/12/2015 03​:54\, Tony Cook via RT a écrit :

Building with -DPERL_USE_SAFE_PUTENV prevents the errors.

Tony

This makes sense and matches what I've been seeing intermittently from valgrind. Does this mean that this option should always be enabled on darwin then? What are the trade-offs?

I suspect that we should be using -DPERL_USE_SAFE_PUTENV on any system we can\, since it prevents conflicts with external libraries that use setenv()/unsetenv()\, which was the problem here.

From looking at the hints\, riscos\, sufficiently modern solaris and cygwin enable it.

Please try the attached patch.

Tony

Smoking in branch​: smoke-me/jkeenan/126240-darwin-use-safe-putenv

Built successfully on older Darwin; see attachment.

-- James E Keenan (jkeenan@​cpan.org)

p5pRT commented 8 years ago

From @jkeenan

Summary of my perl5 (revision 5 version 23 subversion 6) configuration​:   Commit id​: 35abf35a36fd489b3434f0435baa68d4c7a31df6   Platform​:   osname=darwin\, osvers=8.11.0\, archname=darwin-2level   uname='darwin macintosh-9.local 8.11.0 darwin kernel version 8.11.0​: wed oct 10 18​:26​:00 pdt 2007; root​:xnu-792.24.17~1release_ppc power macintosh powerpc '   config_args='-des -Dusedevel'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=undef\, usemultiplicity=undef   use64bitint=undef\, use64bitall=undef\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -I/usr/local/include -I/opt/local/include -D_FORTIFY_SOURCE=2 -DPERL_USE_SAFE_PUTENV'\,   optimize='-O3'\,   cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -I/usr/local/include -I/opt/local/include'   ccversion=''\, gccversion='4.0.1 (Apple Computer\, Inc. build 5250)'\, gccosandvers=''   intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=4321\, doublekind=4   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=6   ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'\, ldflags =' -L/usr/local/lib -L/opt/local/lib'   libpth=/usr/local/lib /usr/lib /opt/local/lib   libs=-lpthread -ldbm -ldl -lm -lc   perllibs=-lpthread -ldl -lm -lc   libc=\, so=dylib\, useshrplib=false\, libperl=libperl.a   gnulibc_version=''   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=bundle\, d_dlsymun=undef\, ccdlflags=' '   cccdlflags=' '\, lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib'

Characteristics of this binary (from libperl)​:   Compile-time options​: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE   PERL_DONT_CREATE_GVSV   PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP   PERL_PRESERVE_IVUV PERL_USE_DEVEL   PERL_USE_SAFE_PUTENV USE_LARGE_FILES USE_LOCALE   USE_LOCALE_COLLATE USE_LOCALE_CTYPE   USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO   USE_PERL_ATOF   Built under darwin   Compiled at Dec 8 2015 20​:13​:46   %ENV​:   PERLBREW_BASHRC_VERSION="0.59"   PERLBREW_HOME="/Users/jimk/.perlbrew"   PERLBREW_ROOT="/Users/jimk/perl5/perlbrew"   PERL_WORKDIR="gitwork/perl"   @​INC​:   lib   /usr/local/lib/perl5/site_perl/5.23.6/darwin-2level   /usr/local/lib/perl5/site_perl/5.23.6   /usr/local/lib/perl5/5.23.6/darwin-2level   /usr/local/lib/perl5/5.23.6   /usr/local/lib/perl5/site_perl   .

p5pRT commented 8 years ago

From @iabyn

On Tue\, Dec 08\, 2015 at 03​:38​:02PM -0800\, Tony Cook via RT wrote​:

On Tue Dec 08 03​:34​:17 2015\, perl@​profvince.com wrote​:

Le 08/12/2015 03​:54\, Tony Cook via RT a écrit :

Building with -DPERL_USE_SAFE_PUTENV prevents the errors.

Tony

This makes sense and matches what I've been seeing intermittently from valgrind. Does this mean that this option should always be enabled on darwin then? What are the trade-offs?

I suspect that we should be using -DPERL_USE_SAFE_PUTENV on any system we can

By "can"\, do you mean any system that physically works with PERL_USE_SAFE_PUTENV\, or any system that doesn't leak with PERL_USE_SAFE_PUTENV?

-- "Procrastination grows to fill the available time"   -- Mitchell's corollary to Parkinson's Law

p5pRT commented 8 years ago

From @tonycoz

On Wed Dec 09 07​:08​:21 2015\, davem wrote​:

On Tue\, Dec 08\, 2015 at 03​:38​:02PM -0800\, Tony Cook via RT wrote​:

I suspect that we should be using -DPERL_USE_SAFE_PUTENV on any system we can

By "can"\, do you mean any system that physically works with PERL_USE_SAFE_PUTENV\, or any system that doesn't leak with PERL_USE_SAFE_PUTENV?

I guess that's the hard bit to define.

I'm guessing any system that has unsetenv() will be safe\, since setenv()\, getenv() and unsetenv() *should* be working together to manage memory correctly.

putenv()'s memory handling isn't well defined in POSIX\, ie. there's no assumption in POSIX[1] (nor the man pages for Linux\, BSD\, OS X that I saw) that such memory is allocated with malloc()\, so an implementation can't really free() it safely.

POSIX also mentions[2] that using putenv() can (theoretically) slow down getenv() if an implementation uses an internal structure to speed up environment variable lookups.

Tony

[1] http​://pubs.opengroup.org/onlinepubs/9699919799/functions/putenv.html The rationale talks about free() in passing\, but there's no requirement

[2] http​://pubs.opengroup.org/onlinepubs/9699919799/functions/getenv.html in the rationale.

p5pRT commented 8 years ago

From @tonycoz

On Tue Dec 08 17​:50​:39 2015\, rjbs wrote​:

This patch cleared up my tests failures.

Applied as e3962106e93fa80bf960e71ac7c7727e7e15eee8.

Leaving this open a little longer in case there's more discussion of -DPERL_USE_SAFE_PUTENV in the more general case.

Tony

p5pRT commented 8 years ago

From @tonycoz

On Wed Dec 16 22​:06​:58 2015\, tonyc wrote​:

On Tue Dec 08 17​:50​:39 2015\, rjbs wrote​:

This patch cleared up my tests failures.

Applied as e3962106e93fa80bf960e71ac7c7727e7e15eee8.

Leaving this open a little longer in case there's more discussion of -DPERL_USE_SAFE_PUTENV in the more general case.

This required an extra fix in 12ffbb10fe94c82dda90efa15d691ababa5b8b8f.

Tony

p5pRT commented 8 years ago

From mojca.miklavec.lists@gmail.com

I'm having problems building Perl 5.24.0-RC5 (I didn't try older ones)\, see https​://rt.perl.org/Ticket/Display.html?id=128093 and https://trac.macports.org/ticket/51330

I'm not saying that commits related to this ticket lead to a build problem\, but this is the only ticket related to "environ" that I found in the changelog.

p5pRT commented 7 years ago

From @tonycoz

On Sat\, 07 May 2016 04​:55​:20 -0700\, mojca.miklavec.lists@​gmail.com wrote​:

I'm having problems building Perl 5.24.0-RC5 (I didn't try older ones)\, see https://rt-archive.perl.org/perl5/Ticket/Display.html?id=128093 and https://trac.macports.org/ticket/51330

I'm not saying that commits related to this ticket lead to a build problem\, but this is the only ticket related to "environ" that I found in the changelog.

This was unrelated\, closing.

Tony

p5pRT commented 7 years ago

@tonycoz - Status changed from 'open' to 'pending release'

p5pRT commented 7 years ago

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0\, this and 210 other issues have been resolved.

Perl 5.26.0 may be downloaded via​: https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists\, feel free to reopen this ticket.

p5pRT commented 7 years ago

@khwilliamson - Status changed from 'pending release' to 'resolved'