Closed p5pRT closed 12 years ago
Title Purify located memory errors
Symptom I just completed an application which showed the existence of memory errors in the Perl language itself. My application formerly hung when shelling out to unix with backticks. When I ran the spawned command from a csh\, it executed quickly. Nonetheless\, my perl application hung at the backticks and did not even start a new unix process (verified with ps). Updating unrelated portions of my application made the hang problem disappear or reappear. This is the kind of inconsistent\, erratic behavior which is typified by memory errors.
Purify build To test for memory errors\, I (partially) built the perl language (perl-5.8.4) with purify. (Purify is a commercial source code library used to locate memory errors in "C".) My host is a SGI ("uname -a" shows "IRIX64 e0spg11 6.5 07121149 IP35"). To enable a purify build\, I changed "cc -n32" to "purify cc -g -n32" in 3 places in config.sh.
Build problems I executed "make ; make test". The build proceeded nicely for a while but then miniperl hung. I aborted that miniperl process and the build continued on. Hundreds of the tests failed\, probably because the build partially failed. The build log shows:
*** lib/auto/DynaLoader/DynaLoader.a removed.
I added "set -vx" to perl-5.8.4/ext/util/make_ext and determined that miniperl hung when it was passed the following command line arguments:
-I../../lib Makefile.PL INSTALLDIRS=perl PERL_CORE=1 LIBPERL_A=libperl.a
I tried to debug miniperl via "dbx ../../miniperl" but received "error in step/next command".
Errors located with purify My partial build identified the following errors:
**** Purify instrumented ./perl (pid 92576237) **** ABR: Array bounds read (2 times): * This is occurring while in: Perl_yyparse [perly.c:1599] S_parse_body [perl.c:1765] perl_parse [perl.c:1270] main [perlmain.c:84] __start [crt1text.s:176] * Reading 4 bytes from 0x101dff18 in the heap. * Address 0x101dff18 is 1 byte past end of a malloc'd block at 0x101dfa68 of 1200 bytes. * This block was allocated from: _malloc [malloc.c:1164] Perl_safesysmalloc [util.c:67] Perl_yyparse [perly.c:1456] S_parse_body [perl.c:1765] perl_parse [perl.c:1270] main [perlmain.c:84]
**** Purify instrumented ./perl (pid 93060130) **** PAR: Bad function parameter: * This is occurring while in: XS_DynaLoader_dl_load_file [DynaLoader.xs:191] Perl_pp_entersub [pp_hot.c:2854] Perl_runops_standard [run.c:24] S_call_body [perl.c:2285] Perl_call_sv [perl.c:2203] S_call_list_body [perl.c:4663] * dlopen("../lib/auto/POSIX/POSIX.so"\, 1) Purify'd file (../lib/auto/POSIX/POSIX.so) not found.
I reported the following error to SGI (the hardware vendor): **** Purify instrumented ./miniperl (pid 92083168) **** UMR: Uninitialized memory read (2 times): * This is occurring while in: _setcontext [syscalls.c:15191] _siglongjmp [siglongjmp.c:46] Perl_vdie [util.c:1108] Perl_die [util.c:1132] Perl_pp_die [pp_sys.c:510] Perl_runops_standard [run.c:24] * Reading 800 bytes from 0x7fff28f0 on the stack (52 bytes at 0x7fff28f4 uninit).
Memory leaks Small memory leaks present no problem for short-running applications (like mine). However\, for applications which run a long time\, memory leaks can gradually consume all the physical and virtual memory on a computer. My purify tests located some memory errors which I haven't reported here for the purposes of brevity.
The next step Does anyone test new perl versions for memory errors? My management will allow me to conduct some memory error testing. Note that my build failed part way through and is therefore not an exhaustive test for memory errors. I'm considering trying for a more complete purify build of perl.
On Sat\, Jun 05\, 2004 at 05:26:54AM -0000\, David M Zokaites wrote:
Purify build To test for memory errors\, I (partially) built the perl language (perl-5.8.4) with purify.
In the file Porting/pumpkin.pod\, there is a section entitled 'Running Purify'; you may want to rebuild using those instructions first\, then let us know if the memory errors persist.
Regards\,
Dave.
-- Technology is dominated by two types of people: those who understand what they do not manage\, and those who manage what they do not understand.
The RT System itself - Status changed from 'new' to 'open'
Overview An improved purify build procedure shows more memory errors than my previous effort.
Revised build procedure As suggested\, I followed the purify instructions in Porting/pumpkin.pod. I did make a few minor changes. Here is my procedure:
Ignore an error in SGI source code (this step was not in pumpkin.pod) echo 'suppress umr _setcontext' > ~/.purify
Purify build steps cd PerlBuild2/perl-5.8.4
script today2 rm config.sh Policy.sh sh Configure -des -Doptimize=-g -Uusemymalloc -Dusemultiplicity \ -Accflags=-DPURIFY
setenv PURIFYOPTIONS \ "-chain-length=25 -windows=yes -log-file=perl.plog \ -view-file=perl.pv -append-logfile=yes"
(-windows=yes was not in pumpkin.pod)
make all pureperl
cd t ln -s ../pureperl perl setenv PERL_DESTRUCT_LEVEL 2 ./perl TEST
Build hung The following process hung for hours and had to be killed UID PID PPID C STIME TTY TIME CMD zokaites 103707692 103694410 0 12:51:56 pts/18 0:01 ./perl -I. \ -MTestInit op/readline.t
The following process appeared to be in an infinite loop and was killed UID PID PPID C STIME TTY TIME CMD zokaites 103764694 103694410 0 14:33:35 pts/18 57:00 ./perl -I. \ -MTestInit ../ext/Encode/t/Unicode.t
Most tests successful "perl Test" reported: Failed 10 test scripts out of 807\, 98.76% okay. This success rate is far better than my previous attempt.
Memory errors located I counted the number and types of memory errors logged by purify in file perl.plog:
Error description Count ---------------------------------- ----- ABR: Array bounds read: 4 COR: Fatal core dump: 2 FMR: Free memory read: 1 MLK: \<#> bytes leaked at \
7025 NPR: Null pointer read: 1 PAR: Bad function parameter: 12 SIG: Signal handled: 15 UMR: Uninitialized memory read: 6Hi Folks\,
My addition to this bug report (10 days ago) clearly shows the existence of memory errors in the Perl language itself:
Error description Count ---------------------------------- ----- ABR: Array bounds read: 4 COR: Fatal core dump: 2 FMR: Free memory read: 1 MLK: \<#> bytes leaked at \
7025 NPR: Null pointer read: 1 PAR: Bad function parameter: 12 SIG: Signal handled: 15 UMR: Uninitialized memory read: 6I can readily determine the stack trace for each of the above errors\, some of which are probably redundantly reported.
The above errors were identified while running "make test". Is there any code in the Perl language to print out which test is being executed? To eliminate these memory errors\, we'll have to 1) determine which test exhibited each memory error 2) update the Perl source code to correct the error 3) rebuild with Purify 4) test the corrected code by running the problematic test
What's the best way to encourage the Perl community to work on these problems? Would it help to create separate bug reports for each memory error?
Adios\,
David Zokaites DBA / Software Engineer ECS/E&D/SAIC
"Dave Mitchell
via RT" To: zokaites@usgs.gov
\<perlbug-followup cc:
@perl.org> Subject: Re: [perl #30043] Purify located memory errors
06/05/2004 02:24
PM
Please respond to
perlbug-followup
On Sat\, Jun 05\, 2004 at 05:26:54AM -0000\, David M Zokaites wrote:
Purify build To test for memory errors\, I (partially) built the perl language (perl-5.8.4) with purify.
In the file Porting/pumpkin.pod\, there is a section entitled 'Running Purify'; you may want to rebuild using those instructions first\, then let us know if the memory errors persist.
Regards\,
Dave.
-- Technology is dominated by two types of people: those who understand what they do not manage\, and those who manage what they do not understand.
On Tue\, Jun 15\, 2004 at 10:09:12AM -0500\, David M Zokaites wrote:
My addition to this bug report (10 days ago) clearly shows the existence of memory errors in the Perl language itself:
Error description Count ---------------------------------- ----- ABR: Array bounds read: 4 COR: Fatal core dump: 2 FMR: Free memory read: 1 MLK: \<#> bytes leaked at \
7025 NPR: Null pointer read: 1 PAR: Bad function parameter: 12 SIG: Signal handled: 15 UMR: Uninitialized memory read: 6I can readily determine the stack trace for each of the above errors\, some of which are probably redundantly reported.
The above errors were identified while running "make test". Is there any code in the Perl language to print out which test is being executed?
It's possible to run each test script manually; eg from the build directory
$ ./perl -Ilib t/op/splice.t 1..12 ok 1 ok 2 ok 3 ok 4 ok 5 ok 6 ok 7 ok 8 ok 9 ok 10 ok 11 ok 12 $
To eliminate these memory errors\, we'll have to 1) determine which test exhibited each memory error 2) update the Perl source code to correct the error 3) rebuild with Purify 4) test the corrected code by running the problematic test
What's the best way to encourage the Perl community to work on these problems? Would it help to create separate bug reports for each memory error?
Having identified a particular test script that produces a leak\, try to take a copy of the script and cut it down to the barest minimum that still exhibits the leak. Then post the script and the purify report for it here (as part of the same thread/bugid for now). Only post one or two at first\, then see how we go from there.
-- Now is the discount of our winter tent -- sign seen outside camping shop
On Tue\, Jun 15\, 2004 at 10:09:12AM -0500\, David M Zokaites wrote:
My addition to this bug report (10 days ago) clearly shows the existence of memory errors in the Perl language itself:
Error description Count ---------------------------------- ----- ABR: Array bounds read: 4 COR: Fatal core dump: 2 FMR: Free memory read: 1 MLK: \<#> bytes leaked at \
7025 NPR: Null pointer read: 1 PAR: Bad function parameter: 12 SIG: Signal handled: 15 UMR: Uninitialized memory read: 6I can readily determine the stack trace for each of the above errors\, some of which are probably redundantly reported.
The above errors were identified while running "make test". Is there any code in the Perl language to print out which test is being executed?
make test (on most platforms) runs the t/TEST script. That has various modes it uses\, some based on command line switches (e.g. -compile\, -bytecompile\, -deparse)\, and some based on environment vars (e.g. PERL_VALGRIND\, PERL_3LOG). You could fairly easily adapt it to output the Purity stuff to a log file along with the names of the tests being run.
On Tue\, Jun 15\, 2004 at 10:09:12AM -0500\, David M Zokaites wrote:
MLK: \<#> bytes leaked at \
7025
N.B. There are known problems with failing evals leaking\, that are both very hard and a low priority to fix.
@iabyn - Status changed from 'open' to 'stalled'
The new slab allocator for ops should have fixed most of the cases of failed evals leaking\, right? Are there any other issues here that we know of? Has anyone tried to build perl with Purify recently?
-doy
The RT System itself - Status changed from 'stalled' to 'open'
On Thu Jul 05 13:07:10 2012\, doy wrote:
The new slab allocator for ops should have fixed most of the cases of failed evals leaking\, right?
I hope so. :-)
Are there any other issues here that we know of? Has anyone tried to build perl with Purify recently?
I donāt know.
--
Father Chrysostomos
On Thu\, Jul 5\, 2012 at 3:40 PM\, Father Chrysostomos via RT \perlbug\-followup@​perl\.org wrote:
On Thu Jul 05 13:07:10 2012\, doy wrote:
The new slab allocator for ops should have fixed most of the cases of failed evals leaking\, right?
I hope so. :-)
Are there any other issues here that we know of? Has anyone tried to build perl with Purify recently?
I donāt know.
I am constantly testing with AddressSanitzer which detects more errors than Purify or valgrind. All of the remaining core issues have been fixed recently\, just some CPAN modules are still broken. -- Reini Urban http://cpanel.net/ http://www.perl-compiler.org/
@cpansprout - Status changed from 'open' to 'resolved'
Migrated from rt.perl.org#30043 (status was 'resolved')
Searchable as RT30043$