Perl / perl5

šŸŖ The Perl programming language
https://dev.perl.org/perl5/
Other
1.98k stars 559 forks source link

Purify located memory errors #7336

Closed p5pRT closed 12 years ago

p5pRT commented 20 years ago

Migrated from rt.perl.org#30043 (status was 'resolved')

Searchable as RT30043$

p5pRT commented 20 years ago

From zokaites@usgs.gov

Title   Purify located memory errors

Symptom   I just completed an application which showed the existence of memory   errors in the Perl language itself. My application formerly hung   when shelling out to unix with backticks. When I ran the spawned   command from a csh\, it executed quickly. Nonetheless\, my perl   application hung at the backticks and did not even start a new unix   process (verified with ps). Updating unrelated portions of my   application made the hang problem disappear or reappear. This is   the kind of inconsistent\, erratic behavior which is typified by   memory errors.

Purify build   To test for memory errors\, I (partially) built the perl language   (perl-5.8.4) with purify. (Purify is a commercial source code   library used to locate memory errors in "C".) My host is a SGI   ("uname -a" shows "IRIX64 e0spg11 6.5 07121149 IP35"). To enable a   purify build\, I changed "cc -n32" to "purify cc -g -n32" in 3 places   in config.sh.

Build problems   I executed "make ; make test". The build proceeded nicely for a   while but then miniperl hung. I aborted that miniperl process and   the build continued on. Hundreds of the tests failed\, probably   because the build partially failed. The build log shows​:

  *** lib/auto/DynaLoader/DynaLoader.a removed.

  I added "set -vx" to perl-5.8.4/ext/util/make_ext and determined that   miniperl hung when it was passed the following command line arguments​:

  -I../../lib Makefile.PL INSTALLDIRS=perl PERL_CORE=1 LIBPERL_A=libperl.a

  I tried to debug miniperl via "dbx ../../miniperl" but received   "error in step/next command".

Errors located with purify   My partial build identified the following errors​:

  **** Purify instrumented ./perl (pid 92576237) ****   ABR​: Array bounds read (2 times)​:   * This is occurring while in​:   Perl_yyparse [perly.c​:1599]   S_parse_body [perl.c​:1765]   perl_parse [perl.c​:1270]   main [perlmain.c​:84]   __start [crt1text.s​:176]   * Reading 4 bytes from 0x101dff18 in the heap.   * Address 0x101dff18 is 1 byte past end of a malloc'd block at 0x101dfa68 of 1200 bytes.   * This block was allocated from​:   _malloc [malloc.c​:1164]   Perl_safesysmalloc [util.c​:67]   Perl_yyparse [perly.c​:1456]   S_parse_body [perl.c​:1765]   perl_parse [perl.c​:1270]   main [perlmain.c​:84]

  **** Purify instrumented ./perl (pid 93060130) ****   PAR​: Bad function parameter​:   * This is occurring while in​:   XS_DynaLoader_dl_load_file [DynaLoader.xs​:191]   Perl_pp_entersub [pp_hot.c​:2854]   Perl_runops_standard [run.c​:24]   S_call_body [perl.c​:2285]   Perl_call_sv [perl.c​:2203]   S_call_list_body [perl.c​:4663]   * dlopen("../lib/auto/POSIX/POSIX.so"\, 1) Purify'd file (../lib/auto/POSIX/POSIX.so) not found.

  I reported the following error to SGI (the hardware vendor)​:   **** Purify instrumented ./miniperl (pid 92083168) ****   UMR​: Uninitialized memory read (2 times)​:   * This is occurring while in​:   _setcontext [syscalls.c​:15191]   _siglongjmp [siglongjmp.c​:46]   Perl_vdie [util.c​:1108]   Perl_die [util.c​:1132]   Perl_pp_die [pp_sys.c​:510]   Perl_runops_standard [run.c​:24]   * Reading 800 bytes from 0x7fff28f0 on the stack (52 bytes at 0x7fff28f4 uninit).

Memory leaks   Small memory leaks present no problem for short-running applications   (like mine). However\, for applications which run a long time\,   memory leaks can gradually consume all the physical and virtual   memory on a computer. My purify tests located some memory errors   which I haven't reported here for the purposes of brevity.

The next step   Does anyone test new perl versions for memory errors? My management   will allow me to conduct some memory error testing. Note that my   build failed part way through and is therefore not an exhaustive test   for memory errors. I'm considering trying for a more complete   purify build of perl.

p5pRT commented 20 years ago

From @iabyn

On Sat\, Jun 05\, 2004 at 05​:26​:54AM -0000\, David M Zokaites wrote​:

Purify build To test for memory errors\, I (partially) built the perl language (perl-5.8.4) with purify.

In the file Porting/pumpkin.pod\, there is a section entitled 'Running Purify'; you may want to rebuild using those instructions first\, then let us know if the memory errors persist.

Regards\,

Dave.

-- Technology is dominated by two types of people​: those who understand what they do not manage\, and those who manage what they do not understand.

p5pRT commented 20 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 20 years ago

From zokaites@usgs.gov

Overview   An improved purify build procedure shows more memory errors than my   previous effort.

Revised build procedure   As suggested\, I followed the purify instructions in   Porting/pumpkin.pod. I did make a few minor changes. Here is my   procedure​:

  Ignore an error in SGI source code (this step was not in pumpkin.pod)   echo 'suppress umr _setcontext' > ~/.purify

  Purify build steps   cd PerlBuild2/perl-5.8.4

  script today2   rm config.sh Policy.sh   sh Configure -des -Doptimize=-g -Uusemymalloc -Dusemultiplicity \   -Accflags=-DPURIFY

  setenv PURIFYOPTIONS \   "-chain-length=25 -windows=yes -log-file=perl.plog \   -view-file=perl.pv -append-logfile=yes"

  (-windows=yes was not in pumpkin.pod)

  make all pureperl

  cd t   ln -s ../pureperl perl   setenv PERL_DESTRUCT_LEVEL 2   ./perl TEST

Build hung   The following process hung for hours and had to be killed   UID PID PPID C STIME TTY TIME CMD   zokaites 103707692 103694410 0 12​:51​:56 pts/18 0​:01 ./perl -I. \   -MTestInit op/readline.t

  The following process appeared to be in an infinite loop and was   killed   UID PID PPID C STIME TTY TIME CMD   zokaites 103764694 103694410 0 14​:33​:35 pts/18 57​:00 ./perl -I. \   -MTestInit ../ext/Encode/t/Unicode.t

Most tests successful   "perl Test" reported​:   Failed 10 test scripts out of 807\, 98.76% okay.   This success rate is far better than my previous attempt.

Memory errors located   I counted the number and types of memory errors logged by purify in   file perl.plog​:

  Error description Count   ---------------------------------- -----   ABR​: Array bounds read​: 4   COR​: Fatal core dump​: 2   FMR​: Free memory read​: 1   MLK​: \<#> bytes leaked at \

7025   NPR​: Null pointer read​: 1   PAR​: Bad function parameter​: 12   SIG​: Signal handled​: 15   UMR​: Uninitialized memory read​: 6

p5pRT commented 20 years ago

From zokaites@usgs.gov

Hi Folks\,

My addition to this bug report (10 days ago) clearly shows the existence of memory errors in the Perl language itself​:

Error description Count ---------------------------------- ----- ABR​: Array bounds read​: 4 COR​: Fatal core dump​: 2 FMR​: Free memory read​: 1 MLK​: \<#> bytes leaked at \

7025 NPR​: Null pointer read​: 1 PAR​: Bad function parameter​: 12 SIG​: Signal handled​: 15 UMR​: Uninitialized memory read​: 6

I can readily determine the stack trace for each of the above errors\, some of which are probably redundantly reported.

The above errors were identified while running "make test". Is there any code in the Perl language to print out which test is being executed? To eliminate these memory errors\, we'll have to   1) determine which test exhibited each memory error   2) update the Perl source code to correct the error   3) rebuild with Purify   4) test the corrected code by running the problematic test

What's the best way to encourage the Perl community to work on these problems? Would it help to create separate bug reports for each memory error?

Adios\,

David Zokaites DBA / Software Engineer ECS/E&D/SAIC

 
  "Dave Mitchell
  via RT" To​: zokaites@​usgs.gov
  \<perlbug-followup cc​:
  @​perl.org> Subject​: Re​: [perl #30043] Purify located memory errors
 
  06/05/2004 02​:24
  PM
  Please respond to
  perlbug-followup
 
 

On Sat\, Jun 05\, 2004 at 05​:26​:54AM -0000\, David M Zokaites wrote​:

Purify build To test for memory errors\, I (partially) built the perl language (perl-5.8.4) with purify.

In the file Porting/pumpkin.pod\, there is a section entitled 'Running Purify'; you may want to rebuild using those instructions first\, then let us know if the memory errors persist.

Regards\,

Dave.

-- Technology is dominated by two types of people​: those who understand what they do not manage\, and those who manage what they do not understand.

p5pRT commented 20 years ago

From @iabyn

On Tue\, Jun 15\, 2004 at 10​:09​:12AM -0500\, David M Zokaites wrote​:

My addition to this bug report (10 days ago) clearly shows the existence of memory errors in the Perl language itself​:

Error description Count ---------------------------------- ----- ABR​: Array bounds read​: 4 COR​: Fatal core dump​: 2 FMR​: Free memory read​: 1 MLK​: \<#> bytes leaked at \

7025 NPR​: Null pointer read​: 1 PAR​: Bad function parameter​: 12 SIG​: Signal handled​: 15 UMR​: Uninitialized memory read​: 6

I can readily determine the stack trace for each of the above errors\, some of which are probably redundantly reported.

The above errors were identified while running "make test". Is there any code in the Perl language to print out which test is being executed?

It's possible to run each test script manually; eg from the build directory

$ ./perl -Ilib t/op/splice.t 1..12 ok 1 ok 2 ok 3 ok 4 ok 5 ok 6 ok 7 ok 8 ok 9 ok 10 ok 11 ok 12 $

To eliminate these memory errors\, we'll have to 1) determine which test exhibited each memory error 2) update the Perl source code to correct the error 3) rebuild with Purify 4) test the corrected code by running the problematic test

What's the best way to encourage the Perl community to work on these problems? Would it help to create separate bug reports for each memory error?

Having identified a particular test script that produces a leak\, try to take a copy of the script and cut it down to the barest minimum that still exhibits the leak. Then post the script and the purify report for it here (as part of the same thread/bugid for now). Only post one or two at first\, then see how we go from there.

-- Now is the discount of our winter tent   -- sign seen outside camping shop

p5pRT commented 20 years ago

From @ysth

On Tue\, Jun 15\, 2004 at 10​:09​:12AM -0500\, David M Zokaites wrote​:

My addition to this bug report (10 days ago) clearly shows the existence of memory errors in the Perl language itself​:

Error description Count ---------------------------------- ----- ABR​: Array bounds read​: 4 COR​: Fatal core dump​: 2 FMR​: Free memory read​: 1 MLK​: \<#> bytes leaked at \

7025 NPR​: Null pointer read​: 1 PAR​: Bad function parameter​: 12 SIG​: Signal handled​: 15 UMR​: Uninitialized memory read​: 6

I can readily determine the stack trace for each of the above errors\, some of which are probably redundantly reported.

The above errors were identified while running "make test". Is there any code in the Perl language to print out which test is being executed?

make test (on most platforms) runs the t/TEST script. That has various modes it uses\, some based on command line switches (e.g. -compile\, -bytecompile\, -deparse)\, and some based on environment vars (e.g. PERL_VALGRIND\, PERL_3LOG). You could fairly easily adapt it to output the Purity stuff to a log file along with the names of the tests being run.

p5pRT commented 20 years ago

From @ysth

On Tue\, Jun 15\, 2004 at 10​:09​:12AM -0500\, David M Zokaites wrote​:

MLK​: \<#> bytes leaked at \

7025

N.B. There are known problems with failing evals leaking\, that are both very hard and a low priority to fix.

p5pRT commented 20 years ago

@iabyn - Status changed from 'open' to 'stalled'

p5pRT commented 12 years ago

From @doy

The new slab allocator for ops should have fixed most of the cases of failed evals leaking\, right? Are there any other issues here that we know of? Has anyone tried to build perl with Purify recently?

-doy

p5pRT commented 12 years ago

The RT System itself - Status changed from 'stalled' to 'open'

p5pRT commented 12 years ago

From @cpansprout

On Thu Jul 05 13​:07​:10 2012\, doy wrote​:

The new slab allocator for ops should have fixed most of the cases of failed evals leaking\, right?

I hope so. :-)

Are there any other issues here that we know of? Has anyone tried to build perl with Purify recently?

I donā€™t know.

--

Father Chrysostomos

p5pRT commented 12 years ago

From @rurban

On Thu\, Jul 5\, 2012 at 3​:40 PM\, Father Chrysostomos via RT \perlbug\-followup@&#8203;perl\.org wrote​:

On Thu Jul 05 13​:07​:10 2012\, doy wrote​:

The new slab allocator for ops should have fixed most of the cases of failed evals leaking\, right?

I hope so. :-)

Are there any other issues here that we know of? Has anyone tried to build perl with Purify recently?

I donā€™t know.

I am constantly testing with AddressSanitzer which detects more errors than Purify or valgrind. All of the remaining core issues have been fixed recently\, just some CPAN modules are still broken. -- Reini Urban http​://cpanel.net/ http​://www.perl-compiler.org/

p5pRT commented 12 years ago

@cpansprout - Status changed from 'open' to 'resolved'