With long chains of references both Storable and Data::Dumper cause perl to fail

[p5pRT]

Migrated from rt.perl.org#36427 (status was 'open')

Searchable as RT36427$

[p5pRT]

From kynn@panix.com

I first observed this behavior when attempting to save a directed acyclic graph (DAG) consisting of of about 5\,000 nodes\, but below I give a minimalistic example that exhibits the same behavior.

The following script creates a long sequence of nested references. The command-line argument to the script should be a positive integer\, corresponding to the length of this sequence of references.

With an argument greater than some system-dependent critical size\, the script causes perl to segfault​:

use strict; use warnings;

my $size = shift;

my $x = []; $x = [ $x ] for 1..$size;

require Storable; Storable​::store( \$x\, '/tmp/$0.$$' ) or die;

warn "ok\n";

__END__

For this script\, in the system described in this report\, the critical value of the argument is 30826. With any argument below this number\, the script terminates normally; with an argument greater than or equal to it\, perl segfaults.

Similarly\, if the two lines indicated by the "# *" in the script above are replaced by

require Data​::Dumper; Data​::Dumper->Dump( [ \$x ] ) or die;

the resulting script fails for any argument greater than or equal to 15406.

Two important differences between the two versions of the script is that the one using Storable fails after the 'warn "ok\n"' line\, and always terminates quickly (whether successfully or not)\, even for values of the input close to the critical value. In contrast\, for the version using Data​::Dumper\, when the script fails the 'warn "ok\n"' line never executes\, and the time to success or failure grows sharply as the value of the input approaches the critical value. But with the DAG object that originally triggered the segfaulting\, I observed this pronounced slowing down near the critical input size with both Data​::Dumper and Storable.

Most likely the reason for the segfaulting is that a sufficiently long chain of links will cause the serialization process to start thrashing. Simply breaking the chain of links in two unconnected halves allows the serialization to succeed\, even though the total number of links has been reduced by only 1.

Perl Info

``` Flags: category=library severity=medium Site configuration information for perl v5.8.6: Configured by kynn at Tue Apr 12 12:24:11 EDT 2005. Summary of my perl5 (revision 5 version 8 subversion 6) configuration: Platform: osname=linux, osvers=2.4.18-686-smp, archname=i686-linux-ld uname='linux luna 2.4.18-686-smp #1 smp sun apr 14 12:07:19 est 2002 i686 unknown ' config_args='' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=define usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='2.95.4 20011002 (Debian prerelease)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='long double', nvsize=12, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lgdbm -ldbm -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=/lib/libc-2.2.5.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.2.5' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Locally applied patches: @INC for perl v5.8.6: /home/kynn/local/lib/perl/my /home/kynn/local/lib/perl/5.8.4 /home/kynn/local/lib/perl/5.6.1 /home/kynn/local/lib/perl5/5.8.6/i686-linux-ld /home/kynn/local/lib/perl5/5.8.6 /home/kynn/local/lib/perl5/i686-linux-ld /home/kynn/local/lib/perl5 /home/kynn/local/lib/perl5/5.8.6/i686-linux-ld /home/kynn/local/lib/perl5/5.8.6 /home/kynn/local/lib/perl5/site_perl/5.8.6/i686-linux-ld /home/kynn/local/lib/perl5/site_perl/5.8.6 /home/kynn/local/lib/perl5/site_perl . Environment for perl v5.8.6: HOME=/home/kynn LANG=C LANGUAGE (unset) LD_LIBRARY_PATH=/home/kynn/local/lib:/opt/java/jdk/lib/i386 LOGDIR (unset) PATH=.:/home/kynn/local/bin:/home/kynn/local/usr/bin:/home/kynn/local/local/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/home/kynn/java/javacc/bin:/opt/java/jdk/bin:/usr/pbs/bin:/home/kynn/opt/ant/bin PERL5LIB=/home/kynn/local/lib/perl/my:/home/kynn/local/lib/perl/5.8.4:/home/kynn/local/lib/perl/5.6.1:/home/kynn/local/lib/perl5 PERL5_CPANPLUS_CONFIG=/home/kynn/.cpanplus/config PERL_BADLANG (unset) SHELL=/usr/bin/zsh ```

[p5pRT]

From @iabyn

On Wed\, Jun 29\, 2005 at 09​:46​:33PM -0000\, kynn jones wrote​:

With an argument greater than some system-dependent critical size\, the script causes perl to segfault​:

use strict; use warnings;

my $size = shift;

my $x = []; $x = [ $x ] for 1..$size;

require Storable; Storable​::store( \$x\, '/tmp/$0.$$' ) or die;

It's the stack being trashed due to excessive recursion. No real way to fix it short of rewriting Storable to be iterative rather than recursive.

The workaround is to increase the stack size available to the process with 'uname -s'

-- In defeat\, indomitable; in victory\, insufferable   -- Churchill on Montgomery

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'