p5pRT
commented
12 years ago From @bulk88
Created by @bulk88
Perl lexer (???) has a deep recursion bug in Perl_scalarvoid when given a very large Perl subroutine to run. Perl_scalarvoid calls itself over and over\, for what I blindly guess is every opcode in the sub. If there are too many opcodes\, C stack overflows.
I could not replicate this with any stock ActivePerl (AP 5.10.0\, AP 5.12.3). While the test script is running\, ActivePerls peak at 8 MB on the C stack. I checked with VMMAP. If I lower the stack reserve from 16 MB to 256KB in the PE header on perl.exe from ActivePerls\, I will get the same exact Perl_scalarvoid stack overflow. There is no reason for the interpreter to peak at many MBs of C stack\, which is irreversibly expanded (there are OS specific ways to shrink C stacks.... I know it can be done with VirtualFree and VirtualAlloc on Windows\, but that is the wrong way to fix C stack recursion) and wasted\, never to be used again after BEGIN blocks run. I have not tested doing the "require" at runtime. On other perl scripts and apps I run\, all the interpreter (APs and my compiled Perl) peak at 20-50KB of C stack which is fine. I assume that the reason that my self-compiled Perl will stack overflow at 16MB and ActivePerls do not is because of high C stack overhead of each recursive Perl_scalarvoid when Perl is compiled with -Od (no registers) meanwhile ActivePerls are compiled with -O1 (small code).
I wrote this test script to test how efficient Perl is at returning memory to the OS after a Perl Module is "use"d and if\, and how well\, memory behind the BEGIN blocks (SVs\, CVs\, Opcodes\, pads\, raw malloced blocks) is returned to the OS. If I lower the 100000 to 50000 in the test script\, the script will run without a stack overflow. On sucessful runs\, malloced memory went down from ~60MB inside the module BEGIN block (at "print "after branches\n";") to ~3MB at runtime ("print "in main\n";" ) according to VMMAP (private/yellow memory). C Stack remained at many MBs (7 to 8MB at 50000 in test script).
Perl_scalarvoid\, whatever it does to opcodes\, should run in a loop down the optree\, not recursively on every opcode and thus eating away the C stack. I request that the above-reported bug be fixed.
Migrated from rt.perl.org#108276 (status was 'open')
Searchable as RT108276$