p5pRT
commented
10 years ago From @bulk88
Created by @bulk88
I decided to try something novel. Replacing the Perl stack\, which is a very-often-bounds-checked malloc block\, with the Win32 C stack (its not the interp's OS thread's stack\, but a thread stack without any thread). There are around 160 calls to Perl_stack_grow\, and often in hot opcodes like pp_undef\, pp_shift\, pp_padsv\, pp_padrange\, and pp_aelemfast. Plus we now have fancy microchips called MMUs and support for them in AT&T SVR4. My proposal\, replace the Perl stack (and maybe more stacks like mortal stack and save stack if this works) with a self allocating/expanding memory block\, no more runtime bounds checks+function call branches. EXTEND macro is a noop. The bounds check is a SEGV. In POSIX land\, I think this scheme would be a SIGSEGV handler that calls mprotect to turn reserved address space into R/W memory. Reserved address space is free. So I hacked up a POC which is attached. And most test files pass (the others crash). So far the only 2 remaining crash/panics problems to resolve is
-address space exhaustion on 32 bits\, op/threads.t trys to launch 100 threads simultaniously\, and Module::CoreList causes huge C stack recursion in what I guess is deeply nested tied hashes\, and each recursion of magic creates a new perl stack -pp_split puts a AvREAL AV * into PL_curstack/PL_stack_sp for unknown to me reason\, then XPUSHs (EXTEND is a noop) onto faux-stack causing heap corruption
There are some API design problems of how get the code integrated in a sane way into blead.
We have SvLEN == 0 for foreign PV buffers. But what is the equivalent for AVs?
Should PERL_FLEXIBLE_EXCEPTIONS come back since I'm use MSC's
__try/__catch?
http://perl5.git.perl.org/perl.git/commitdiff/766f891612bf493b0430beb068ead367d70cdef6
http://www.nntp.perl.org/group/perl.perl5.porters/2004/07/msg93066.html
In pp_mapwhile there was code starting at a very high address in the Perl stack and wrote backwards towards lower addresses. This broke/crashed since "guard pages" must be sequentially accesses\, Win32 alloca/_chkstk (see http://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libgcc/config/i386/cygwin.S#l53 )/C99 non-const length array declarations have to call alloca/_chkstk to touch each page in sequence every 4096 bytes to allocate the block and move the guard/trap page down. IDK if this is necessary since no perl macros write backwards/lowering addresses to the perl stack and the couple rare times its found (like in pp_mapwhile) can be fixed by hand. The other choice\, I dont like\, is for the EXTEND macro to be
# define EXTEND(p\,n) (void)(UNLIKELY( (4096/sizeof(SV*)) \< (SSize_t)(n)) && \ (sp = stack_grow(sp\,p\, (SSize_t) (n))))
which will constant fold away by C compiler for all const "n"s but won't optimize away for non-const "n"s.
Currently I set the maximum VM size of Perl stack to 32 MB\, which is
( 62000000[maximum # of SVs I could push() into a @array in PP before OOM] /8[reasonable largest list processing with enough src array\, dest array\, and perl stack space to process the data set] )*4[bytes per SV*]
For comparison\, the Win32 C stack's default (and all Win32 Perls ever) maximum size is fixed at 1MB. After 1 MB it stops self-allocating and just crashes. There is a Win32 C compiler option to set it as high as you want but I know Perl and I dont think any other apps ever touch the setting.
How Windows/ReactOS self-allocates a thread's C stack http://doxygen.reactos.org/d5/d43/pagfault_8c_source.html#l00029 .
The rest of this the result of a "nmake test" on self allocating stack perl and my notes on each test failure. If there is no note above the test name\, it means I didn't investigate the failure.
--------------------------------------------------------------- x2p/s2p.t ......................................................... ok
Test Summary Report -------------------
\|/Heap corruption\, pp_split swaps Perl stack (PL_stack_sp) with random AV op/push.t (Wstat: 1280 Tes ts: 60 Failed: 0) Non-zero exit status: 5
\|/pp_split+Error #6: UNADDRESSABLE ACCESS: writing 0x1147cb28-0x1147cb2c 4 byte(s) # 0 perl520.dll!Perl_av_clear [c:\perl519\src\av.c:481] # 1 perl520.dll!Perl_pp_split [c:\perl519\src\pp.c:5473] # 2 perl520.dll!Perl_runops_debug [c:\perl519\src\dump.c:2459] # 3 perl520.dll!S_run_body [c:\perl519\src\win32\perl.c:2459] # 4 perl520.dll!perl_run [c:\perl519\src\win32\perl.c:2375] # 5 perl520.dll!RunPerl [c:\perl519\src\win32\perllib.c:258] # 6 main [c:\perl519\src\win32\perlmain.c:23] op/split.t (Wstat: 1280 Tes ts: 0 Failed: 0) Non-zero exit status: 5 Parse errors: No plan found in TAP output
\|/ creates 100 threads in 1 process\, address space exhaustion op/threads.t (Wstat: 256 Test s: 9 Failed: 0) Non-zero exit status: 1 Parse errors: Bad plan. You planned 27 tests but ran 9.
\|/Heap corruption\, pp_split swaps Perl stack (PL_stack_sp) with random AV ../cpan/Text-Tabs/t/sep.t (Wstat: 1280 Tes ts: 25 Failed: 0) Non-zero exit status: 5 ../cpan/Text-Tabs/t/sep2.t (Wstat: 1280 Tes ts: 25 Failed: 0) Non-zero exit status: 5 ../cpan/Text-Tabs/t/tabs.t (Wstat: 2304 Tes ts: 11 Failed: 0) Non-zero exit status: 9 ../cpan/Text-Tabs/t/wrap.t (Wstat: 1280 Tes ts: 30 Failed: 0) Non-zero exit status: 5
\|/Address space exhaustion at
http://perl5.git.perl.org/perl.git/blob/90b0b922bfc0b803c3769f65fbf086e6eab214fd:/dist/Module-CoreList/lib/Module/CoreList/TieHashDelta.pm#l35
ntdll.dll!_DbgBreakPoint@0()
perl520.dll!Perl_new_stackinfo(interpreter * my_perl=0x003641e4\,
long stitems=32\, long cxitems=41) Line 78 C
perl520.dll!Perl_magic_methcall(interpreter * my_perl=0x003641e4\,
sv * sv=0x08acfa24\, const magic * mg=0x08ae75e4\, sv * meth=0x028fe44c\,
unsigned long flags=0\, unsigned long argc=1\, ...) Line 1781 + 0x76 C
perl520.dll!S_magic_methcall1(interpreter * my_perl=0x003641e4\, sv
* sv=0x08acfa24\, const magic * mg=0x08ae75e4\, sv * meth=0x028fe44c\,
unsigned long flags=0\, int n=1\, sv * val=0x00000000) Line 1841 + 0x25 C
perl520.dll!S_magic_methpack(interpreter * my_perl=0x003641e4\, sv *
sv=0x08acfa24\, const magic * mg=0x08ae75e4\, sv * meth=0x028fe44c) Line
1852 + 0x1b C
perl520.dll!Perl_magic_existspack(interpreter * my_perl=0x003641e4\,
sv * sv=0x08acfa24\, const magic * mg=0x08ae75e4) Line 1960 + 0x57 C
perl520.dll!Perl_hv_common(interpreter * my_perl=0x003641e4\, hv *
hv=0x08bf0b24\, sv * keysv=0x08acfa04\, const char * key=0x089ff4b4\,
unsigned int klen=8\, int flags=0\, int action=8\, sv * val=0x00000000\,
unsigned long hash=0) Line 501 + 0x20 C
perl520.dll!Perl_pp_exists(interpreter * my_perl=0x003641e4) Line
4725 + 0x1d C
perl520.dll!Perl_runops_debug(interpreter * my_perl=0x003641e4)
Line 2459 + 0xd C
perl520.dll!Perl_call_sv(interpreter * my_perl=0x003641e4\, sv *
sv=0x028fe44c\, volatile long flags=4098) Line 2759 + 0x36 C
perl520.dll!Perl_magic_methcall(interpreter * my_perl=0x003641e4\,
sv * sv=0x08acfac4\, const magic * mg=0x08ae7654\, sv * meth=0x028fe44c\,
unsigned long flags=0\, unsigned long argc=0\, ...) Line 1806 + 0x12 C
perl520.dll!S_magic_methcall1(interpreter * my_perl=0x003641e4\, sv
* sv=0x08acfac4\, const magic * mg=0x08ae7654\, sv * meth=0x028fe44c\,
unsigned long flags=0\, int n=1\, sv * val=0x00000000) Line 1841 + 0x25 C
perl520.dll!S_magic_methpack(interpreter * my_perl=0x003641e4\, sv *
sv=0x08acfac4\, const magic * mg=0x08ae7654\, sv * meth=0x028fe44c) Line
1852 + 0x1b C
perl520.dll!Perl_magic_existspack(interpreter * my_perl=0x003641e4\,
sv * sv=0x08acfac4\, const magic * mg=0x08ae7654) Line 1960 + 0x57 C
perl520.dll!Perl_hv_common(interpreter * my_perl=0x003641e4\, hv *
hv=0x08bf3294\, sv * keysv=0x08acfaa4\, const char * key=0x089ff584\,
unsigned int klen=8\, int flags=0\, int action=8\, sv * val=0x00000000\,
unsigned long hash=0) Line 501 + 0x20 C
perl520.dll!Perl_pp_exists(interpreter * my_perl=0x003641e4) Line
4725 + 0x1d C
perl520.dll!Perl_runops_debug(interpreter * my_perl=0x003641e4)
Line 2459 + 0xd C
perl520.dll!Perl_call_sv(interpreter * my_perl=0x003641e4\, sv *
sv=0x028fe44c\, volatile long flags=4098) Line 2759 + 0x36 C
******cut see attachment for entire callstack*****
../dist/Module-CoreList/t/corelist.t (Wstat:
256 Test
s: 13 Failed: 0)
Non-zero exit status: 1
Parse errors: Bad plan. You planned 25 tests but ran 13.
\|/Same as above\, address space exhaustion ../dist/Module-CoreList/t/deprecated.t (Wstat: 256 Test s: 3 Failed: 0) Non-zero exit status: 1 Parse errors: Bad plan. You planned 9 tests but ran 3. ../dist/Module-CoreList/t/is_core.t (Wstat: 256 Test s: 2 Failed: 0) Non-zero exit status: 1 Parse errors: Bad plan. You planned 34 tests but ran 2. ../dist/Module-CoreList/t/utils.t (Wstat: 256 Test s: 2 Failed: 0) Non-zero exit status: 1 Parse errors: Bad plan. You planned 9 tests but ran 2. \|/Heap corruption\, pp_split swaps Perl stack (PL_stack_sp) with random AV ../lib/Tie/Array/stdpush.t (Wstat: 1280 Tes ts: 60 Failed: 0) Non-zero exit status: 5 Files=2409\, Tests=691333\, 3367 wallclock secs (146.34 usr + 4.20 sys = 150.55 C PU) Result: FAIL NMAKE : fatal error U1077: '..\perl.exe' : return code '0x6' Stop.
C:\perl519\src\win32> -------------------------------------------------------------------
toddr
Migrated from rt.perl.org#121923 (status was 'open')
Searchable as RT121923$