p5pRT
commented
10 years ago From prumpf@gmail.com
Created by prumpf@gmail.com
This is a bug report for perl from prumpf@gmail.com\, generated with the help of perlbug 1.40 running under perl 5.19.11.
-----------------------------------------------------------------
Hi! I've run into a deadlock situation with the current git versions of perl (5.19.11) and glibc (2.19)\, on x86_64-pc-linux-gnu with ithreads and MY_MALLOC\, though I've run into it with other setups (recent Debian versions of Perl and glibc\, no MY_MALLOC) as well. I believe I've been able to track down the issue and come up with a workaround\, although I've not yet found the time to come up with a small reproducible test case. Please feel free to ask me for one if it's absolutely required\, though\, or ask for other information\, and I'll do my best.
In summary\, the problem is inconsistent lock ordering between Perl's PL_malloc_mutex and glibc's malloc/arena.c's list_lock. The situation arises when one thread tries to fork() at the same time that another thread calls malloc().
Perl runs pthread_atfork before the first malloc() makes glibc install its atfork handlers\, so fork() calls ptmalloc_lock_all() first\, then Perl_atfork_lock(). That means locking glibc's list_lock first\, then PL_malloc_mutex. (pthread_atfork() has LIFO semantics)
However\, Perl's malloc implementation locks PL_malloc_mutex first\, then (sometimes) runs out of memory and calls the real malloc()\, which tries to lock list_lock. We thus have a race condition and a deadlock\, which I've seen in practice.
I believe this is fundamentally a glibc bug: its implementation of pthread_atfork() behaves erratically depending on whether malloc() is first called before or after pthread_atfork(). However\, since the broken versions of glibc are out there and multiplying\, we should also work around the issue in Perl itself.
The workaround should be as easy as including an extra PerlMem_free(PerlMem_malloc(1024)) call before calling PTHREAD_ATFORK\, but gcc has started "optimizing" such (otherwise) useless calls. I've found a deliberately duplicate call to perl_alloc() works\, but that's both a one-time memory leak and horribly ugly\, and most likely breaks whatever code uses PL_do_undump.
Nevertheless\, I'll include it here\, because most of the work was probably in tracking down the bug\, and fixing it should be easier\, even if I cannot presently think of a good fix.
diff --git a/ext/ExtUtils-Miniperl/lib/ExtUtils/Miniperl.pm b/ext/ExtUtils-Miniperl/lib/ExtUtils/Miniperl.pm index 730c565..a8092bf 100644 --- a/ext/ExtUtils-Miniperl/lib/ExtUtils/Miniperl.pm +++ b/ext/ExtUtils-Miniperl/lib/ExtUtils/Miniperl.pm @@ -129\,6 +129\,19 @@ main(int argc\, char **argv\, char **env) * call PTHREAD_ATFORK() explicitly\, but if and only if it hasn't * been called at least once before in the current process. * --GSAR 2001-07-20 */ + /* There's a nasty race condition with the current versions of Perl and + * glibc: the call to PTHREAD_ATFORK in Perl's main() might be reached + * before the first malloc happens\, in which + * case fork() locks malloc/arena.c's list_lock first\, then tries to lock + * PL_malloc_lock; another thread might have locked PL_malloc_lock first\, + * then tries to lock list_lock\, resulting in a deadlock. + * + * A proper fix would be in glibc\, ensuring that ptmalloc_init() is called + * earlier\, but a workaround is to make a malloc call ourselves. */ + /* This leaks memory\, but works. */ + (void)perl_alloc(); + /* This doesn't leak memory\, but is optimized away by gcc */ + PerlMem_free(PerlMem_malloc(1024)); PTHREAD_ATFORK(Perl_atfork_lock\, Perl_atfork_unlock\, Perl_atfork_unlock);
Migrated from rt.perl.org#121490 (status was 'open')
Searchable as RT121490$