Moose fails in List::MoreUtils::all use-after-free

p5pRT commented 12 years ago

Migrated from rt.perl.org#115602 (status was 'resolved')

Searchable as RT115602$

p5pRT commented 12 years ago

From @rurban

This is a bug report for perl from rurban@cpanel.net\, generated with the help of perlbug 1.39 running under perl 5.17.6.

Moose triggers a List::MoreUtils::all refcnt error in POP_MULTICALL

gdb --args /usr/local/bin/perl5.17.6d-nt-asan@6b54ddc5 -Mblib t/metaclasses/metarole_w_metaclass_pm.t b __asan_report_error r ==12013== ERROR: AddressSanitizer heap-use-after-free on address 0x7ffff45178c0 at pc 0x7ffff2c3ecb7 bp 0x7fffffff7750 sp 0x7fffffff7748 READ of size 4 at 0x7ffff45178c0 thread T0 #0 0x7ffff2c3ecb7 #(/usr/local/lib/perl5/site_perl/5.17.6/x86_64-linux-debug-asan@6b54ddc5/auto/List/MoreUtils/MoreUtils.so+0x1ccb7) #1 0x7ffff6ffeeec #(/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@6b54ddc5/CORE/libperl.so+0xbf9eec) #2 0x7ffff6cb16a1 #(/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@6b54ddc5/CORE/libperl.so+0x8ac6a1) #3 0x7ffff6672305 #(/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@6b54ddc5/CORE/libperl.so+0x26d305) #4 0x7ffff666dd85 #(/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@6b54ddc5/CORE/libperl.so+0x268d85) #5 0x407020 (/usr/local/bin/perl5.17.6d-nt-asan@6b54ddc5+0x407020) #6 0x7ffff55c4ead (/lib/x86_64-linux-gnu/libc-2.13.so+0x1eead) 0x7ffff45178c0 is located 64 bytes inside of 1920-byte region [0x7ffff4517880\,0x7ffff4518000) freed by thread T0 here: previously allocated by thread T0 here: ==12013== ABORTING Stats: 29M malloced (55M for red zones) by 176798 calls Stats: 2M realloced by 19857 calls Stats: 13M freed by 122268 calls Stats: 0M really freed by 0 calls Stats: 108M (27660 full pages) mmaped in 27 calls mmaps by size class: 8:163830; 9:8191; 10:8190; 11:4094; 12:2048; 13:2048; 14:512; 15:256; 16:64; 17:32; mallocs by size class: 8:161326; 9:3816; 10:4283; 11:3281; 12:1621; 13:1883; 14:443; 15:140; 16:3; 17:2; frees by size class: 8:114900; 9:2287; 10:2282; 11:1626; 12:357; 13:530; 14:227; 15:57; 16:2; rfrees by size class: Stats: malloc large: 2 small slow: 679 Shadow byte and word: 0x1ffffe8a2f18: fd 0x1ffffe8a2f18: fd fd fd fd fd fd fd fd More shadow bytes: 0x1ffffe8a2ef8: fa fa fa fa fa fa fa fa 0x1ffffe8a2f00: fa fa fa fa fa fa fa fa 0x1ffffe8a2f08: fa fa fa fa fa fa fa fa 0x1ffffe8a2f10: fd fd fd fd fd fd fd fd =>0x1ffffe8a2f18: fd fd fd fd fd fd fd fd 0x1ffffe8a2f20: fd fd fd fd fd fd fd fd 0x1ffffe8a2f28: fd fd fd fd fd fd fd fd 0x1ffffe8a2f30: fd fd fd fd fd fd fd fd 0x1ffffe8a2f38: fd fd fd fd fd fd fd fd [Inferior 1 (process 12013) exited with code 01]

(gdb) l 270 GvSV(PL_defgv) = args[i]; 271 MULTICALL; 272 if (!SvTRUE(*PL_stack_sp)) { 273 POP_MULTICALL; 274 XSRETURN_NO; 275 } 276 } => 277 POP_MULTICALL; 278 XSRETURN_YES; 279 }

Maybe there's a FREETMPS missing in POP_MULTICALL

Flags: category=library severity=medium

This perlbug was built using Perl 5.17.3 - Mon Jul 30 16:28:27 CDT 2012 It is being executed now by Perl 5.17.6 - Fri Oct 26 14:23:20 CDT 2012.

Site configuration information for perl 5.17.6:

Configured by rurban at Fri Oct 26 14:23:20 CDT 2012.

Summary of my perl5 (revision 5 version 17 subversion 6) configuration: Commit id: 0db252d9bf45de9a19d214e875f71fa3f0597ce5 Platform: osname=linux\, osvers=3.2.0-2-amd64\, archname=x86_64-linux-debug-asan@6b54ddc5 uname='linux reini 3.2.0-2-amd64 #1 smp mon may 21 17:45:41 utc 2012 x86_64 gnulinux ' config_args='-de -Dusedevel -Uversiononly -Dinstallman1dir=none -Dinstallman3dir=none -Dinstallsiteman1dir=none -Dinstallsiteman3dir=none -DEBUGGING -Doptimize=-g3 -Uuseithreads -D'cc=clang' -D'ld=clang' -A'ccflags=-faddress-sanitizer' -Aldflags=-faddress-sanitizer -Alddlflags='-shared\ -faddress-sanitizer' -Duseshrplib -Dcf_email='rurban@cpanel.net' -Dperladmin='rurban@cpanel.net' -Duseshrplib -Accflags=-Wno-unused-value' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=define\, use64bitall=define\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='clang'\, ccflags ='-faddress-sanitizer -Wno-unused-value -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-g3'\, cppflags='-faddress-sanitizer -Wno-unused-value -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include' ccversion=''\, gccversion='4.2.1 Compatible Clang 3.2 (trunk)'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='clang'\, ldflags =' -faddress-sanitizer -L/usr/local/lib' libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@6b54ddc5/CORE' cccdlflags='-fPIC'\, lddlflags=' -shared -faddress-sanitizer -L/usr/local/lib '

Locally applied patches:

@INC for perl 5.17.6: /usr/local/lib/perl5/site_perl/5.17.6/x86_64-linux-debug-asan@6b54ddc5 /usr/local/lib/perl5/site_perl/5.17.6 /usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@6b54ddc5 /usr/local/lib/perl5/5.17.6 /usr/local/lib/perl5/site_perl/5.17.5 /usr/local/lib/perl5/site_perl/5.17.4 /usr/local/lib/perl5/site_perl/5.17.3 /usr/local/lib/perl5/site_perl/5.17.2 /usr/local/lib/perl5/site_perl/5.17.1 /usr/local/lib/perl5/site_perl/5.17.0 /usr/local/lib/perl5/site_perl/5.17 /usr/local/lib/perl5/site_perl/5.16.1 /usr/local/lib/perl5/site_perl/5.16.0 /usr/local/lib/perl5/site_perl/5.15.9 /usr/local/lib/perl5/site_perl/5.15.8 /usr/local/lib/perl5/site_perl/5.15.7 /usr/local/lib/perl5/site_perl/5.15.6 /usr/local/lib/perl5/site_perl/5.15.5 /usr/local/lib/perl5/site_perl/5.15.4 /usr/local/lib/perl5/site_perl/5.14.3 /usr/local/lib/perl5/site_perl/5.14.2 /usr/local/lib/perl5/site_perl/5.14.1 /usr/local/lib/perl5/site_perl/5.12.4 /usr/local/lib/perl5/site_perl/5.10.1 /usr/local/lib/perl5/site_perl/5.8.9 /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl/5.8.7 /usr/local/lib/perl5/site_perl/5.8.6 /usr/local/lib/perl5/site_perl/5.8.5 /usr/local/lib/perl5/site_perl/5.8.4 /usr/local/lib/perl5/site_perl/5.8.3 /usr/local/lib/perl5/site_perl/5.8.2 /usr/local/lib/perl5/site_perl/5.8.1 /usr/local/lib/perl5/site_perl/5.6.2 /usr/local/lib/perl5/site_perl .

Environment for perl 5.17.6: HOME=/home/rurban LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/rurban/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games PERL_BADLANG (unset) SHELL=/bin/bash

p5pRT commented 12 years ago

From @rurban

The cpan ticket with some more analysis is at: https://rt.cpan.org/Ticket/Display.html?id=77874

On 11/05/2012 11:24 AM\, perlbug-followup@perl.org wrote:

Greetings\,

This message has been automatically generated in response to the creation of a perl bug report regarding: "Moose fails in List::MoreUtils::all use-after-free".

There is no need to reply to this message right now. Your ticket has been assigned an ID of [perl #115602].

You can view your ticket at https://rt-archive.perl.org/perl5/Ticket/Display.html?id=115602

Within the next 24 to 72 hours\, your message will be posted to the Perl 5 Porters mailing list. Please be patient!

Please include the string:
\[perl \#115602\]
in the subject line of all future correspondence about this issue. To do so\, you may reply to this message (please delete unnecessary quotes and text.)

Thank you\, perlbug-followup@perl.org

------------------------------------------------------------------------- Received: (qmail 28525 invoked by uid 225); 5 Nov 2012 17:24:22 -0000 Received: (qmail 28521 invoked by alias); 5 Nov 2012 17:24:22 -0000 Received: from mx1.cpanel.net (HELO mx1.cpanel.net) (208.74.121.68) by la.mx.develooper.com (qpsmtpd/0.28) with ESMTP; Mon\, 05 Nov 2012 09:24:18 -0800 Received: from ng1.cptxoffice.net ([208.74.121.102]:12033 helo=reini) by mx1.cpanel.net with esmtps (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.80) (envelope-from \rurban@cpanel\.net) id 1TVQP9-000469-AE for perlbug@perl.org; Mon\, 05 Nov 2012 11:24:07 -0600 Received: from rurban by reini with local (Exim 4.80) (envelope-from \rurban@cpanel\.net) id 1TVQP6-0003u5-RM for perlbug@perl.org; Mon\, 05 Nov 2012 11:24:05 -0600 From rurban@cpanel.net Mon Nov 05 17:24:22 2012 Delivered-To: rt-perl5@rt.perl.org Delivered-To: perlbug@perl.org Subject: Moose fails in List::MoreUtils::all use-after-free X-Spam-Status: No\, hits=-8.2 required=8.0 tests=BAYES_00\,PERLBUG_CONF\,RCVD_IN_DNSWL_MED\,SPF_HELO_PASS\,SPF_PASS\,T_RP_MATCHES_RCVD Return-Path: \rurban@cpanel\.net X-Spam-Check-BY: la.mx.develooper.com Date: Mon\, 05 Nov 2012 11:24:04 -0600 X-Virus-Checked: Checked X-Get-Message-Sender-Via: mx1.cpanel.net: acl_c_relayhosts_text_entry: -unknown-@cpanel.net|cpanel.net Reply-To: rurban@cpanel.net Message-ID: \5\.17\.6\_12470\_1352135295@reini To: perlbug@perl.org X-Antiabuse: This header was added to track abuse\, please include it with any abuse report X-Antiabuse: Primary Hostname - mx1.cpanel.net X-Antiabuse: Original Domain - perl.org X-Antiabuse: Originator/Caller UID/GID - [47 12] / [47 12] X-Antiabuse: Sender Address Domain - cpanel.net From: rurban@cpanel.net X-RT-Original-Encoding: ascii content-type: text/plain; charset="utf-8"

-- Reini

Working towards a true Modern Perl. Slim\, functional\, unbloated\, compile-time optimizable

p5pRT commented 12 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 12 years ago

From @jkeenan

On Mon Nov 05 09:24:34 2012\, rurban@cpanel.net wrote:

This is a bug report for perl from rurban@cpanel.net\, generated with the help of perlbug 1.39 running under perl 5.17.6.

----------------------------------------------------------------- Moose triggers a List::MoreUtils::all refcnt error in POP_MULTICALL

Neither Moose nor List::MoreUtils is part of the Perl 5 core distribution.

This bug report would be better filed at https://rt.cpan.org/Dist/Display.html?Queue=List- MoreUtils.

p5pRT commented 12 years ago

From @rurban

On Mon\, Nov 5\, 2012 at 12:46 PM\, James E Keenan via RT \perlbug\-followup@perl\.org wrote:

On Mon Nov 05 09:24:34 2012\, rurban@cpanel.net wrote:

This is a bug report for perl from rurban@cpanel.net\, generated with the help of perlbug 1.39 running under perl 5.17.6.

----------------------------------------------------------------- Moose triggers a List::MoreUtils::all refcnt error in POP_MULTICALL

Neither Moose nor List::MoreUtils is part of the Perl 5 core distribution.

This bug report would be better filed at https://rt.cpan.org/Dist/Display.html?Queue=List-MoreUtils.

Sure\, but I tried this already in June\, List-MoreUtils seems to be unmaintained\, and due to the nature of the problem Moose and p5p are better targets. Even I fail to properly understand the MULTICALL failure.

The original cpan ticket with some more analysis is at: https://rt.cpan.org/Ticket/Display.html?id=77874 -- Reini Urban http://cpanel.net/ http://www.perl-compiler.org/

p5pRT commented 12 years ago

From @iabyn

On Mon\, Nov 05\, 2012 at 09:24:35AM -0800\, rurban@cpanel.net wrote:

0x7ffff45178c0 is located 64 bytes inside of 1920-byte region [0x7ffff4517880\,0x7ffff4518000) freed by thread T0 here:

270 GvSV(PL_defgv) = args[i]; 271 MULTICALL; 272 if (!SvTRUE(*PL_stack_sp)) { 273 POP_MULTICALL; 274 XSRETURN_NO; 275 } 276 } => 277 POP_MULTICALL; 278 XSRETURN_YES;

#define dMULTICALL \ SV **newsp; /* set by POPBLOCK */ \ PERL_CONTEXT *cx; \ ...

#define PUSH_MULTICALL_WITHDEPTH(the_cv\, depth) \ ... PUSHBLOCK(cx\, CXt_SUB|CXp_MULTICALL\, PL_stack_sp); \ ...

#define POP_MULTICALL \ STMT_START { \ if (! ((CvDEPTH(multicall_cv) = cx->blk_sub.olddepth)) ) { \ LEAVESUB(multicall_cv); \ } \ ...

Looks like MULTICALL expects cx to continue pointing to the current context frame\, which isn't true if the ctx stack gets extended and ralloced in the meantime.

The issue can be reproduced with just core modules:

use List::Util qw(first); sub rec { my $n = shift; rec($n-1) if $n; } @b = first { rec(1000); 1 } qw(1 2 3);

$ valgrind ./perl -Ilib /tmp/p ==4313== Memcheck\, a memory error detector ==4313== Copyright (C) 2002-2012\, and GNU GPL'd\, by Julian Seward et al. ==4313== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==4313== Command: ./perl -Ilib /tmp/p ==4313== ==4313== Invalid read of size 4 ==4313== at 0xB477214: XS_List__Util_first (ListUtil.xs:303) ==4313== by 0x5BF01C: Perl_pp_entersub (pp_hot.c:2770) ==4313== by 0x54AB40: Perl_runops_debug (dump.c:2146) ==4313== by 0x45B350: S_run_body (perl.c:2392) ==4313== by 0x45A429: perl_run (perl.c:2308) ==4313== by 0x41A8BC: main (perlmain.c:114) ==4313== Address 0x4cfe9f8 is 88 bytes inside a block of size 1\,944 free'd ==4313== at 0x4A08A0E: realloc (vg_replace_malloc.c:662) ==4313== by 0x54B928: Perl_safesysrealloc (util.c:194) ==4313== by 0x65E574: Perl_cxinc (scope.c:80) ==4313== by 0x5BE1C1: Perl_pp_entersub (pp_hot.c:2681) ==4313== by 0x54AB40: Perl_runops_debug (dump.c:2146) ==4313== by 0xB476B09: XS_List__Util_first (ListUtil.xs:301) ==4313== by 0x5BF01C: Perl_pp_entersub (pp_hot.c:2770) ==4313== by 0x54AB40: Perl_runops_debug (dump.c:2146) ==4313== by 0x45B350: S_run_body (perl.c:2392) ==4313== by 0x45A429: perl_run (perl.c:2308) ==4313== by 0x41A8BC: main (perlmain.c:114)

Looks like the fix is to store the context offset rather than a pointer. I'll do this sometime soon.

-- I've often wanted to drown my troubles\, but I can't get my wife to go swimming.

p5pRT commented 12 years ago

From @iabyn

On Tue\, Nov 06\, 2012 at 02:39:40PM +0000\, Dave Mitchell wrote:

Looks like MULTICALL expects cx to continue pointing to the current context frame\, which isn't true if the ctx stack gets extended and ralloced in the meantime.

Now fixed by

commit 3d26b81e83dca7175e314b31d265a01e1e9b0320 Author: David Mitchell \davem@iabyn\.com AuthorDate: Sun Nov 11 00:01:21 2012 +0000 Commit: David Mitchell \davem@iabyn\.com CommitDate: Sun Nov 11 00:01:21 2012 +0000

make MULTICALL safe across cxstack reallocs
[perl #115602] MUTLICALL sets a local var\, cx\, to point to the current context stack frame. When a function is called\, the context stack might be realloc()ed\, in which case cx would point to freed memory.

M cop.h M ext/XS-APItest/t/multicall.t

-- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.

p5pRT commented 12 years ago

@iabyn - Status changed from 'open' to 'resolved'

Perl / perl5