Closed p6rt closed 8 years ago
This probably is an issue with moarvm.
When trying to benchmark the concurrent and non-concurrent versions of Damien Conway's 'bogosort' algorithm, I ran into numerous problems with Rakudo Star 2016.07 - depending on the MAX_RAKUDO_THREADS setting, the process might succeed, or it might slowly eat up all of memory, and if more than a few threads were used only some would actually process anything (accumulate CPU time). rakudo-star-2016.10-RC0 looks like it solved the deadlock problem, so that all threads are actually working.
However, now this version of the program crashes after the first thread completes with: Internal error: zeroed target thread ID in work pass
or occasionally: Internal error: invalid thread ID 8770096 in GC work pass
Test runs:
doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 1 seconds. Internal error: invalid thread ID 8770096 in GC work pass doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 4 seconds. Internal error: zeroed target thread ID in work pass doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 24 seconds. Internal error: zeroed target thread ID in work pass doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 6 seconds. Internal error: zeroed target thread ID in work pass
I also found that the problem is likely related to a channel not being explicitly closed, so I was able to get normal test runs as follows:
[ With a LEAVE block to close the channels ]
doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 4 seconds. [e i l p r s x] 29 seconds. [e i l p r s x] 57 seconds. [e i l p r s x] 58 seconds. doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 4 seconds. [e i l p r s x] 10 seconds. [e i l p r s x] 20 seconds. [e i l p r s x] 31 seconds.
I'm figuring that failing to close the channels is bad practice to start, but we don't really want to crash the VM with an obscure error message. Also, I saw a bug report earlier this year where this message was produced, but closed because of uncertainty in reproducing it. This test, however fails quite repeatably, at least on my system (linux).
Code file is attached. System report follows.
Doug Schrag \<>\< aka dmaestro Houston, TX Traverse City, MI
Summary of my perl5 (revision 5 version 18 subversion 1) configuration:
Platform: osname=linux, osvers=3.10.9, archname=x86_64-linux-thread-multi uname='linux hive64 3.10.9 #1 smp wed aug 21 14:50:51 cdt 2013 x86_64 amd phenom(tm) ii x6 1100t processor authenticamd gnulinux ' config_args='-de -Dprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Darchlib=/usr/lib64/perl5 -Dvendorprefix=/usr -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Dvendorarch=/usr/lib64/perl5/vendor_perl -Dscriptdir=/usr/bin -Dcccdlflags=-fPIC -Dinstallprefix=/usr -Dlibpth=/usr/local/lib64 /usr/lib64 /lib64 -Doptimize=-O2 -fPIC -Dusethreads -Duseithreads -Duseshrplib -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Darchname=x86_64-linux' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -fPIC', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.8.1', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector' libpth=/usr/local/lib64 /usr/lib64 /lib64 libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=/lib64/libc-2.17.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.17' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib64/perl5/CORE' cccdlflags='-fPIC', lddlflags='-shared -O2 -fPIC -fstack-protector'
Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_PRESERVE_IVUV PERL_SAWAMPERSAND USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API Built under linux Compiled at Sep 2 2013 18:11:30 %ENV: PERL5LIB="/home/doug/perl5/lib/perl5" PERLBREW_BASHRC_VERSION="0.72" PERLBREW_HOME="/home/doug/.perlbrew" PERLBREW_ROOT="/home/doug/perl5/perlbrew" PERL_MB_OPT="--install_base "/home/doug/perl5"" PERL_MM_OPT="INSTALL_BASE=/home/doug/perl5" PERL_STRICTURES_EXTRA="" @INC: /home/doug/perl5/lib/perl5/x86_64-linux-thread-multi /home/doug/perl5/lib/perl5 /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .
use v6;
sub bogosort_concurrent ( *@list ) { my $sorted = Channel.new;
start until $sorted.closed { start { my @guess = @list.pick(*);
$sorted.send( @guess ) if [!after] @guess; } }
return $sorted.receive; # Using the LEAVE block prevents the error # LEAVE { $sorted.close } }
my @test_data = \< p e r l s i x >;
for ^4 { say bogosort_concurrent(@test_data); say time - INIT { time }, ' seconds.'; }
Sorry, wrong system report. Here is the correct one (perl6 interpreter):
moar::dcbrule=@: moar::ar=ar moar::mastdir=/usr/local/opt/rakudo-star-2016.10-RC0/share/nqp/lib/MAST moar::tomrule=$(AR) $(ARFLAGS) $@ 3rdparty/libtommath/*.o moar::syslibs[2]=rt moar::shaobjects=3rdparty/sha1/sha1.o moar::ccshared=-fPIC moar::be=0 moar::dynasmlua=./3rdparty/dynasm/dynasm.lua moar::ld=gcc moar::mtlib=3rdparty/tinymt/libtinymt.a moar::mtobjects=3rdparty/tinymt/tinymt64.o moar::noreturnspecifier= moar::perl=/usr/bin/perl5.18.1 moar::shaclean=$(RM) 3rdparty/sha1/libsha1.a 3rdparty/sha1/*.o moar::static_inline=static __inline__ moar::ccdebugflags=-g3 moar::auxclean=@: moar::thirdpartylibs=3rdparty/dyncall/dyncall/libdyncall_s.a 3rdparty/dyncall/dyncallback/libdyncallback_s.a 3rdparty/dyncall/dynload/libdynload_s.a 3rdparty/libatomic_ops/src/libatomic_ops.a 3rdparty/tinymt/libtinymt.a 3rdparty/sha1/libsha1.a 3rdparty/libtommath/libtommath.a 3rdparty/libuv/libuv.a moar::config=--optimize --prefix=/usr/local/opt/rakudo-star-2016.10-RC0 --make-install moar::ccinc=-I moar::booltype=_Bool moar::ldout=-o moar::moarlib=libmoar.a moar::mtrule=$(AR) $(ARFLAGS) $@ 3rdparty/tinymt/*.o moar::syslibs[0]=m moar::dcrule=cd 3rdparty/dyncall && ./configure && CC='$(CC)' CFLAGS='$(CFLAGS)' $(MAKE) -f Makefile moar::dcbobjects= moar::cflags=-Wdeclaration-after-statement -Werror=declaration-after-statement -O3 -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -fPIC moar::laolib=3rdparty/libatomic_ops/src/libatomic_ops.a moar::libdir=/usr/local/opt/rakudo-star-2016.10-RC0/lib moar::nul=/dev/null moar::osvers=3.10.9 moar::translate_newline_output=0 moar::dlrule=@: moar::cat=cat moar::can_unaligned_int32=1 moar::ldimp= moar::pkgconfig=/usr/bin/pkg-config moar::ccdef=-D moar::lib=lib%s.a moar::tomclean=$(RM) 3rdparty/libtommath/libtommath.a 3rdparty/libtommath/*.o moar::dlllocal=__attribute__ ((visibility ("hidden"))) moar::dcbclean=$(RM) 3rdparty/dyncall/dyncallback/libdyncallback_s.a moar::noreturnattribute=__attribute__((noreturn)) moar::ptr_size=8 moar::dllimport=__attribute__ ((visibility ("default"))) moar::ccout=-o moar::can_unaligned_num64=1 moar::has_pthread_yield=1 moar::ldrpath=-Wl,-rpath,"//usr/local/opt/rakudo-star-2016.10-RC0/lib" -Wl,-rpath,"/usr/local/opt/rakudo-star-2016.10-RC0/share/perl6/site/lib" moar::ldusr=-l%s moar::mainflags=-DMVM_SHARED moar::name=moar moar::dll=lib%s.so moar::cincludes= -I3rdparty/libuv/include -I3rdparty/libuv/src -I3rdparty/libatomic_ops/src -I3rdparty/libtommath -I3rdparty/dynasm -I3rdparty/dyncall/dynload -I3rdparty/dyncall/dyncall -I3rdparty/dyncall/dyncallback moar::uvrule=$(AR) $(ARFLAGS) $@ $(UV_LINUX) moar::tomlib=3rdparty/libtommath/libtommath.a moar::syslibs[3]=dl moar::staticlib= moar::dlclean=$(RM) 3rdparty/dyncall/dynload/libdynload_s.a moar::make=make moar::ccinstflags=-pg moar::ldflags= -O3 -DNDEBUG -Wl,-rpath,"//usr/local/opt/rakudo-star-2016.10-RC0/lib" -Wl,-rpath,"/usr/local/opt/rakudo-star-2016.10-RC0/share/perl6/site/lib" moar::crossconf= moar::obj=.o moar::dlobjects= moar::defs[0]=_REENTRANT moar::cppout=> moar::ccoptiflags=-O3 -DNDEBUG moar::asmswitch=-S moar::ldoptiflags=-O3 -DNDEBUG moar::mkflags= moar::moar=libmoar.so moar::version=2016.10 moar::dcclean=cd 3rdparty/dyncall && $(MAKE) -f Makefile clean moar::moarshared= moar::sharule=$(AR) $(ARFLAGS) $@ 3rdparty/sha1/*.o moar::ccwarnflags= moar::jit=$(JIT_POSIX_X64) moar::moardll=libmoar.so moar::pkgconfig_works=1 moar::asm=.s moar::mknoisy=ifneq ($(NOISY), 1)MSG = @echoCMD = @NOOUT = > /dev/nullNOERR = 2> /dev/nullendif moar::dclib=3rdparty/dyncall/dyncall/libdyncall_s.a moar::ccdefflags=-D_REENTRANT -D_FILE_OFFSET_BITS=64 moar::cc=gcc moar::exe= moar::lddir=-L moar::uvlib=3rdparty/libuv/libuv.a moar::dllib=3rdparty/dyncall/dynload/libdynload_s.a moar::shaincludedir=3rdparty/sha1 moar::versionmajor=2016 moar::cppswitch=-E moar::can_unaligned_int64=1 moar::lua=./3rdparty/dynasm/minilua moar::dcobjects= moar::cancgoto=1 moar::ldlibs=-lm -lpthread -lrt -ldl moar::mainlibs=-L. -lmoar moar::nativecall_backend=dyncall moar::objflags=-DMVM_BUILD_SHARED -fPIC moar::versionpatch=0 moar::laoclean=cd 3rdparty/libatomic_ops/src && $(MAKE) distclean moar::versionminor=10 moar::ccmiscflags=-Wdeclaration-after-statement -Werror=declaration-after-statement moar::install= $(MKPATH) $(DESTDIR)$(PREFIX)/include/libuv $(CP) 3rdparty/libuv/include/*.h $(DESTDIR)$(PREFIX)/include/libuv $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/armcc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/gcc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/hpc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/ibmc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/icc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/loadstore $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/msftc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/sunc $(CP) 3rdparty/libatomic_ops/src/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops $(CP) 3rdparty/libatomic_ops/src/atomic_ops/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/armcc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/armcc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/gcc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/gcc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/hpc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/hpc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/ibmc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/ibmc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/icc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/icc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/loadstore/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/loadstore $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/msftc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/msftc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/sunc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/sunc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libtommath $(CP) 3rdparty/libtommath/*.h $(DESTDIR)$(PREFIX)/include/libtommath $(MKPATH) $(DESTDIR)$(PREFIX)/include/dyncall $(CP) 3rdparty/dyncall/dynload/*.h $(DESTDIR)$(PREFIX)/include/dyncall $(CP) 3rdparty/dyncall/dyncall/*.h $(DESTDIR)$(PREFIX)/include/dyncall $(CP) 3rdparty/dyncall/dyncallback/*.h $(DESTDIR)$(PREFIX)/include/dyncall moar::laoobjects= moar::mtclean=$(RM) 3rdparty/tinymt/libtinymt.a 3rdparty/tinymt/*.o moar::rm=rm -f moar::uvclean=$(RM) 3rdparty/libuv/libuv.a $(UV_LINUX) moar::sharedlib= moar::dllexport=__attribute__ ((visibility ("default"))) moar::shalib=3rdparty/sha1/libsha1.a moar::arout= moar::lddebugflags=-g3 moar::osname=linux moar::prefix=/usr/local/opt/rakudo-star-2016.10-RC0 moar::uvobjects=$(UV_LINUX) moar::dcblib=3rdparty/dyncall/dyncallback/libdyncallback_s.a moar::bindir=/usr/local/opt/rakudo-star-2016.10-RC0/bin moar::asmout=-o moar::havebooltype=1 moar::laorule=cd 3rdparty/libatomic_ops && CC='$(CC)' CFLAGS='$(CFLAGS)' ./configure && cd src && $(MAKE) && cd .. moar::os=linux moar::sh=/bin/sh moar::arflags=rcs moar::ldshared=-shared -fPIC moar::defs[1]=_FILE_OFFSET_BITS=64 moar::impinst= moar::ldinstflags=-pg moar::ldmiscflags= moar::ldsys=-l%s moar::platform=$(PLATFORM_POSIX) moar::ccswitch=-c moar::formatattribute=__attribute__((format(X, Y, Z))) moar::tomobjects=3rdparty/libtommath/bn_error.o 3rdparty/libtommath/bn_fast_mp_invmod.o 3rdparty/libtommath/bn_fast_mp_montgomery_reduce.o 3rdparty/libtommath/bn_fast_s_mp_mul_digs.o 3rdparty/libtommath/bn_fast_s_mp_mul_high_digs.o 3rdparty/libtommath/bn_fast_s_mp_sqr.o 3rdparty/libtommath/bn_mp_2expt.o 3rdparty/libtommath/bn_mp_abs.o 3rdparty/libtommath/bn_mp_add.o 3rdparty/libtommath/bn_mp_add_d.o 3rdparty/libtommath/bn_mp_addmod.o 3rdparty/libtommath/bn_mp_and.o 3rdparty/libtommath/bn_mp_clamp.o 3rdparty/libtommath/bn_mp_clear.o 3rdparty/libtommath/bn_mp_clear_multi.o 3rdparty/libtommath/bn_mp_cmp.o 3rdparty/libtommath/bn_mp_cmp_d.o 3rdparty/libtommath/bn_mp_cmp_mag.o 3rdparty/libtommath/bn_mp_cnt_lsb.o 3rdparty/libtommath/bn_mp_copy.o 3rdparty/libtommath/bn_mp_count_bits.o 3rdparty/libtommath/bn_mp_div.o 3rdparty/libtommath/bn_mp_div_2.o 3rdparty/libtommath/bn_mp_div_2d.o 3rdparty/libtommath/bn_mp_div_3.o 3rdparty/libtommath/bn_mp_div_d.o 3rdparty/libtommath/bn_mp_dr_is_modulus.o 3rdparty/libtommath/bn_mp_dr_reduce.o 3rdparty/libtommath/bn_mp_dr_setup.o 3rdparty/libtommath/bn_mp_exch.o 3rdparty/libtommath/bn_mp_expt_d.o 3rdparty/libtommath/bn_mp_exptmod.o 3rdparty/libtommath/bn_mp_exptmod_fast.o 3rdparty/libtommath/bn_mp_exteuclid.o 3rdparty/libtommath/bn_mp_fread.o 3rdparty/libtommath/bn_mp_fwrite.o 3rdparty/libtommath/bn_mp_gcd.o 3rdparty/libtommath/bn_mp_get_int.o 3rdparty/libtommath/bn_mp_get_long.o 3rdparty/libtommath/bn_mp_grow.o 3rdparty/libtommath/bn_mp_init.o 3rdparty/libtommath/bn_mp_init_copy.o 3rdparty/libtommath/bn_mp_init_multi.o 3rdparty/libtommath/bn_mp_init_set.o 3rdparty/libtommath/bn_mp_init_set_int.o 3rdparty/libtommath/bn_mp_init_size.o 3rdparty/libtommath/bn_mp_invmod.o 3rdparty/libtommath/bn_mp_invmod_slow.o 3rdparty/libtommath/bn_mp_is_square.o 3rdparty/libtommath/bn_mp_jacobi.o 3rdparty/libtommath/bn_mp_karatsuba_mul.o 3rdparty/libtommath/bn_mp_karatsuba_sqr.o 3rdparty/libtommath/bn_mp_lcm.o 3rdparty/libtommath/bn_mp_lshd.o 3rdparty/libtommath/bn_mp_mod.o 3rdparty/libtommath/bn_mp_mod_2d.o 3rdparty/libtommath/bn_mp_mod_d.o 3rdparty/libtommath/bn_mp_montgomery_calc_normalization.o 3rdparty/libtommath/bn_mp_montgomery_reduce.o 3rdparty/libtommath/bn_mp_montgomery_setup.o 3rdparty/libtommath/bn_mp_mul.o 3rdparty/libtommath/bn_mp_mul_2.o 3rdparty/libtommath/bn_mp_mul_2d.o 3rdparty/libtommath/bn_mp_mul_d.o 3rdparty/libtommath/bn_mp_mulmod.o 3rdparty/libtommath/bn_mp_n_root.o 3rdparty/libtommath/bn_mp_neg.o 3rdparty/libtommath/bn_mp_or.o 3rdparty/libtommath/bn_mp_prime_fermat.o 3rdparty/libtommath/bn_mp_prime_is_divisible.o 3rdparty/libtommath/bn_mp_prime_is_prime.o 3rdparty/libtommath/bn_mp_prime_miller_rabin.o 3rdparty/libtommath/bn_mp_prime_next_prime.o 3rdparty/libtommath/bn_mp_prime_rabin_miller_trials.o 3rdparty/libtommath/bn_mp_prime_random_ex.o 3rdparty/libtommath/bn_mp_radix_size.o 3rdparty/libtommath/bn_mp_radix_smap.o 3rdparty/libtommath/bn_mp_rand.o 3rdparty/libtommath/bn_mp_read_radix.o 3rdparty/libtommath/bn_mp_read_signed_bin.o 3rdparty/libtommath/bn_mp_read_unsigned_bin.o 3rdparty/libtommath/bn_mp_reduce.o 3rdparty/libtommath/bn_mp_reduce_2k.o 3rdparty/libtommath/bn_mp_reduce_2k_l.o 3rdparty/libtommath/bn_mp_reduce_2k_setup.o 3rdparty/libtommath/bn_mp_reduce_2k_setup_l.o 3rdparty/libtommath/bn_mp_reduce_is_2k.o 3rdparty/libtommath/bn_mp_reduce_is_2k_l.o 3rdparty/libtommath/bn_mp_reduce_setup.o 3rdparty/libtommath/bn_mp_rshd.o 3rdparty/libtommath/bn_mp_set.o 3rdparty/libtommath/bn_mp_set_int.o 3rdparty/libtommath/bn_mp_set_long.o 3rdparty/libtommath/bn_mp_shrink.o 3rdparty/libtommath/bn_mp_signed_bin_size.o 3rdparty/libtommath/bn_mp_sqr.o 3rdparty/libtommath/bn_mp_sqrmod.o 3rdparty/libtommath/bn_mp_sqrt.o 3rdparty/libtommath/bn_mp_sub.o 3rdparty/libtommath/bn_mp_sub_d.o 3rdparty/libtommath/bn_mp_submod.o 3rdparty/libtommath/bn_mp_to_signed_bin.o 3rdparty/libtommath/bn_mp_to_signed_bin_n.o 3rdparty/libtommath/bn_mp_to_unsigned_bin.o 3rdparty/libtommath/bn_mp_to_unsigned_bin_n.o 3rdparty/libtommath/bn_mp_toom_mul.o 3rdparty/libtommath/bn_mp_toom_sqr.o 3rdparty/libtommath/bn_mp_toradix.o 3rdparty/libtommath/bn_mp_toradix_n.o 3rdparty/libtommath/bn_mp_unsigned_bin_size.o 3rdparty/libtommath/bn_mp_xor.o 3rdparty/libtommath/bn_mp_zero.o 3rdparty/libtommath/bn_prime_tab.o 3rdparty/libtommath/bn_reverse.o 3rdparty/libtommath/bn_s_mp_add.o 3rdparty/libtommath/bn_s_mp_exptmod.o 3rdparty/libtommath/bn_s_mp_mul_digs.o 3rdparty/libtommath/bn_s_mp_mul_high_digs.o 3rdparty/libtommath/bn_s_mp_sqr.o 3rdparty/libtommath/bn_s_mp_sub.o 3rdparty/libtommath/bncore.o moar::syslibs[1]=pthread perl6::language_version=6.c perl6::codename= perl6::release-number= perl6::build-date=2016-10-21T02:29:17Z perl6::version=2016.10 perl6::implementation=Rakudo
Doug Schrag \<>\< Houston, TX Traverse City, MI
Sorry, wrong system report previously attached.
moar::dcbrule=@: moar::ar=ar moar::mastdir=/usr/local/opt/rakudo-star-2016.10-RC0/share/nqp/lib/MAST moar::tomrule=$(AR) $(ARFLAGS) $@ 3rdparty/libtommath/*.o moar::syslibs[2]=rt moar::shaobjects=3rdparty/sha1/sha1.o moar::ccshared=-fPIC moar::be=0 moar::dynasmlua=./3rdparty/dynasm/dynasm.lua moar::ld=gcc moar::mtlib=3rdparty/tinymt/libtinymt.a moar::mtobjects=3rdparty/tinymt/tinymt64.o moar::noreturnspecifier= moar::perl=/usr/bin/perl5.18.1 moar::shaclean=$(RM) 3rdparty/sha1/libsha1.a 3rdparty/sha1/*.o moar::static_inline=static __inline__ moar::ccdebugflags=-g3 moar::auxclean=@: moar::thirdpartylibs=3rdparty/dyncall/dyncall/libdyncall_s.a 3rdparty/dyncall/dyncallback/libdyncallback_s.a 3rdparty/dyncall/dynload/libdynload_s.a 3rdparty/libatomic_ops/src/libatomic_ops.a 3rdparty/tinymt/libtinymt.a 3rdparty/sha1/libsha1.a 3rdparty/libtommath/libtommath.a 3rdparty/libuv/libuv.a moar::config=--optimize --prefix=/usr/local/opt/rakudo-star-2016.10-RC0 --make-install moar::ccinc=-I moar::booltype=_Bool moar::ldout=-o moar::moarlib=libmoar.a moar::mtrule=$(AR) $(ARFLAGS) $@ 3rdparty/tinymt/*.o moar::syslibs[0]=m moar::dcrule=cd 3rdparty/dyncall && ./configure && CC='$(CC)' CFLAGS='$(CFLAGS)' $(MAKE) -f Makefile moar::dcbobjects= moar::cflags=-Wdeclaration-after-statement -Werror=declaration-after-statement -O3 -DNDEBUG -D_REENTRANT -D_FILE_OFFSET_BITS=64 -fPIC moar::laolib=3rdparty/libatomic_ops/src/libatomic_ops.a moar::libdir=/usr/local/opt/rakudo-star-2016.10-RC0/lib moar::nul=/dev/null moar::osvers=3.10.9 moar::translate_newline_output=0 moar::dlrule=@: moar::cat=cat moar::can_unaligned_int32=1 moar::ldimp= moar::pkgconfig=/usr/bin/pkg-config moar::ccdef=-D moar::lib=lib%s.a moar::tomclean=$(RM) 3rdparty/libtommath/libtommath.a 3rdparty/libtommath/*.o moar::dlllocal=__attribute__ ((visibility ("hidden"))) moar::dcbclean=$(RM) 3rdparty/dyncall/dyncallback/libdyncallback_s.a moar::noreturnattribute=__attribute__((noreturn)) moar::ptr_size=8 moar::dllimport=__attribute__ ((visibility ("default"))) moar::ccout=-o moar::can_unaligned_num64=1 moar::has_pthread_yield=1 moar::ldrpath=-Wl,-rpath,"//usr/local/opt/rakudo-star-2016.10-RC0/lib" -Wl,-rpath,"/usr/local/opt/rakudo-star-2016.10-RC0/share/perl6/site/lib" moar::ldusr=-l%s moar::mainflags=-DMVM_SHARED moar::name=moar moar::dll=lib%s.so moar::cincludes= -I3rdparty/libuv/include -I3rdparty/libuv/src -I3rdparty/libatomic_ops/src -I3rdparty/libtommath -I3rdparty/dynasm -I3rdparty/dyncall/dynload -I3rdparty/dyncall/dyncall -I3rdparty/dyncall/dyncallback moar::uvrule=$(AR) $(ARFLAGS) $@ $(UV_LINUX) moar::tomlib=3rdparty/libtommath/libtommath.a moar::syslibs[3]=dl moar::staticlib= moar::dlclean=$(RM) 3rdparty/dyncall/dynload/libdynload_s.a moar::make=make moar::ccinstflags=-pg moar::ldflags= -O3 -DNDEBUG -Wl,-rpath,"//usr/local/opt/rakudo-star-2016.10-RC0/lib" -Wl,-rpath,"/usr/local/opt/rakudo-star-2016.10-RC0/share/perl6/site/lib" moar::crossconf= moar::obj=.o moar::dlobjects= moar::defs[0]=_REENTRANT moar::cppout=> moar::ccoptiflags=-O3 -DNDEBUG moar::asmswitch=-S moar::ldoptiflags=-O3 -DNDEBUG moar::mkflags= moar::moar=libmoar.so moar::version=2016.10 moar::dcclean=cd 3rdparty/dyncall && $(MAKE) -f Makefile clean moar::moarshared= moar::sharule=$(AR) $(ARFLAGS) $@ 3rdparty/sha1/*.o moar::ccwarnflags= moar::jit=$(JIT_POSIX_X64) moar::moardll=libmoar.so moar::pkgconfig_works=1 moar::asm=.s moar::mknoisy=ifneq ($(NOISY), 1)MSG = @echoCMD = @NOOUT = > /dev/nullNOERR = 2> /dev/nullendif moar::dclib=3rdparty/dyncall/dyncall/libdyncall_s.a moar::ccdefflags=-D_REENTRANT -D_FILE_OFFSET_BITS=64 moar::cc=gcc moar::exe= moar::lddir=-L moar::uvlib=3rdparty/libuv/libuv.a moar::dllib=3rdparty/dyncall/dynload/libdynload_s.a moar::shaincludedir=3rdparty/sha1 moar::versionmajor=2016 moar::cppswitch=-E moar::can_unaligned_int64=1 moar::lua=./3rdparty/dynasm/minilua moar::dcobjects= moar::cancgoto=1 moar::ldlibs=-lm -lpthread -lrt -ldl moar::mainlibs=-L. -lmoar moar::nativecall_backend=dyncall moar::objflags=-DMVM_BUILD_SHARED -fPIC moar::versionpatch=0 moar::laoclean=cd 3rdparty/libatomic_ops/src && $(MAKE) distclean moar::versionminor=10 moar::ccmiscflags=-Wdeclaration-after-statement -Werror=declaration-after-statement moar::install= $(MKPATH) $(DESTDIR)$(PREFIX)/include/libuv $(CP) 3rdparty/libuv/include/*.h $(DESTDIR)$(PREFIX)/include/libuv $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/armcc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/gcc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/hpc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/ibmc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/icc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/loadstore $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/msftc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/sunc $(CP) 3rdparty/libatomic_ops/src/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops $(CP) 3rdparty/libatomic_ops/src/atomic_ops/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/armcc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/armcc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/gcc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/gcc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/hpc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/hpc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/ibmc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/ibmc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/icc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/icc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/loadstore/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/loadstore $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/msftc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/msftc $(CP) 3rdparty/libatomic_ops/src/atomic_ops/sysdeps/sunc/*.h $(DESTDIR)$(PREFIX)/include/libatomic_ops/atomic_ops/sysdeps/sunc $(MKPATH) $(DESTDIR)$(PREFIX)/include/libtommath $(CP) 3rdparty/libtommath/*.h $(DESTDIR)$(PREFIX)/include/libtommath $(MKPATH) $(DESTDIR)$(PREFIX)/include/dyncall $(CP) 3rdparty/dyncall/dynload/*.h $(DESTDIR)$(PREFIX)/include/dyncall $(CP) 3rdparty/dyncall/dyncall/*.h $(DESTDIR)$(PREFIX)/include/dyncall $(CP) 3rdparty/dyncall/dyncallback/*.h $(DESTDIR)$(PREFIX)/include/dyncall moar::laoobjects= moar::mtclean=$(RM) 3rdparty/tinymt/libtinymt.a 3rdparty/tinymt/*.o moar::rm=rm -f moar::uvclean=$(RM) 3rdparty/libuv/libuv.a $(UV_LINUX) moar::sharedlib= moar::dllexport=__attribute__ ((visibility ("default"))) moar::shalib=3rdparty/sha1/libsha1.a moar::arout= moar::lddebugflags=-g3 moar::osname=linux moar::prefix=/usr/local/opt/rakudo-star-2016.10-RC0 moar::uvobjects=$(UV_LINUX) moar::dcblib=3rdparty/dyncall/dyncallback/libdyncallback_s.a moar::bindir=/usr/local/opt/rakudo-star-2016.10-RC0/bin moar::asmout=-o moar::havebooltype=1 moar::laorule=cd 3rdparty/libatomic_ops && CC='$(CC)' CFLAGS='$(CFLAGS)' ./configure && cd src && $(MAKE) && cd .. moar::os=linux moar::sh=/bin/sh moar::arflags=rcs moar::ldshared=-shared -fPIC moar::defs[1]=_FILE_OFFSET_BITS=64 moar::impinst= moar::ldinstflags=-pg moar::ldmiscflags= moar::ldsys=-l%s moar::platform=$(PLATFORM_POSIX) moar::ccswitch=-c moar::formatattribute=__attribute__((format(X, Y, Z))) moar::tomobjects=3rdparty/libtommath/bn_error.o 3rdparty/libtommath/bn_fast_mp_invmod.o 3rdparty/libtommath/bn_fast_mp_montgomery_reduce.o 3rdparty/libtommath/bn_fast_s_mp_mul_digs.o 3rdparty/libtommath/bn_fast_s_mp_mul_high_digs.o 3rdparty/libtommath/bn_fast_s_mp_sqr.o 3rdparty/libtommath/bn_mp_2expt.o 3rdparty/libtommath/bn_mp_abs.o 3rdparty/libtommath/bn_mp_add.o 3rdparty/libtommath/bn_mp_add_d.o 3rdparty/libtommath/bn_mp_addmod.o 3rdparty/libtommath/bn_mp_and.o 3rdparty/libtommath/bn_mp_clamp.o 3rdparty/libtommath/bn_mp_clear.o 3rdparty/libtommath/bn_mp_clear_multi.o 3rdparty/libtommath/bn_mp_cmp.o 3rdparty/libtommath/bn_mp_cmp_d.o 3rdparty/libtommath/bn_mp_cmp_mag.o 3rdparty/libtommath/bn_mp_cnt_lsb.o 3rdparty/libtommath/bn_mp_copy.o 3rdparty/libtommath/bn_mp_count_bits.o 3rdparty/libtommath/bn_mp_div.o 3rdparty/libtommath/bn_mp_div_2.o 3rdparty/libtommath/bn_mp_div_2d.o 3rdparty/libtommath/bn_mp_div_3.o 3rdparty/libtommath/bn_mp_div_d.o 3rdparty/libtommath/bn_mp_dr_is_modulus.o 3rdparty/libtommath/bn_mp_dr_reduce.o 3rdparty/libtommath/bn_mp_dr_setup.o 3rdparty/libtommath/bn_mp_exch.o 3rdparty/libtommath/bn_mp_expt_d.o 3rdparty/libtommath/bn_mp_exptmod.o 3rdparty/libtommath/bn_mp_exptmod_fast.o 3rdparty/libtommath/bn_mp_exteuclid.o 3rdparty/libtommath/bn_mp_fread.o 3rdparty/libtommath/bn_mp_fwrite.o 3rdparty/libtommath/bn_mp_gcd.o 3rdparty/libtommath/bn_mp_get_int.o 3rdparty/libtommath/bn_mp_get_long.o 3rdparty/libtommath/bn_mp_grow.o 3rdparty/libtommath/bn_mp_init.o 3rdparty/libtommath/bn_mp_init_copy.o 3rdparty/libtommath/bn_mp_init_multi.o 3rdparty/libtommath/bn_mp_init_set.o 3rdparty/libtommath/bn_mp_init_set_int.o 3rdparty/libtommath/bn_mp_init_size.o 3rdparty/libtommath/bn_mp_invmod.o 3rdparty/libtommath/bn_mp_invmod_slow.o 3rdparty/libtommath/bn_mp_is_square.o 3rdparty/libtommath/bn_mp_jacobi.o 3rdparty/libtommath/bn_mp_karatsuba_mul.o 3rdparty/libtommath/bn_mp_karatsuba_sqr.o 3rdparty/libtommath/bn_mp_lcm.o 3rdparty/libtommath/bn_mp_lshd.o 3rdparty/libtommath/bn_mp_mod.o 3rdparty/libtommath/bn_mp_mod_2d.o 3rdparty/libtommath/bn_mp_mod_d.o 3rdparty/libtommath/bn_mp_montgomery_calc_normalization.o 3rdparty/libtommath/bn_mp_montgomery_reduce.o 3rdparty/libtommath/bn_mp_montgomery_setup.o 3rdparty/libtommath/bn_mp_mul.o 3rdparty/libtommath/bn_mp_mul_2.o 3rdparty/libtommath/bn_mp_mul_2d.o 3rdparty/libtommath/bn_mp_mul_d.o 3rdparty/libtommath/bn_mp_mulmod.o 3rdparty/libtommath/bn_mp_n_root.o 3rdparty/libtommath/bn_mp_neg.o 3rdparty/libtommath/bn_mp_or.o 3rdparty/libtommath/bn_mp_prime_fermat.o 3rdparty/libtommath/bn_mp_prime_is_divisible.o 3rdparty/libtommath/bn_mp_prime_is_prime.o 3rdparty/libtommath/bn_mp_prime_miller_rabin.o 3rdparty/libtommath/bn_mp_prime_next_prime.o 3rdparty/libtommath/bn_mp_prime_rabin_miller_trials.o 3rdparty/libtommath/bn_mp_prime_random_ex.o 3rdparty/libtommath/bn_mp_radix_size.o 3rdparty/libtommath/bn_mp_radix_smap.o 3rdparty/libtommath/bn_mp_rand.o 3rdparty/libtommath/bn_mp_read_radix.o 3rdparty/libtommath/bn_mp_read_signed_bin.o 3rdparty/libtommath/bn_mp_read_unsigned_bin.o 3rdparty/libtommath/bn_mp_reduce.o 3rdparty/libtommath/bn_mp_reduce_2k.o 3rdparty/libtommath/bn_mp_reduce_2k_l.o 3rdparty/libtommath/bn_mp_reduce_2k_setup.o 3rdparty/libtommath/bn_mp_reduce_2k_setup_l.o 3rdparty/libtommath/bn_mp_reduce_is_2k.o 3rdparty/libtommath/bn_mp_reduce_is_2k_l.o 3rdparty/libtommath/bn_mp_reduce_setup.o 3rdparty/libtommath/bn_mp_rshd.o 3rdparty/libtommath/bn_mp_set.o 3rdparty/libtommath/bn_mp_set_int.o 3rdparty/libtommath/bn_mp_set_long.o 3rdparty/libtommath/bn_mp_shrink.o 3rdparty/libtommath/bn_mp_signed_bin_size.o 3rdparty/libtommath/bn_mp_sqr.o 3rdparty/libtommath/bn_mp_sqrmod.o 3rdparty/libtommath/bn_mp_sqrt.o 3rdparty/libtommath/bn_mp_sub.o 3rdparty/libtommath/bn_mp_sub_d.o 3rdparty/libtommath/bn_mp_submod.o 3rdparty/libtommath/bn_mp_to_signed_bin.o 3rdparty/libtommath/bn_mp_to_signed_bin_n.o 3rdparty/libtommath/bn_mp_to_unsigned_bin.o 3rdparty/libtommath/bn_mp_to_unsigned_bin_n.o 3rdparty/libtommath/bn_mp_toom_mul.o 3rdparty/libtommath/bn_mp_toom_sqr.o 3rdparty/libtommath/bn_mp_toradix.o 3rdparty/libtommath/bn_mp_toradix_n.o 3rdparty/libtommath/bn_mp_unsigned_bin_size.o 3rdparty/libtommath/bn_mp_xor.o 3rdparty/libtommath/bn_mp_zero.o 3rdparty/libtommath/bn_prime_tab.o 3rdparty/libtommath/bn_reverse.o 3rdparty/libtommath/bn_s_mp_add.o 3rdparty/libtommath/bn_s_mp_exptmod.o 3rdparty/libtommath/bn_s_mp_mul_digs.o 3rdparty/libtommath/bn_s_mp_mul_high_digs.o 3rdparty/libtommath/bn_s_mp_sqr.o 3rdparty/libtommath/bn_s_mp_sub.o 3rdparty/libtommath/bncore.o moar::syslibs[1]=pthread perl6::language_version=6.c perl6::codename= perl6::release-number= perl6::build-date=2016-10-21T02:29:17Z perl6::version=2016.10 perl6::implementation=Rakudo
Doug Schrag \<>\< Houston, TX Traverse City, MI
On Sun Oct 23 18:01:55 2016, doug@theschrags.net wrote:
This probably is an issue with moarvm.
It was indeed.
When trying to benchmark the concurrent and non-concurrent versions of Damien Conway's 'bogosort' algorithm, I ran into numerous problems with Rakudo Star 2016.07 - depending on the MAX_RAKUDO_THREADS setting, the process might succeed, or it might slowly eat up all of memory, and if more than a few threads were used only some would actually process anything (accumulate CPU time). rakudo-star-2016.10-RC0 looks like it solved the deadlock problem, so that all threads are actually working.
However, now this version of the program crashes after the first thread completes with: Internal error: zeroed target thread ID in work pass
or occasionally: Internal error: invalid thread ID 8770096 in GC work pass
Test runs:
doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 1 seconds. Internal error: invalid thread ID 8770096 in GC work pass doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 4 seconds. Internal error: zeroed target thread ID in work pass doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 24 seconds. Internal error: zeroed target thread ID in work pass doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 6 seconds. Internal error: zeroed target thread ID in work pass
I also found that the problem is likely related to a channel not being explicitly closed, so I was able to get normal test runs as follows:
[ With a LEAVE block to close the channels ]
doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 4 seconds. [e i l p r s x] 29 seconds. [e i l p r s x] 57 seconds. [e i l p r s x] 58 seconds. doug@ender:\~/study/Perl6/benchmarks$ RAKUDO_MAX_THREADS=2 perl6 bench_bogosort_simple.pl6 [e i l p r s x] 4 seconds. [e i l p r s x] 10 seconds. [e i l p r s x] 20 seconds. [e i l p r s x] 31 seconds.
I'm figuring that failing to close the channels is bad practice to start, but we don't really want to crash the VM with an obscure error message. Also, I saw a bug report earlier this year where this message was produced, but closed because of uncertainty in reproducing it. This test, however fails quite repeatably, at least on my system (linux).
The channel really does want closing, otherwise you end up with a bunch of threads sat in a hot loop eating CPU (and each run sets off another thread doing exactly that, which is why the thing gets slower every run with the missing Channel.close). Of course, a VM crash is the wrong response; I've now hunted that down and committed a fix.
It turns out the bug wasn't actually anything to do with closing the channel per se; it's just that missing the `LEAVE` out resulted in a lot more load and GC runs. The problem almost certainly could have happened with the `LEAVE` there; it was just a lot less likely.
I've added this example as a stresstest, to catch any regressions.
Thanks!
/jnthn
The RT System itself - Status changed from 'new' to 'open'
@jnthn - Status changed from 'open' to 'resolved'
Migrated from rt.perl.org#129949 (status was 'resolved')
Searchable as RT129949$