search

open-mpi / ompi

Open MPI main development repository
https://www.open-mpi.org
Other
2.12k stars 858 forks source link

Running MPI JAVA on cluster machine #2300

Closed spagnuolocarmine closed 7 years ago

spagnuolocarmine commented 7 years ago

I have installed and configured MPI and Java, but sometimes (really often) my job was killed and enclosed there is the log. Please help me...

Thank you ...

> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00002ac4ff55259c, pid=119842, tid=47027750721280
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_51-b16) (build 1.8.0_51-b16)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.51-b03 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # V  [libjvm.so+0x8cb59c]  Monitor::wait(bool, long, bool)+0x35c
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #
> 
> ---------------  T H R E A D  ---------------
> 
> Current thread (0x00002ac5002a8800):  JavaThread "C2 CompilerThread2" daemon [_thread_blocked_trans, id=120195, stack(0x00002ac580505000,0x00002ac580606000)]
> 
> siginfo: si_signo: -2141173936 (INVALID), si_code: 0 (SI_USER), si_errno: 10949, sent from pid: 681984 (uid: 10949)
> 
> Registers:
> RAX=0x00002ac504644880, RBX=0x00002ac4ffc21640, RCX=0x0000000000000001, RDX=0x00002ac4ffc21630
> RSP=0x00002ac580604b90, RBP=0x00002ac580604bd0, RSI=0x0000000000000001, RDI=0x00002ac500006b60
> R8 =0x00002ac4ffc21640, R9 =0x000000000001d583, R10=0x00002ac580604b00, R11=0x0000000000000206
> R12=0x0000000000001388, R13=0x00002ac5002a8800, R14=0x00002ac500006b60, R15=0x0000000000000000
> RIP=0x00002ac4ff55259c, EFLAGS=0x0000000000010202, CSGSFS=0x0000000000000033, ERR=0x0000000000000007
>   TRAPNO=0x000000000000000e
> 
> Top of Stack: (sp=0x00002ac580604b90)
> 0x00002ac580604b90:   00002ac5002ac9c0 0000000280000000
> 0x00002ac580604ba0:   000000000ec85240 000000000ec85240
> 0x00002ac580604bb0:   00002ac5002a3690 00002ac500006b60
> 0x00002ac580604bc0:   00002ac5002d68d0 00002ac5002a8800
> 0x00002ac580604bd0:   00002ac580604c00 00002ac4ff12dd4b
> 0x00002ac580604be0:   00002ac4ffbe1458 000000000ec85240
> 0x00002ac580604bf0:   000000000087ed60 00002ac5002a8800
> 0x00002ac580604c00:   00002ac580604d10 00002ac4ff12ff9b
> 0x00002ac580604c10:   0000000000000000 00002ac580604ca0
> 0x00002ac580604c20:   00002ac580604cd0 00002ac580604c60
> 0x00002ac580604c30:   00002ac5002a3690 00000000000003d8
> 0x00002ac580604c40:   00002ac5002a9310 00002ac5002a9738
> 0x00002ac580604c50:   00002ac5002a9360 00002ac5002a9350
> 0x00002ac580604c60:   00002ac5002a8800 00002ac5002a9740
> 0x00002ac580604c70:   00002ac5002a9780 00002ac5002a9790
> 0x00002ac580604c80:   00002ac5002a9868 00000000000000d8
> 0x00002ac580604c90:   00002ac580604d30 0000000000000130
> 0x00002ac580604ca0:   00000000000000fa 00000001f80d029b
> 0x00002ac580604cb0:   00002ac580604d00 00002ac5002a8610
> 0x00002ac580604cc0:   0000000000000000 00002ac5002a8690
> 0x00002ac580604cd0:   00002ac504c1b990 0000000000000012
> 0x00002ac580604ce0:   00000000d55cb438 00002ac580604d30
> 0x00002ac580604cf0:   00002ac5002a8800 00002ac5002a9350
> 0x00002ac580604d00:   00000000000003d8 00002ac5002a9738
> 0x00002ac580604d10:   00002ac580604da0 00002ac4ff6e3bcf
> 0x00002ac580604d20:   00002ac580604d40 00002ac5002a9360
> 0x00002ac580604d30:   00002ac5002a8800 00002ac5002a9740
> 0x00002ac580604d40:   00002ac5002a9780 00002ac5002a9790
> 0x00002ac580604d50:   00002ac5002a9868 00000000000000d8
> 0x00002ac580604d60:   00002ac5002a99a0 00002ac4ff55148f
> 0x00002ac580604d70:   0000000000000000 00002ac5002a8800
> 0x00002ac580604d80:   00002ac5002acab0 00002ac5002ac9c0 
> 
> Instructions: (pc=0x00002ac4ff55259c)
> 0x00002ac4ff55257c:   00 00 00 00 48 8b 05 b1 90 68 00 4c 89 ea 48 c1
> 0x00002ac4ff55258c:   ea 04 8b 00 21 d0 48 8b 15 ff 49 68 00 48 03 02
> 0x00002ac4ff55259c:   c7 00 01 00 00 00 e9 8a fe ff ff 66 0f 1f 84 00
> 0x00002ac4ff5525ac:   00 00 00 00 31 f6 4c 89 e7 e8 06 e6 ff ff e9 e6 
> 
> Register to memory mapping:
> 
> RAX=0x00002ac504644880 is an unknown value
> RBX=0x00002ac4ffc21640: <offset 0xf9a640> in /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/server/libjvm.so at 0x00002ac4fec87000
> RCX=0x0000000000000001 is an unknown value
> RDX=0x00002ac4ffc21630: <offset 0xf9a630> in /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/server/libjvm.so at 0x00002ac4fec87000
> RSP=0x00002ac580604b90 is pointing into the stack for thread: 0x00002ac5002a8800
> RBP=0x00002ac580604bd0 is pointing into the stack for thread: 0x00002ac5002a8800
> RSI=0x0000000000000001 is an unknown value
> RDI=0x00002ac500006b60 is an unknown value
> R8 =0x00002ac4ffc21640: <offset 0xf9a640> in /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/server/libjvm.so at 0x00002ac4fec87000
> R9 =0x000000000001d583 is an unknown value
> R10=0x00002ac580604b00 is pointing into the stack for thread: 0x00002ac5002a8800
> R11=0x0000000000000206 is an unknown value
> R12=0x0000000000001388 is an unknown value
> R13=0x00002ac5002a8800 is a thread
> R14=0x00002ac500006b60 is an unknown value
> R15=0x0000000000000000 is an unknown value
> 
> 
> Stack: [0x00002ac580505000,0x00002ac580606000],  sp=0x00002ac580604b90,  free space=1022k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V  [libjvm.so+0x8cb59c]  Monitor::wait(bool, long, bool)+0x35c
> V  [libjvm.so+0x4a6d4b]  CompileQueue::get()+0x13b
> V  [libjvm.so+0x4a8f9b]  CompileBroker::compiler_thread_loop()+0x18b
> V  [libjvm.so+0xa5cbcf]  JavaThread::thread_main_inner()+0xdf
> V  [libjvm.so+0xa5ccfc]  JavaThread::run()+0x11c
> V  [libjvm.so+0x911048]  java_start(Thread*)+0x108
> C  [libpthread.so.0+0x7aa1]  start_thread+0xd1
> 
> 
> ---------------  P R O C E S S  ---------------
> 
> Java Threads: ( => current thread )
>   0x00002ac5002d4000 JavaThread "Service Thread" daemon [_thread_blocked, id=120278, stack(0x00002ac515016000,0x00002ac515117000)]
>   0x00002ac5002c7000 JavaThread "C1 CompilerThread11" daemon [_thread_blocked, id=120237, stack(0x00002ac5c8000000,0x00002ac5c8101000)]
>   0x00002ac5002c5000 JavaThread "C1 CompilerThread10" daemon [_thread_blocked, id=120235, stack(0x00002ac5bc101000,0x00002ac5bc202000)]
>   0x00002ac5002c2800 JavaThread "C1 CompilerThread9" daemon [_thread_blocked, id=120230, stack(0x00002ac5bc000000,0x00002ac5bc101000)]
>   0x00002ac5002c0800 JavaThread "C1 CompilerThread8" daemon [_thread_blocked, id=120229, stack(0x00002ac5b0101000,0x00002ac5b0202000)]
>   0x00002ac5002be000 JavaThread "C2 CompilerThread7" daemon [_thread_blocked, id=120209, stack(0x00002ac5b0000000,0x00002ac5b0101000)]
>   0x00002ac5002bc000 JavaThread "C2 CompilerThread6" daemon [_thread_blocked, id=120203, stack(0x00002ac59c303000,0x00002ac59c404000)]
>   0x00002ac5002b9800 JavaThread "C2 CompilerThread5" daemon [_thread_blocked, id=120202, stack(0x00002ac59c202000,0x00002ac59c303000)]
>   0x00002ac5002af800 JavaThread "C2 CompilerThread4" daemon [_thread_blocked, id=120200, stack(0x00002ac59c101000,0x00002ac59c202000)]
>   0x00002ac5002ad800 JavaThread "C2 CompilerThread3" daemon [_thread_blocked, id=120197, stack(0x00002ac59c000000,0x00002ac59c101000)]
> =>0x00002ac5002a8800 JavaThread "C2 CompilerThread2" daemon [_thread_blocked_trans, id=120195, stack(0x00002ac580505000,0x00002ac580606000)]
>   0x00002ac5002a7000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=120194, stack(0x00002ac580404000,0x00002ac580505000)]
>   0x00002ac5002a4000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=120190, stack(0x00002ac580303000,0x00002ac580404000)]
>   0x00002ac5002a1800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=120185, stack(0x00002ac580202000,0x00002ac580303000)]
>   0x00002ac500277000 JavaThread "Finalizer" daemon [_thread_blocked, id=120158, stack(0x00002ac580101000,0x00002ac580202000)]
>   0x00002ac500275000 JavaThread "Reference Handler" daemon [_thread_blocked, id=120157, stack(0x00002ac580000000,0x00002ac580101000)]
>   0x00002ac500007800 JavaThread "main" [_thread_blocked, id=119871, stack(0x00002ac4ffee6000,0x00002ac4fffe7000)]
> 
> Other Threads:
>   0x00002ac500270000 VMThread [stack: 0x00002ac55794a000,0x00002ac557a4b000] [id=120142]
>   0x00002ac5002d7000 WatcherThread [stack: 0x00002ac5d4000000,0x00002ac5d4101000] [id=120281]
> 
> VM state:synchronizing (normal execution)
> 
> VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
> [0x00002ac500005fe0] Safepoint_lock - owner thread: 0x00002ac500270000
> [0x00002ac500006060] Threads_lock - owner thread: 0x00002ac500270000
> [0x00002ac500006a60] Compile_lock - owner thread: 0x00002ac500007800
> 
> Heap:
>  PSYoungGen      total 76288K, used 19664K [0x00000000d5580000, 0x00000000daa80000, 0x0000000100000000)
>   eden space 65536K, 30% used [0x00000000d5580000,0x00000000d68b40c0,0x00000000d9580000)
>   from space 10752K, 0% used [0x00000000da000000,0x00000000da000000,0x00000000daa80000)
>   to   space 10752K, 0% used [0x00000000d9580000,0x00000000d9580000,0x00000000da000000)
>  ParOldGen       total 175104K, used 0K [0x0000000080000000, 0x000000008ab00000, 0x00000000d5580000)
>   object space 175104K, 0% used [0x0000000080000000,0x0000000080000000,0x000000008ab00000)
>  Metaspace       used 5540K, capacity 5584K, committed 5760K, reserved 1056768K
>   class space    used 601K, capacity 624K, committed 640K, reserved 1048576K
> 
> Card table byte_map: [0x00002ac513c5d000,0x00002ac51405e000] byte_map_base: 0x00002ac51385d000
> 
> Marking Bits: (ParMarkBitMap*) 0x00002ac4ffc25660
>  Begin Bits: [0x00002ac54c000000, 0x00002ac54e000000)
>  End Bits:   [0x00002ac54e000000, 0x00002ac550000000)
> 
> Polling page: 0x00002ac504643000
> 
> CodeCache: size=245760Kb used=3641Kb max_used=3641Kb free=242118Kb
>  bounds [0x00002ac50489d000, 0x00002ac504c2d000, 0x00002ac51389d000]
>  total_blobs=754 nmethods=404 adapters=261
>  compilation: enabled
> 
> Compilation events (10 events):
> Event: 8.492 Thread 0x00002ac5002c7000 nmethod 403 0x00002ac504c28510 code [0x00002ac504c286c0, 0x00002ac504c28d98]
> Event: 8.497 Thread 0x00002ac5002c7000  404  s    3       java.util.Hashtable::get (69 bytes)
> Event: 8.497 Thread 0x00002ac5002c7000 nmethod 404 0x00002ac504c2a450 code [0x00002ac504c2a600, 0x00002ac504c2ac18]
> Event: 8.497 Thread 0x00002ac5002c7000  405       3       java.util.ArrayList::add (29 bytes)
> Event: 8.497 Thread 0x00002ac5002c0800  406       3       java.util.ArrayList::ensureCapacityInternal (23 bytes)
> Event: 8.498 Thread 0x00002ac5002c0800 nmethod 406 0x00002ac504c2ae90 code [0x00002ac504c2b020, 0x00002ac504c2b348]
> Event: 8.498 Thread 0x00002ac5002c0800  407       3       java.util.ArrayList::ensureExplicitCapacity (26 bytes)
> Event: 8.498 Thread 0x00002ac5002c7000 nmethod 405 0x00002ac504c2b490 code [0x00002ac504c2b640, 0x00002ac504c2bb48]
> Event: 8.498 Thread 0x00002ac5002c0800 nmethod 407 0x00002ac504c2bd90 code [0x00002ac504c2bf00, 0x00002ac504c2c0e8]
> Event: 8.499 Thread 0x00002ac5002c7000  408       3       java.lang.String::regionMatches (149 bytes)
> 
> GC Heap History (0 events):
> No events
> 
> Deoptimization events (10 events):
> Event: 6.589 Thread 0x00002ac500007800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00002ac504b97ed0 method=java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object; @ 56
> Event: 6.609 Thread 0x00002ac500007800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00002ac504b9a620 method=java.lang.StringCoding$StringDecoder.decode([BII)[C @ 30
> Event: 6.614 Thread 0x00002ac500007800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00002ac504b8759c method=java.lang.String.hashCode()I @ 14
> Event: 6.615 Thread 0x00002ac500007800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00002ac504b9a620 method=java.lang.StringCoding$StringDecoder.decode([BII)[C @ 30
> Event: 6.630 Thread 0x00002ac500007800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00002ac504ba3da0 method=java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object; @ 69
> Event: 6.630 Thread 0x00002ac500007800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00002ac504ba6108 method=java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object; @ 69
> Event: 6.638 Thread 0x00002ac500007800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00002ac504b9a620 method=java.lang.StringCoding$StringDecoder.decode([BII)[C @ 30
> Event: 6.638 Thread 0x00002ac500007800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00002ac504b9a620 method=java.lang.StringCoding$StringDecoder.decode([BII)[C @ 30
> Event: 6.767 Thread 0x00002ac500007800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00002ac504bd6668 method=java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object; @ 181
> Event: 6.779 Thread 0x00002ac500007800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00002ac504bd37c0 method=java.util.HashMap.putVal(ILjava/lang/Object;Ljava/lang/Object;ZZ)Ljava/lang/Object; @ 181
> 
> Internal exceptions (10 events):
> Event: 6.757 Thread 0x00002ac500007800 Exception <a 'java/lang/ArrayIndexOutOfBoundsException': 22> (0x00000000d664a918) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/interpreter/interpreterRuntime.cpp, line 366]
> Event: 6.762 Thread 0x00002ac500007800 Exception <a 'java/lang/ArrayIndexOutOfBoundsException'> (0x00000000d664ccb8) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/runtime/sharedRuntime.cpp, line 605]
> Event: 6.765 Thread 0x00002ac500007800 Exception <a 'java/lang/ClassNotFoundException': org/apache/log4j/LayoutCustomizer> (0x00000000d66509c0) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 210]
> Event: 6.767 Thread 0x00002ac500007800 Exception <a 'java/lang/ClassNotFoundException': org/apache/log4j/PatternLayoutCustomizer> (0x00000000d665fd20) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 210]
> Event: 6.768 Thread 0x00002ac500007800 Exception <a 'java/lang/ClassNotFoundException': org/apache/log4j/ConsoleAppenderBeanInfo> (0x00000000d6668340) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 210]
> Event: 6.768 Thread 0x00002ac500007800 Exception <a 'java/lang/ClassNotFoundException': org/apache/log4j/WriterAppenderBeanInfo> (0x00000000d6669bc0) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 210]
> Event: 6.768 Thread 0x00002ac500007800 Exception <a 'java/lang/ClassNotFoundException': org/apache/log4j/AppenderSkeletonBeanInfo> (0x00000000d666b4c8) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 210]
> Event: 6.768 Thread 0x00002ac500007800 Exception <a 'java/lang/ClassNotFoundException': org/apache/log4j/AppenderSkeletonCustomizer> (0x00000000d666cd78) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 210]
> Event: 6.777 Thread 0x00002ac500007800 Exception <a 'java/lang/ClassNotFoundException': org/apache/log4j/WriterAppenderCustomizer> (0x00000000d6680270) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 210]
> Event: 6.780 Thread 0x00002ac500007800 Exception <a 'java/lang/ClassNotFoundException': org/apache/log4j/ConsoleAppenderCustomizer> (0x00000000d668e1e8) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u51/3951/hotspot/src/share/vm/classfile/systemDictionary.cpp, line 210]
> 
> Events (10 events):
> Event: 8.495 loading class it/isislab/dmason/sim/field/continuous/region/RegionDouble done
> Event: 8.498 loading class it/isislab/dmason/experimentals/util/visualization/globalviewer/VisualizationUpdateMap
> Event: 8.498 loading class it/isislab/dmason/experimentals/util/visualization/globalviewer/VisualizationUpdateMap done
> Event: 8.498 loading class it/isislab/dmason/sim/field/support/field2D/region/RegionMap
> Event: 8.498 loading class it/isislab/dmason/sim/field/support/field2D/region/RegionMap done
> Event: 8.499 loading class it/isislab/dmason/sim/field/support/field2D/UpdateMap
> Event: 8.499 loading class it/isislab/dmason/sim/field/support/field2D/UpdateMap done
> Event: 8.499 loading class it/isislab/dmason/experimentals/util/visualization/zoomviewerapp/ZoomArrayList
> Event: 8.499 loading class it/isislab/dmason/experimentals/util/visualization/zoomviewerapp/ZoomArrayList done
> Event: 8.500 Executing VM operation: Deoptimize
> 
> 
> Dynamic libraries:
> 00400000-00401000 r-xp 00000000 00:1d 6377881433                         /blues/gpfs/home/software/jdk/1.8.0_51/bin/java
> 00600000-00601000 rw-p 00000000 00:1d 6377881433                         /blues/gpfs/home/software/jdk/1.8.0_51/bin/java
> 0087d000-008d2000 rw-p 00000000 00:00 0                                  [heap]
> 80000000-8ab00000 rw-p 00000000 00:00 0 
> 8ab00000-d5580000 ---p 00000000 00:00 0 
> d5580000-daa80000 rw-p 00000000 00:00 0 
> daa80000-100000000 ---p 00000000 00:00 0 
> 100000000-1000a0000 rw-p 00000000 00:00 0 
> 1000a0000-140000000 ---p 00000000 00:00 0 
> 2ac4fe097000-2ac4fe0b7000 r-xp 00000000 00:01 15121                      /lib64/ld-2.12.so
> 2ac4fe0b7000-2ac4fe0b8000 rw-p 00000000 00:00 0 
> 2ac4fe0ca000-2ac4fe0cb000 rw-p 00000000 00:00 0 
> 2ac4fe2b6000-2ac4fe2b7000 r--p 0001f000 00:01 15121                      /lib64/ld-2.12.so
> 2ac4fe2b7000-2ac4fe2b8000 rw-p 00020000 00:01 15121                      /lib64/ld-2.12.so
> 2ac4fe2b8000-2ac4fe2b9000 rw-p 00000000 00:00 0 
> 2ac4fe2b9000-2ac4fe2d0000 r-xp 00000000 00:01 15334                      /lib64/libpthread-2.12.so
> 2ac4fe2d0000-2ac4fe4d0000 ---p 00017000 00:01 15334                      /lib64/libpthread-2.12.so
> 2ac4fe4d0000-2ac4fe4d1000 r--p 00017000 00:01 15334                      /lib64/libpthread-2.12.so
> 2ac4fe4d1000-2ac4fe4d2000 rw-p 00018000 00:01 15334                      /lib64/libpthread-2.12.so
> 2ac4fe4d2000-2ac4fe4d6000 rw-p 00000000 00:00 0 
> 2ac4fe4d6000-2ac4fe4eb000 r-xp 00000000 00:1d 6377936031                 /blues/gpfs/home/software/jdk/1.8.0_51/lib/amd64/jli/libjli.so
> 2ac4fe4eb000-2ac4fe6eb000 ---p 00015000 00:1d 6377936031                 /blues/gpfs/home/software/jdk/1.8.0_51/lib/amd64/jli/libjli.so
> 2ac4fe6eb000-2ac4fe6ec000 rw-p 00015000 00:1d 6377936031                 /blues/gpfs/home/software/jdk/1.8.0_51/lib/amd64/jli/libjli.so
> 2ac4fe6ec000-2ac4fe6ee000 r-xp 00000000 00:01 15463                      /lib64/libdl-2.12.so
> 2ac4fe6ee000-2ac4fe8ee000 ---p 00002000 00:01 15463                      /lib64/libdl-2.12.so
> 2ac4fe8ee000-2ac4fe8ef000 r--p 00002000 00:01 15463                      /lib64/libdl-2.12.so
> 2ac4fe8ef000-2ac4fe8f0000 rw-p 00003000 00:01 15463                      /lib64/libdl-2.12.so
> 2ac4fe8f0000-2ac4fe8f1000 rw-p 00000000 00:00 0 
> 2ac4fe8f1000-2ac4fe9d1000 r-xp 00000000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fe9d1000-2ac4fe9d2000 r-xp 000e0000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fe9d2000-2ac4fe9d6000 r-xp 000e1000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fe9d6000-2ac4fe9d7000 r-xp 000e5000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fe9d7000-2ac4fe9da000 r-xp 000e6000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fe9da000-2ac4fe9db000 r-xp 000e9000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fe9db000-2ac4fe9dc000 r-xp 000ea000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fe9dc000-2ac4fe9dd000 r-xp 000eb000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fe9dd000-2ac4fea7b000 r-xp 000ec000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fea7b000-2ac4fec7b000 ---p 0018a000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fec7b000-2ac4fec7f000 r--p 0018a000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fec7f000-2ac4fec81000 rw-p 0018e000 00:01 15456                      /lib64/libc-2.12.so
> 2ac4fec81000-2ac4fec87000 rw-p 00000000 00:00 0 
> 2ac4fec87000-2ac4ff935000 r-xp 00000000 00:1d 5304218513                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/server/libjvm.so
> 2ac4ff935000-2ac4ffb35000 ---p 00cae000 00:1d 5304218513                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/server/libjvm.so
> 2ac4ffb35000-2ac4ffc0c000 rw-p 00cae000 00:1d 5304218513                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/server/libjvm.so
> 2ac4ffc0c000-2ac4ffc50000 rw-p 00000000 00:00 0 
> 2ac4ffc62000-2ac4ffce5000 r-xp 00000000 00:01 15470                      /lib64/libm-2.12.so
> 2ac4ffce5000-2ac4ffee4000 ---p 00083000 00:01 15470                      /lib64/libm-2.12.so
> 2ac4ffee4000-2ac4ffee5000 r--p 00082000 00:01 15470                      /lib64/libm-2.12.so
> 2ac4ffee5000-2ac4ffee6000 rw-p 00083000 00:01 15470                      /lib64/libm-2.12.so
> 2ac4ffee6000-2ac4ffee9000 ---p 00000000 00:00 0 
> 2ac4ffee9000-2ac4fffe7000 rw-p 00000000 00:00 0 
> 2ac500000000-2ac500515000 rw-p 00000000 00:00 0 
> 2ac500515000-2ac50051a000 rw-p 00000000 00:00 0 
> 2ac50051a000-2ac50059f000 rw-p 00000000 00:00 0 
> 2ac50059f000-2ac5005b0000 rw-p 00000000 00:00 0 
> 2ac5005b0000-2ac500e24000 rw-p 00000000 00:00 0 
> 2ac500e24000-2ac504000000 ---p 00000000 00:00 0 
> 2ac504000000-2ac504007000 r-xp 00000000 00:01 15240                      /lib64/librt-2.12.so
> 2ac504007000-2ac504206000 ---p 00007000 00:01 15240                      /lib64/librt-2.12.so
> 2ac504206000-2ac504207000 r--p 00006000 00:01 15240                      /lib64/librt-2.12.so
> 2ac504207000-2ac504208000 rw-p 00007000 00:01 15240                      /lib64/librt-2.12.so
> 2ac504208000-2ac504215000 r-xp 00000000 00:1d 5304218509                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libverify.so
> 2ac504215000-2ac504415000 ---p 0000d000 00:1d 5304218509                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libverify.so
> 2ac504415000-2ac504417000 rw-p 0000d000 00:1d 5304218509                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libverify.so
> 2ac504417000-2ac504441000 r-xp 00000000 00:1d 6378121578                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libjava.so
> 2ac504441000-2ac504641000 ---p 0002a000 00:1d 6378121578                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libjava.so
> 2ac504641000-2ac504643000 rw-p 0002a000 00:1d 6378121578                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libjava.so
> 2ac504643000-2ac504644000 ---p 00000000 00:00 0 
> 2ac504644000-2ac504645000 rw-p 00000000 00:00 0 
> 2ac504645000-2ac50467a000 r--s 00000000 00:01 208174                     /var/db/nscd/passwd
> 2ac50467a000-2ac504682000 rw-s 00000000 00:13 228154185                  /tmp/hsperfdata_spagnuolo/119842
> 2ac504682000-2ac50469c000 r-xp 00000000 00:1d 5304218510                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libzip.so
> 2ac50469c000-2ac50489c000 ---p 0001a000 00:1d 5304218510                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libzip.so
> 2ac50489c000-2ac50489d000 rw-p 0001a000 00:1d 5304218510                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/amd64/libzip.so
> 2ac50489d000-2ac504c2d000 rwxp 00000000 00:00 0 
> 2ac504c2d000-2ac51389d000 ---p 00000000 00:00 0 
> 2ac51389d000-2ac5138ac000 rw-p 00000000 00:00 0 
> 2ac5138ac000-2ac513c5d000 ---p 00000000 00:00 0 
> 2ac513c5d000-2ac513cb3000 rw-p 00000000 00:00 0 
> 2ac513cb3000-2ac513f07000 ---p 00000000 00:00 0 
> 2ac513f07000-2ac513f33000 rw-p 00000000 00:00 0 
> 2ac513f33000-2ac51405d000 ---p 00000000 00:00 0 
> 2ac51405d000-2ac5140b4000 rw-p 00000000 00:00 0 
> 2ac5140b4000-2ac514309000 ---p 00000000 00:00 0 
> 2ac514309000-2ac51430a000 ---p 00000000 00:00 0 
> 2ac51430a000-2ac51440a000 rw-p 00000000 00:00 0 
> 2ac51440a000-2ac51440b000 ---p 00000000 00:00 0 
> 2ac51440b000-2ac51450b000 rw-p 00000000 00:00 0 
> 2ac51450b000-2ac51450c000 ---p 00000000 00:00 0 
> 2ac51450c000-2ac51460c000 rw-p 00000000 00:00 0 
> 2ac51460c000-2ac51460d000 ---p 00000000 00:00 0 
> 2ac51460d000-2ac51470d000 rw-p 00000000 00:00 0 
> 2ac51470d000-2ac51470e000 ---p 00000000 00:00 0 
> 2ac51470e000-2ac51480e000 rw-p 00000000 00:00 0 
> 2ac51480e000-2ac51480f000 ---p 00000000 00:00 0 
> 2ac51480f000-2ac51490f000 rw-p 00000000 00:00 0 
> 2ac51490f000-2ac514910000 ---p 00000000 00:00 0 
> 2ac514910000-2ac514a10000 rw-p 00000000 00:00 0 
> 2ac514a10000-2ac514a11000 ---p 00000000 00:00 0 
> 2ac514a11000-2ac514b11000 rw-p 00000000 00:00 0 
> 2ac514b11000-2ac514b12000 ---p 00000000 00:00 0 
> 2ac514b12000-2ac514c12000 rw-p 00000000 00:00 0 
> 2ac514c12000-2ac514c13000 ---p 00000000 00:00 0 
> 2ac514c13000-2ac514d13000 rw-p 00000000 00:00 0 
> 2ac514d13000-2ac514d14000 ---p 00000000 00:00 0 
> 2ac514d14000-2ac514e14000 rw-p 00000000 00:00 0 
> 2ac514e14000-2ac514e15000 ---p 00000000 00:00 0 
> 2ac514e15000-2ac514f15000 rw-p 00000000 00:00 0 
> 2ac514f15000-2ac514f16000 ---p 00000000 00:00 0 
> 2ac514f16000-2ac515016000 rw-p 00000000 00:00 0 
> 2ac515016000-2ac515019000 ---p 00000000 00:00 0 
> 2ac515019000-2ac515117000 rw-p 00000000 00:00 0 
> 2ac515117000-2ac5154f0000 r--s 0381d000 00:1d 470717                     /blues/gpfs/home/spagnuolo/dmason/dmason.jar
> 2ac5154f0000-2ac5154f4000 r--s 00095000 00:1d 5304730694                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/jsse.jar
> 2ac5154f4000-2ac515519000 r-xp 00000000 00:1d 5368895802                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmpi_java.so.20.0.1
> 2ac515519000-2ac515718000 ---p 00025000 00:1d 5368895802                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmpi_java.so.20.0.1
> 2ac515718000-2ac51571a000 rw-p 00024000 00:1d 5368895802                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmpi_java.so.20.0.1
> 2ac51571a000-2ac5157dc000 r-xp 00000000 00:1d 3221460736                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmpi.so.20.0.1
> 2ac5157dc000-2ac5159dc000 ---p 000c2000 00:1d 3221460736                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmpi.so.20.0.1
> 2ac5159dc000-2ac5159ec000 rw-p 000c2000 00:1d 3221460736                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmpi.so.20.0.1
> 2ac5159ec000-2ac5159fe000 rw-p 00000000 00:00 0 
> 2ac5159fe000-2ac515a79000 r-xp 00000000 00:1d 2147743269                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libopen-rte.so.20.0.0
> 2ac515a79000-2ac515c78000 ---p 0007b000 00:1d 2147743269                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libopen-rte.so.20.0.0
> 2ac515c78000-2ac515c7d000 rw-p 0007a000 00:1d 2147743269                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libopen-rte.so.20.0.0
> 2ac515c7d000-2ac515c7e000 rw-p 00000000 00:00 0 
> 2ac515c7e000-2ac515d5f000 r-xp 00000000 00:1d 1074537938                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libopen-pal.so.20.1.0
> 2ac515d5f000-2ac515f5f000 ---p 000e1000 00:1d 1074537938                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libopen-pal.so.20.1.0
> 2ac515f5f000-2ac515f68000 rw-p 000e1000 00:1d 1074537938                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libopen-pal.so.20.1.0
> 2ac515f68000-2ac515f6f000 rw-p 00000000 00:00 0 
> 2ac515f6f000-2ac515f79000 r-xp 00000000 00:01 102846                     /usr/lib64/libnuma.so.1
> 2ac515f79000-2ac516179000 ---p 0000a000 00:01 102846                     /usr/lib64/libnuma.so.1
> 2ac516179000-2ac51617a000 rw-p 0000a000 00:01 102846                     /usr/lib64/libnuma.so.1
> 2ac51617a000-2ac516182000 r-xp 00000000 00:01 104054                     /usr/lib64/libpciaccess.so.0.11.1
> 2ac516182000-2ac516382000 ---p 00008000 00:01 104054                     /usr/lib64/libpciaccess.so.0.11.1
> 2ac516382000-2ac516383000 rw-p 00008000 00:01 104054                     /usr/lib64/libpciaccess.so.0.11.1
> 2ac516395000-2ac516397000 r-xp 00000000 00:01 15336                      /lib64/libutil-2.12.so
> 2ac516397000-2ac516596000 ---p 00002000 00:01 15336                      /lib64/libutil-2.12.so
> 2ac516596000-2ac516597000 r--p 00001000 00:01 15336                      /lib64/libutil-2.12.so
> 2ac516597000-2ac516598000 rw-p 00002000 00:01 15336                      /lib64/libutil-2.12.so
> 2ac516598000-2ac516599000 r-xp 00000000 00:1d 2147743232                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_patcher_overwrite.so
> 2ac516599000-2ac516799000 ---p 00001000 00:1d 2147743232                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_patcher_overwrite.so
> 2ac516799000-2ac51679a000 rw-p 00001000 00:1d 2147743232                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_patcher_overwrite.so
> 2ac51679a000-2ac51679d000 r-xp 00000000 00:1d 2147743243                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_shmem_mmap.so
> 2ac51679d000-2ac51699c000 ---p 00003000 00:1d 2147743243                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_shmem_mmap.so
> 2ac51699c000-2ac51699d000 rw-p 00002000 00:1d 2147743243                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_shmem_mmap.so
> 2ac51699d000-2ac51699e000 r-xp 00000000 00:1d 2147743240                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_sec_basic.so
> 2ac51699e000-2ac516b9e000 ---p 00001000 00:1d 2147743240                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_sec_basic.so
> 2ac516b9e000-2ac516b9f000 rw-p 00001000 00:1d 2147743240                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_sec_basic.so
> 2ac516b9f000-2ac516ba3000 r-xp 00000000 00:1d 2147743297                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_ess_pmi.so
> 2ac516ba3000-2ac516da2000 ---p 00004000 00:1d 2147743297                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_ess_pmi.so
> 2ac516da2000-2ac516da3000 rw-p 00003000 00:1d 2147743297                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_ess_pmi.so
> 2ac516da3000-2ac516e01000 r-xp 00000000 00:1d 2147743234                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_pmix_pmix112.so
> 2ac516e01000-2ac517000000 ---p 0005e000 00:1d 2147743234                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_pmix_pmix112.so
> 2ac517000000-2ac517002000 rw-p 0005d000 00:1d 2147743234                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_pmix_pmix112.so
> 2ac517002000-2ac517004000 rw-p 00000000 00:00 0 
> 2ac517004000-2ac517005000 ---p 00000000 00:00 0 
> 2ac517005000-2ac517306000 rw-p 00000000 00:00 0 
> 2ac518000000-2ac518021000 rw-p 00000000 00:00 0 
> 2ac518021000-2ac51c000000 ---p 00000000 00:00 0 
> 2ac51c000000-2ac51c021000 rw-p 00000000 00:00 0 
> 2ac51c021000-2ac520000000 ---p 00000000 00:00 0 
> 2ac520000000-2ac520021000 rw-p 00000000 00:00 0 
> 2ac520021000-2ac524000000 ---p 00000000 00:00 0 
> 2ac528000000-2ac528021000 rw-p 00000000 00:00 0 
> 2ac528021000-2ac52c000000 ---p 00000000 00:00 0 
> 2ac530000000-2ac530021000 rw-p 00000000 00:00 0 
> 2ac530021000-2ac534000000 ---p 00000000 00:00 0 
> 2ac538000000-2ac538021000 rw-p 00000000 00:00 0 
> 2ac538021000-2ac53c000000 ---p 00000000 00:00 0 
> 2ac540000000-2ac540021000 rw-p 00000000 00:00 0 
> 2ac540021000-2ac544000000 ---p 00000000 00:00 0 
> 2ac548000000-2ac548021000 rw-p 00000000 00:00 0 
> 2ac548021000-2ac54c000000 ---p 00000000 00:00 0 
> 2ac54c000000-2ac550000000 rw-p 00000000 00:00 0 
> 2ac550000000-2ac550021000 rw-p 00000000 00:00 0 
> 2ac550021000-2ac554000000 ---p 00000000 00:00 0 
> 2ac554000000-2ac554928000 rw-p 00000000 00:00 0 
> 2ac554928000-2ac554c28000 ---p 00000000 00:00 0 
> 2ac554c28000-2ac554cf9000 rw-p 00000000 00:00 0 
> 2ac554cf9000-2ac554ed1000 r--s 03d0c000 00:1d 6377936006                 /blues/gpfs/home/software/jdk/1.8.0_51/jre/lib/rt.jar
> 2ac554ed1000-2ac55794a000 rw-p 00000000 00:00 0 
> 2ac55794a000-2ac55794b000 ---p 00000000 00:00 0 
> 2ac55794b000-2ac557a4b000 rw-p 00000000 00:00 0 
> 2ac558000000-2ac558021000 rw-p 00000000 00:00 0 
> 2ac558021000-2ac55c000000 ---p 00000000 00:00 0 
> 2ac560000000-2ac560021000 rw-p 00000000 00:00 0 
> 2ac560021000-2ac564000000 ---p 00000000 00:00 0 
> 2ac568000000-2ac568021000 rw-p 00000000 00:00 0 
> 2ac568021000-2ac56c000000 ---p 00000000 00:00 0 
> 2ac570000000-2ac570021000 rw-p 00000000 00:00 0 
> 2ac570021000-2ac574000000 ---p 00000000 00:00 0 
> 2ac578000000-2ac578021000 rw-p 00000000 00:00 0 
> 2ac578021000-2ac57c000000 ---p 00000000 00:00 0 
> 2ac57c000000-2ac57c021000 rw-p 00000000 00:00 0 
> 2ac57c021000-2ac580000000 ---p 00000000 00:00 0 
> 2ac580000000-2ac580003000 ---p 00000000 00:00 0 
> 2ac580003000-2ac580101000 rw-p 00000000 00:00 0 
> 2ac580101000-2ac580104000 ---p 00000000 00:00 0 
> 2ac580104000-2ac580202000 rw-p 00000000 00:00 0 
> 2ac580202000-2ac580205000 ---p 00000000 00:00 0 
> 2ac580205000-2ac580303000 rw-p 00000000 00:00 0 
> 2ac580303000-2ac580306000 ---p 00000000 00:00 0 
> 2ac580306000-2ac580404000 rw-p 00000000 00:00 0 
> 2ac580404000-2ac580407000 ---p 00000000 00:00 0 
> 2ac580407000-2ac580505000 rw-p 00000000 00:00 0 
> 2ac580505000-2ac580508000 ---p 00000000 00:00 0 
> 2ac580508000-2ac580606000 rw-p 00000000 00:00 0 
> 2ac584000000-2ac584021000 rw-p 00000000 00:00 0 
> 2ac584021000-2ac588000000 ---p 00000000 00:00 0 
> 2ac588000000-2ac588021000 rw-p 00000000 00:00 0 
> 2ac588021000-2ac58c000000 ---p 00000000 00:00 0 
> 2ac58c000000-2ac58c021000 rw-p 00000000 00:00 0 
> 2ac58c021000-2ac590000000 ---p 00000000 00:00 0 
> 2ac590000000-2ac5900fd000 rw-p 00000000 00:00 0 
> 2ac5900fd000-2ac594000000 ---p 00000000 00:00 0 
> 2ac594000000-2ac594080000 rw-p 00000000 00:00 0 
> 2ac594080000-2ac598000000 ---p 00000000 00:00 0 
> 2ac598000000-2ac5982b4000 rw-p 00000000 00:00 0 
> 2ac5982b4000-2ac59c000000 ---p 00000000 00:00 0 
> 2ac59c000000-2ac59c003000 ---p 00000000 00:00 0 
> 2ac59c003000-2ac59c101000 rw-p 00000000 00:00 0 
> 2ac59c101000-2ac59c104000 ---p 00000000 00:00 0 
> 2ac59c104000-2ac59c202000 rw-p 00000000 00:00 0 
> 2ac59c202000-2ac59c205000 ---p 00000000 00:00 0 
> 2ac59c205000-2ac59c303000 rw-p 00000000 00:00 0 
> 2ac59c303000-2ac59c306000 ---p 00000000 00:00 0 
> 2ac59c306000-2ac59c404000 rw-p 00000000 00:00 0 
> 2ac5a0000000-2ac5a00f0000 rw-p 00000000 00:00 0 
> 2ac5a00f0000-2ac5a4000000 ---p 00000000 00:00 0 
> 2ac5a4000000-2ac5a40e2000 rw-p 00000000 00:00 0 
> 2ac5a40e2000-2ac5a8000000 ---p 00000000 00:00 0 
> 2ac5a8000000-2ac5a80fd000 rw-p 00000000 00:00 0 
> 2ac5a80fd000-2ac5ac000000 ---p 00000000 00:00 0 
> 2ac5ac000000-2ac5ac0c6000 rw-p 00000000 00:00 0 
> 2ac5ac0c6000-2ac5b0000000 ---p 00000000 00:00 0 
> 2ac5b0000000-2ac5b0003000 ---p 00000000 00:00 0 
> 2ac5b0003000-2ac5b0101000 rw-p 00000000 00:00 0 
> 2ac5b0101000-2ac5b0104000 ---p 00000000 00:00 0 
> 2ac5b0104000-2ac5b0202000 rw-p 00000000 00:00 0 
> 2ac5b4000000-2ac5b42fc000 rw-p 00000000 00:00 0 
> 2ac5b42fc000-2ac5b8000000 ---p 00000000 00:00 0 
> 2ac5b8000000-2ac5b80bc000 rw-p 00000000 00:00 0 
> 2ac5b80bc000-2ac5bc000000 ---p 00000000 00:00 0 
> 2ac5bc000000-2ac5bc003000 ---p 00000000 00:00 0 
> 2ac5bc003000-2ac5bc101000 rw-p 00000000 00:00 0 
> 2ac5bc101000-2ac5bc104000 ---p 00000000 00:00 0 
> 2ac5bc104000-2ac5bc202000 rw-p 00000000 00:00 0 
> 2ac5c0000000-2ac5c00db000 rw-p 00000000 00:00 0 
> 2ac5c00db000-2ac5c4000000 ---p 00000000 00:00 0 
> 2ac5c4000000-2ac5c40d8000 rw-p 00000000 00:00 0 
> 2ac5c40d8000-2ac5c8000000 ---p 00000000 00:00 0 
> 2ac5c8000000-2ac5c8003000 ---p 00000000 00:00 0 
> 2ac5c8003000-2ac5c8101000 rw-p 00000000 00:00 0 
> 2ac5cc000000-2ac5cc0b2000 rw-p 00000000 00:00 0 
> 2ac5cc0b2000-2ac5d0000000 ---p 00000000 00:00 0 
> 2ac5d0000000-2ac5d0021000 rw-p 00000000 00:00 0 
> 2ac5d0021000-2ac5d4000000 ---p 00000000 00:00 0 
> 2ac5d4000000-2ac5d4001000 ---p 00000000 00:00 0 
> 2ac5d4001000-2ac5d4101000 rw-p 00000000 00:00 0 
> 2ac5d4101000-2ac5d4102000 ---p 00000000 00:00 0 
> 2ac5d4102000-2ac5d4302000 rw-p 00000000 00:00 0 
> 2ac5d8000000-2ac5d802e000 rw-p 00000000 00:00 0 
> 2ac5d802e000-2ac5dc000000 ---p 00000000 00:00 0 
> 2ac5dc000000-2ac5dc021000 rw-p 00000000 00:00 0 
> 2ac5dc021000-2ac5e0000000 ---p 00000000 00:00 0 
> 2ac5e0000000-2ac5e0002000 r-xp 00000000 00:1d 2148255395                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_state_app.so
> 2ac5e0002000-2ac5e0201000 ---p 00002000 00:1d 2148255395                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_state_app.so
> 2ac5e0201000-2ac5e0202000 rw-p 00001000 00:1d 2148255395                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_state_app.so
> 2ac5e0202000-2ac5e0204000 r-xp 00000000 00:1d 2147743285                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_errmgr_default_app.so
> 2ac5e0204000-2ac5e0403000 ---p 00002000 00:1d 2147743285                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_errmgr_default_app.so
> 2ac5e0403000-2ac5e0404000 rw-p 00001000 00:1d 2147743285                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_errmgr_default_app.so
> 2ac5e0404000-2ac5e0407000 r-xp 00000000 00:1d 2147743310                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_grpcomm_direct.so
> 2ac5e0407000-2ac5e0606000 ---p 00003000 00:1d 2147743310                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_grpcomm_direct.so
> 2ac5e0606000-2ac5e0607000 rw-p 00002000 00:1d 2147743310                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_grpcomm_direct.so
> 2ac5e0607000-2ac5e0609000 r-xp 00000000 00:1d 1074537955                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_allocator_basic.so
> 2ac5e0609000-2ac5e0808000 ---p 00002000 00:1d 1074537955                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_allocator_basic.so
> 2ac5e0808000-2ac5e0809000 rw-p 00001000 00:1d 1074537955                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_allocator_basic.so
> 2ac5e0809000-2ac5e080b000 r-xp 00000000 00:1d 1074537957                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_allocator_bucket.so
> 2ac5e080b000-2ac5e0a0a000 ---p 00002000 00:1d 1074537957                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_allocator_bucket.so
> 2ac5e0a0a000-2ac5e0a0b000 rw-p 00001000 00:1d 1074537957                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_allocator_bucket.so
> 2ac5e0a0e000-2ac5e0a21000 r-xp 00000000 00:1d 2147743328                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_tcp.so
> 2ac5e0a21000-2ac5e0c21000 ---p 00013000 00:1d 2147743328                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_tcp.so
> 2ac5e0c21000-2ac5e0c22000 rw-p 00013000 00:1d 2147743328                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_tcp.so
> 2ac5e0c22000-2ac5e0c32000 r-xp 00000000 00:1d 2147743332                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_ud.so
> 2ac5e0c32000-2ac5e0e32000 ---p 00010000 00:1d 2147743332                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_ud.so
> 2ac5e0e32000-2ac5e0e33000 rw-p 00010000 00:1d 2147743332                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_ud.so
> 2ac5e0e33000-2ac5e0e37000 r-xp 00000000 00:1d 1074537949                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmca_common_verbs.so.20.0.0
> 2ac5e0e37000-2ac5e1036000 ---p 00004000 00:1d 1074537949                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmca_common_verbs.so.20.0.0
> 2ac5e1036000-2ac5e1037000 rw-p 00003000 00:1d 1074537949                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmca_common_verbs.so.20.0.0
> 2ac5e1037000-2ac5e1048000 r-xp 00000000 00:01 106741                     /usr/lib64/libibverbs.so.1.0.0
> 2ac5e1048000-2ac5e1248000 ---p 00011000 00:01 106741                     /usr/lib64/libibverbs.so.1.0.0
> 2ac5e1248000-2ac5e1249000 rw-p 00011000 00:01 106741                     /usr/lib64/libibverbs.so.1.0.0
> 2ac5e125b000-2ac5e12a8000 r-xp 00000000 00:01 15002                      /lib64/libnl.so.1.1.4
> 2ac5e12a8000-2ac5e14a7000 ---p 0004d000 00:01 15002                      /lib64/libnl.so.1.1.4
> 2ac5e14a7000-2ac5e14ad000 rw-p 0004c000 00:01 15002                      /lib64/libnl.so.1.1.4
> 2ac5e14ad000-2ac5e14b9000 r-xp 00000000 00:1d 2147743334                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_usock.so
> 2ac5e14b9000-2ac5e16b9000 ---p 0000c000 00:1d 2147743334                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_usock.so
> 2ac5e16b9000-2ac5e16ba000 rw-p 0000c000 00:1d 2147743334                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_oob_usock.so
> 2ac5e16ba000-2ac5e16c1000 r-xp 00000000 00:01 108072                     /usr/lib64/libmthca-rdmav2.so
> 2ac5e16c1000-2ac5e18c1000 ---p 00007000 00:01 108072                     /usr/lib64/libmthca-rdmav2.so
> 2ac5e18c1000-2ac5e18c2000 rw-p 00007000 00:01 108072                     /usr/lib64/libmthca-rdmav2.so
> 2ac5e18c2000-2ac5e18c7000 r-xp 00000000 00:01 104505                     /usr/lib64/libcxgb3-rdmav2.so
> 2ac5e18c7000-2ac5e1ac6000 ---p 00005000 00:01 104505                     /usr/lib64/libcxgb3-rdmav2.so
> 2ac5e1ac6000-2ac5e1ac7000 rw-p 00004000 00:01 104505                     /usr/lib64/libcxgb3-rdmav2.so
> 2ac5e1ac7000-2ac5e1ace000 r-xp 00000000 00:01 108152                     /usr/lib64/libmlx4-rdmav2.so
> 2ac5e1ace000-2ac5e1cce000 ---p 00007000 00:01 108152                     /usr/lib64/libmlx4-rdmav2.so
> 2ac5e1cce000-2ac5e1ccf000 rw-p 00007000 00:01 108152                     /usr/lib64/libmlx4-rdmav2.so
> 2ac5e1ccf000-2ac5e1cd4000 r-xp 00000000 00:01 103355                     /usr/lib64/libnes-rdmav2.so
> 2ac5e1cd4000-2ac5e1ed3000 ---p 00005000 00:01 103355                     /usr/lib64/libnes-rdmav2.so
> 2ac5e1ed3000-2ac5e1ed4000 rw-p 00004000 00:01 103355                     /usr/lib64/libnes-rdmav2.so
> 2ac5e1ed4000-2ac5e1ed7000 r-xp 00000000 00:01 108141                     /usr/lib64/libipathverbs-rdmav2.so
> 2ac5e1ed7000-2ac5e20d7000 ---p 00003000 00:01 108141                     /usr/lib64/libipathverbs-rdmav2.so
> 2ac5e20d7000-2ac5e20d8000 rw-p 00003000 00:01 108141                     /usr/lib64/libipathverbs-rdmav2.so
> 2ac5e20d8000-2ac5e2199000 rw-s 18224000 00:05 206254                     /dev/infiniband/uverbs0
> 2ac5e2199000-2ac5e225a000 rw-s 182e5000 00:05 206254                     /dev/infiniband/uverbs0
> 2ac5e225a000-2ac5e228b000 rw-s 183a6000 00:05 206254                     /dev/infiniband/uverbs0
> 2ac5e228b000-2ac5e228c000 rw-p 00000000 00:00 0 
> 2ac5e228c000-2ac5e248c000 rw-p 00000000 00:00 0 
> 2ac5e248c000-2ac5e248d000 rw-p 00000000 00:00 0 
> 2ac5e248d000-2ac5e2490000 r-xp 00000000 00:1d 2148255378                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_rml_oob.so
> 2ac5e2490000-2ac5e2690000 ---p 00003000 00:1d 2148255378                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_rml_oob.so
> 2ac5e2690000-2ac5e2691000 rw-p 00003000 00:1d 2148255378                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_rml_oob.so
> 2ac5e2691000-2ac5e2694000 r-xp 00000000 00:1d 2148255386                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_routed_radix.so
> 2ac5e2694000-2ac5e2894000 ---p 00003000 00:1d 2148255386                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_routed_radix.so
> 2ac5e2894000-2ac5e2895000 rw-p 00003000 00:1d 2148255386                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_routed_radix.so
> 2ac5e2895000-2ac5e2899000 r-xp 00000000 00:1d 2147743238                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_rcache_vma.so
> 2ac5e2899000-2ac5e2a98000 ---p 00004000 00:1d 2147743238                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_rcache_vma.so
> 2ac5e2a98000-2ac5e2a99000 rw-p 00003000 00:1d 2147743238                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_rcache_vma.so
> 2ac5e2a99000-2ac5e2a9d000 r-xp 00000000 00:1d 1074537979                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mpool_grdma.so
> 2ac5e2a9d000-2ac5e2c9c000 ---p 00004000 00:1d 1074537979                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mpool_grdma.so
> 2ac5e2c9c000-2ac5e2c9d000 rw-p 00003000 00:1d 1074537979                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mpool_grdma.so
> 2ac5e2c9d000-2ac5e2c9f000 r-xp 00000000 00:1d 1074537982                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mpool_sm.so
> 2ac5e2c9f000-2ac5e2e9e000 ---p 00002000 00:1d 1074537982                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mpool_sm.so
> 2ac5e2e9e000-2ac5e2e9f000 rw-p 00001000 00:1d 1074537982                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mpool_sm.so
> 2ac5e2e9f000-2ac5e2ea1000 r-xp 00000000 00:1d 1074537944                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmca_common_sm.so.20.0.0
> 2ac5e2ea1000-2ac5e30a0000 ---p 00002000 00:1d 1074537944                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmca_common_sm.so.20.0.0
> 2ac5e30a0000-2ac5e30a1000 rw-p 00001000 00:1d 1074537944                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/libmca_common_sm.so.20.0.0
> 2ac5e30a1000-2ac5e30a5000 r-xp 00000000 00:1d 4295690432                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_bml_r2.so
> 2ac5e30a5000-2ac5e32a4000 ---p 00004000 00:1d 4295690432                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_bml_r2.so
> 2ac5e32a4000-2ac5e32a5000 rw-p 00003000 00:1d 4295690432                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_bml_r2.so
> 2ac5e32a5000-2ac5e32a8000 r-xp 00000000 00:1d 1074537959                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_self.so
> 2ac5e32a8000-2ac5e34a8000 ---p 00003000 00:1d 1074537959                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_self.so
> 2ac5e34a8000-2ac5e34a9000 rw-p 00003000 00:1d 1074537959                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_self.so
> 2ac5e34a9000-2ac5e34db000 r-xp 00000000 00:1d 1074537967                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_openib.so
> 2ac5e34db000-2ac5e36da000 ---p 00032000 00:1d 1074537967                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_openib.so
> 2ac5e36da000-2ac5e36dd000 rw-p 00031000 00:1d 1074537967                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_openib.so
> 2ac5e36dd000-2ac5e36f1000 r-xp 00000000 00:01 104519                     /usr/lib64/librdmacm.so.1.0.0
> 2ac5e36f1000-2ac5e38f0000 ---p 00014000 00:01 104519                     /usr/lib64/librdmacm.so.1.0.0
> 2ac5e38f0000-2ac5e38f1000 rw-p 00013000 00:01 104519                     /usr/lib64/librdmacm.so.1.0.0
> 2ac5e38f1000-2ac5e38f2000 rw-p 00000000 00:00 0 
> 2ac5e38f2000-2ac5e38f8000 r-xp 00000000 00:1d 1074537971                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_sm.so
> 2ac5e38f8000-2ac5e3af8000 ---p 00006000 00:1d 1074537971                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_sm.so
> 2ac5e3af8000-2ac5e3af9000 rw-p 00006000 00:1d 1074537971                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_sm.so
> 2ac5e3af9000-2ac5e3b06000 r-xp 00000000 00:1d 1074537974                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_tcp.so
> 2ac5e3b06000-2ac5e3d06000 ---p 0000d000 00:1d 1074537974                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_tcp.so
> 2ac5e3d06000-2ac5e3d08000 rw-p 0000d000 00:1d 1074537974                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_tcp.so
> 2ac5e3d08000-2ac5e3d88000 rw-p 00000000 00:00 0 
> 2ac5e3d88000-2ac5e3d8e000 r-xp 00000000 00:1d 1074537977                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_vader.so
> 2ac5e3d8e000-2ac5e3f8e000 ---p 00006000 00:1d 1074537977                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_vader.so
> 2ac5e3f8e000-2ac5e3f90000 rw-p 00006000 00:1d 1074537977                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_btl_vader.so
> 2ac5e3f90000-2ac5e3f97000 rw-s 183db000 00:05 206254                     /dev/infiniband/uverbs0
> 2ac5e3f97000-2ac5e3f9e000 rw-s 183e9000 00:05 206254                     /dev/infiniband/uverbs0
> 2ac5e3f9e000-2ac5e3fa3000 rw-s 183f7000 00:05 206254                     /dev/infiniband/uverbs0
> 2ac5e3fa3000-2ac5e43a4000 rw-s 00000000 00:10 228154649                  /scratch/1425304.bmgt1.lcrc.anl.gov/openmpi-sessions-7933@b17_0/45864/1/24/vader_segment.b17.8
> 2ac5e43a4000-2ac5e43aa000 r-xp 00000000 00:1d 4295690473                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_pml_cm.so
> 2ac5e43aa000-2ac5e45aa000 ---p 00006000 00:1d 4295690473                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_pml_cm.so
> 2ac5e45aa000-2ac5e45ab000 rw-p 00006000 00:1d 4295690473                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_pml_cm.so
> 2ac5e47c8000-2ac5e47cf000 r-xp 00000000 00:1d 4295690486                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_vprotocol_pessimist.so
> 2ac5e47cf000-2ac5e49ce000 ---p 00007000 00:1d 4295690486                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_vprotocol_pessimist.so
> 2ac5e49ce000-2ac5e49cf000 rw-p 00006000 00:1d 4295690486                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_vprotocol_pessimist.so
> 2ac5e49cf000-2ac5e49d4000 r-xp 00000000 00:1d 4295690465                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mtl_psm.so
> 2ac5e49d4000-2ac5e4bd3000 ---p 00005000 00:1d 4295690465                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mtl_psm.so
> 2ac5e4bd3000-2ac5e4bd4000 rw-p 00004000 00:1d 4295690465                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_mtl_psm.so
> 2ac5e4bd4000-2ac5e4c25000 r-xp 00000000 00:01 103639                     /usr/lib64/libpsm_infinipath.so.1.15
> 2ac5e4c25000-2ac5e4e25000 ---p 00051000 00:01 103639                     /usr/lib64/libpsm_infinipath.so.1.15
> 2ac5e4e25000-2ac5e4e27000 rw-p 00051000 00:01 103639                     /usr/lib64/libpsm_infinipath.so.1.15
> 2ac5e4e27000-2ac5e4e28000 rw-p 00000000 00:00 0 
> 2ac5e4e28000-2ac5e4e35000 r-xp 00000000 00:01 104503                     /usr/lib64/libinfinipath.so.4.0
> 2ac5e4e35000-2ac5e5035000 ---p 0000d000 00:01 104503                     /usr/lib64/libinfinipath.so.4.0
> 2ac5e5035000-2ac5e5036000 rw-p 0000d000 00:01 104503                     /usr/lib64/libinfinipath.so.4.0
> 2ac5e5048000-2ac5e504c000 r-xp 00000000 00:01 15293                      /lib64/libuuid.so.1.3.0
> 2ac5e504c000-2ac5e524b000 ---p 00004000 00:01 15293                      /lib64/libuuid.so.1.3.0
> 2ac5e524b000-2ac5e524c000 rw-p 00003000 00:01 15293                      /lib64/libuuid.so.1.3.0
> 2ac5e524c000-2ac5e5259000 r-xp 00000000 00:1d 4295690434                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_basic.so
> 2ac5e5259000-2ac5e5459000 ---p 0000d000 00:1d 4295690434                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_basic.so
> 2ac5e5459000-2ac5e545a000 rw-p 0000d000 00:1d 4295690434                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_basic.so
> 2ac5e545a000-2ac5e545e000 r-xp 00000000 00:1d 4295690436                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_inter.so
> 2ac5e545e000-2ac5e565d000 ---p 00004000 00:1d 4295690436                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_inter.so
> 2ac5e565d000-2ac5e565e000 rw-p 00003000 00:1d 4295690436                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_inter.so
> 2ac5e565e000-2ac5e5678000 r-xp 00000000 00:1d 4295690438                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_libnbc.so
> 2ac5e5678000-2ac5e5877000 ---p 0001a000 00:1d 4295690438                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_libnbc.so
> 2ac5e5877000-2ac5e5878000 rw-p 00019000 00:1d 4295690438                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_libnbc.so
> 2ac5e5878000-2ac5e587a000 r-xp 00000000 00:1d 4295690440                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_self.so
> 2ac5e587a000-2ac5e5a7a000 ---p 00002000 00:1d 4295690440                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_self.so
> 2ac5e5a7a000-2ac5e5a7b000 rw-p 00002000 00:1d 4295690440                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_self.so
> 2ac5e5a7b000-2ac5e5a81000 r-xp 00000000 00:1d 4295690444                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_sm.so
> 2ac5e5a81000-2ac5e5c80000 ---p 00006000 00:1d 4295690444                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_sm.so
> 2ac5e5c80000-2ac5e5c81000 rw-p 00005000 00:1d 4295690444                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_sm.so
> 2ac5e5c81000-2ac5e5c91000 r-xp 00000000 00:1d 4295690446                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_tuned.so
> 2ac5e5c91000-2ac5e5e90000 ---p 00010000 00:1d 4295690446                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_tuned.so
> 2ac5e5e90000-2ac5e5e92000 rw-p 0000f000 00:1d 4295690446                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_coll_tuned.so
> 2ac5e5e92000-2ac5e5e97000 r-xp 00000000 00:1d 4295690467                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_sm.so
> 2ac5e5e97000-2ac5e6097000 ---p 00005000 00:1d 4295690467                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_sm.so
> 2ac5e6097000-2ac5e6099000 rw-p 00005000 00:1d 4295690467                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_sm.so
> 2ac5e6099000-2ac5e60b1000 r-xp 00000000 00:1d 4295690469                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_pt2pt.so
> 2ac5e60b1000-2ac5e62b0000 ---p 00018000 00:1d 4295690469                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_pt2pt.so
> 2ac5e62b0000-2ac5e62b2000 rw-p 00017000 00:1d 4295690469                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_pt2pt.so
> 2ac5e62b2000-2ac5e62cf000 r-xp 00000000 00:1d 4295690471                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_rdma.so
> 2ac5e62cf000-2ac5e64cf000 ---p 0001d000 00:1d 4295690471                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_rdma.so
> 2ac5e64cf000-2ac5e64d0000 rw-p 0001d000 00:1d 4295690471                 /blues/gpfs/home/spagnuolo/openmpi-2.0.1/build/lib/openmpi/mca_osc_rdma.so
> 2ac5e64d0000-2ac5e64d2000 rw-s d0e70000 00:05 204175                     /dev/ipath
> 2ac5e64d2000-2ac5e64da000 -w-s d0d28000 00:05 204175                     /dev/ipath
> 2ac5e64da000-2ac5e64db000 r--s ffffc9002491a000 00:05 204175             /dev/ipath
> 2ac5e64db000-2ac5e655b000 r--s 6fa700000 00:05 204175                    /dev/ipath
> 2ac5e655b000-2ac5e695b000 r--s 733d08000 00:05 204175                    /dev/ipath
> 2ac5e695b000-2ac5e695c000 r--s 7867d7000 00:05 204175                    /dev/ipath
> 2ac5e695c000-2ac5e7160000 rw-p 00000000 00:00 0 
> 2ac5e7160000-2ac5e7195000 r--s 00000000 00:01 208177                     /var/db/nscd/hosts
> 2ac5e7195000-2ac5e7196000 ---p 00000000 00:00 0 
> 2ac5e7196000-2ac5e7396000 rw-p 00000000 00:00 0 
> 2ac5e7396000-2ac6460b8000 rw-s 00000000 00:05 228154357                  /dev/shm/psm_shm.0a076f83-58eb-dd40-f643-ac3fc2e710af
> 7ffc4db68000-7ffc4db7e000 rw-p 00000000 00:00 0                          [stack]
> 7ffc4db96000-7ffc4db97000 r-xp 00000000 00:00 0                          [vdso]
> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
> 
> VM Arguments:
> jvm_args: -Djava.library.path=/home/spagnuolo/openmpi-2.0.1/build/lib -Xmx2048m -Xms256m 
> java_command: /home/spagnuolo/dmason/dmason.jar 10000 10000 10 32 32 1000000 100 b16
> java_class_path (initial): /home/spagnuolo/dmason/dmason.jar
> Launcher Type: SUN_STANDARD
> 
> Environment Variables:
> JAVA_HOME=/soft/jdk/1.8.0_51
> JRE_HOME=/soft/jdk/1.8.0_51/jre
> PATH=/home/spagnuolo/openmpi-2.0.1/build/bin:/home/spagnuolo/openmpi-2.0.1/build/bin:/home/spagnuolo/openmpi-2.0.1/build/bin:/soft/jdk/1.8.0_51/bin:/soft/jdk/1.8.0_51/jre/bin:/soft/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/bin:/software/mvapich2-intel-psm-1.9.5/bin:/soft/intel/13.1.3/bin:/soft/softenv/1.6.2/bin:/soft/lcrc/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/home/spagnuolo/bin:.:/home/spagnuolo/bin
> LD_LIBRARY_PATH=/home/spagnuolo/openmpi-2.0.1/build/lib:/home/spagnuolo/openmpi-2.0.1/build/lib:/home/spagnuolo/openmpi-2.0.1/build/lib:/soft/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/lib64:/soft/spack/opt/spack/linux-x86_64/gcc-4.4.7/gcc-5.3.0-fygfl7rvyuiteto27dlhmilp5cstw2o2/libexec/gcc/x86_64-unknown-linux-gnu/5.3.0:/software/mvapich2-intel-psm-1.9.5/lib:/soft/intel/13.1.3/lib/intel64:/soft/intel/13.1.3/composer_xe_2013.5.192/mpirt/lib/intel64:/soft/intel/13.1.3/ipp/lib/intel64:/soft/intel/13.1.3/tbb/lib/intel64/gcc4.4:/soft/lcrc/lib:/usr/lib64:/usr/lib
> SHELL=/bin/bash
> ARCH=linux-rhel6-x86_64
> 
> Signal Handlers:
> SIGSEGV: [libjvm.so+0xaad540], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART, flags was changed from 0x10000004, consider using jsig library
> SIGBUS: [libjvm.so+0xaad540], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART, flags was changed from 0x10000004, consider using jsig library
> SIGFPE: [libjvm.so+0x90b6b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
> SIGPIPE: [libjvm.so+0x90b6b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
> SIGXFSZ: [libjvm.so+0x90b6b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
> SIGILL: [libjvm.so+0x90b6b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
> SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
> SIGUSR2: [libjvm.so+0x90ccf0], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
> SIGHUP: [libjvm.so+0x90e040], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
> SIGINT: [libjvm.so+0x90e040], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
> SIGTERM: [libjvm.so+0x90e040], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
> SIGQUIT: [libjvm.so+0x90e040], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
> 
> 
> ---------------  S Y S T E M  ---------------
> 
> OS:CentOS release 6.8 (Final)
> 
> uname:Linux 2.6.32-642.1.1.el6.x86_64 #1 SMP Tue May 31 21:57:07 UTC 2016 x86_64
> libc:glibc 2.12 NPTL 2.12 
> rlimit: STACK infinity, CORE 0k, NPROC 252569, NOFILE 65535, AS infinity
> load average:14.65 15.65 15.96
> 
> /proc/meminfo:
> MemTotal:       66053188 kB
> MemFree:        57095080 kB
> Buffers:               0 kB
> Cached:          4634644 kB
> SwapCached:            0 kB
> Active:          1468100 kB
> Inactive:        4329268 kB
> Active(anon):    1166732 kB
> Inactive(anon):   151980 kB
> Active(file):     301368 kB
> Inactive(file):  4177288 kB
> Unevictable:     1058884 kB
> Mlocked:           12356 kB
> SwapTotal:             0 kB
> SwapFree:              0 kB
> Dirty:                 0 kB
> Writeback:             0 kB
> AnonPages:       2187376 kB
> Mapped:           218872 kB
> Shmem:            152684 kB
> Slab:             338008 kB
> SReclaimable:     226468 kB
> SUnreclaim:       111540 kB
> KernelStack:       21104 kB
> PageTables:        19436 kB
> NFS_Unstable:          0 kB
> Bounce:                0 kB
> WritebackTmp:          0 kB
> CommitLimit:    32502304 kB
> Committed_AS:    8755192 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:      491784 kB
> VmallocChunk:   34326561200 kB
> HardwareCorrupted:     0 kB
> AnonHugePages:   1515520 kB
> HugePages_Total:     512
> HugePages_Free:      512
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> DirectMap4k:        4096 kB
> DirectMap2M:     2060288 kB
> DirectMap1G:    65011712 kB
> 
> 
> CPU:total 16 (8 cores per cpu, 2 threads per core) family 6 model 45 stepping 6, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, aes, clmul, ht, tsc, tscinvbit, tscinv
> 
> /proc/cpuinfo:
> processor : 0
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 2601.000
> cache size    : 20480 KB
> physical id   : 0
> siblings  : 8
> core id       : 0
> cpu cores : 8
> apicid        : 0
> initial apicid    : 0
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5187.39
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 1
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 2601.000
> cache size    : 20480 KB
> physical id   : 0
> siblings  : 8
> core id       : 1
> cpu cores : 8
> apicid        : 2
> initial apicid    : 2
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5187.39
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 2
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 0
> siblings  : 8
> core id       : 2
> cpu cores : 8
> apicid        : 4
> initial apicid    : 4
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5187.39
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 3
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 0
> siblings  : 8
> core id       : 3
> cpu cores : 8
> apicid        : 6
> initial apicid    : 6
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5187.39
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 4
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 0
> siblings  : 8
> core id       : 4
> cpu cores : 8
> apicid        : 8
> initial apicid    : 8
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5187.39
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 5
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 0
> siblings  : 8
> core id       : 5
> cpu cores : 8
> apicid        : 10
> initial apicid    : 10
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5187.39
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 6
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 0
> siblings  : 8
> core id       : 6
> cpu cores : 8
> apicid        : 12
> initial apicid    : 12
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5187.39
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 7
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 0
> siblings  : 8
> core id       : 7
> cpu cores : 8
> apicid        : 14
> initial apicid    : 14
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5187.39
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 8
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 1
> siblings  : 8
> core id       : 0
> cpu cores : 8
> apicid        : 32
> initial apicid    : 32
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5186.75
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 9
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 1
> siblings  : 8
> core id       : 1
> cpu cores : 8
> apicid        : 34
> initial apicid    : 34
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5186.75
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 10
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 1
> siblings  : 8
> core id       : 2
> cpu cores : 8
> apicid        : 36
> initial apicid    : 36
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5186.75
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 11
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 1
> siblings  : 8
> core id       : 3
> cpu cores : 8
> apicid        : 38
> initial apicid    : 38
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5186.75
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 12
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 1
> siblings  : 8
> core id       : 4
> cpu cores : 8
> apicid        : 40
> initial apicid    : 40
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5186.75
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 13
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 1
> siblings  : 8
> core id       : 5
> cpu cores : 8
> apicid        : 42
> initial apicid    : 42
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5186.75
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 14
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 1
> siblings  : 8
> core id       : 6
> cpu cores : 8
> apicid        : 44
> initial apicid    : 44
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5186.75
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> processor : 15
> vendor_id : GenuineIntel
> cpu family    : 6
> model     : 45
> model name    : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
> stepping  : 6
> microcode : 1561
> cpu MHz       : 1200.000
> cache size    : 20480 KB
> physical id   : 1
> siblings  : 8
> core id       : 7
> cpu cores : 8
> apicid        : 46
> initial apicid    : 46
> fpu       : yes
> fpu_exception : yes
> cpuid level   : 13
> wp        : yes
> flags     : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
> bogomips  : 5186.75
> clflush size  : 64
> cache_alignment   : 64
> address sizes : 46 bits physical, 48 bits virtual
> power management:
> 
> 
> 
> Memory: 4k page, physical 66053188k(57095080k free), swap 0k(0k free)
> 
> vm_info: Java HotSpot(TM) 64-Bit Server VM (25.51-b03) for linux-amd64 JRE (1.8.0_51-b16), built on Jun  8 2015 19:28:07 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)
> 
> time: Wed Oct 26 16:43:30 2016
> elapsed time: 8 seconds (0d 0h 0m 8s)
jsquyres commented 7 years ago

You should probably look into the call stack and/or run this application through a debugger (and/or enable core dumps) to see exactly where it is failing. From there, you can start trying to figure out why it is failing.

I don't see any mention of MPI in the diagnostic output -- do you know that this issue is caused by Open MPI somehow?

The diagnostic output shows that this is the call stack when it crashed -- you should probably start looking there.

> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> V  [libjvm.so+0x8cb59c]  Monitor::wait(bool, long, bool)+0x35c
> V  [libjvm.so+0x4a6d4b]  CompileQueue::get()+0x13b
> V  [libjvm.so+0x4a8f9b]  CompileBroker::compiler_thread_loop()+0x18b
> V  [libjvm.so+0xa5cbcf]  JavaThread::thread_main_inner()+0xdf
> V  [libjvm.so+0xa5ccfc]  JavaThread::run()+0x11c
> V  [libjvm.so+0x911048]  java_start(Thread*)+0x108
> C  [libpthread.so.0+0x7aa1]  start_thread+0xd1

I'm going to tentatively close this issue because I don't see the relation to MPI in the information that you have provided. If you find further information to indicate that Open MPI is somehow involved in the crash, feel free to re-open the bug (or open a new one). Thanks!