Segmentation fault for event_based_risk on macOS

raoanirudh commented 7 years ago

Running the test case event_based_risk/case_master without the --nd flag leads to a segmentation fault on macOS.

Replicated on two different systems: OS X 10.12.6 + Python 3.5.3 OS X 10.10.5 + Python 2.7.12

Engine log

[2017-10-17 08:42:51.01 #6 INFO] Running /Users/GEM/oq-risk-tests/test/event_based_risk/inputs/case_master/job.ini
[2017-10-17 08:42:51.01 #6 INFO] Using engine version 2.7.0-gitbeeb2f879a
[2017-10-17 08:42:51.02 #6 INFO] Reading the exposure
[2017-10-17 08:42:51.03 #6 INFO] Read 7 assets within the region_constraint and discarded 0 assets outside the region
[2017-10-17 08:42:51.04 #6 INFO] Read 7 assets on 7 sites
[2017-10-17 08:42:51.31 #6 INFO] Parsing /Users/GEM/oq-risk-tests/test/event_based_risk/inputs/case_master/source_model_1.xml
[2017-10-17 08:42:51.32 #6 INFO] Processed source model 1 with 4 potential gsim path(s) and 2 sources
[2017-10-17 08:42:51.32 #6 INFO] Parsing /Users/GEM/oq-risk-tests/test/event_based_risk/inputs/case_master/source_model_2.xml
[2017-10-17 08:42:51.34 #6 INFO] Processed source model 2 with 4 potential gsim path(s) and 2 sources
[2017-10-17 08:42:51.34 #6 WARNING] Found different sources with the same ID
[2017-10-17 08:42:51.34 #6 INFO] Prefiltering the CompositeSourceModel
[2017-10-17 08:42:51.34 #6 INFO] Using distance filtering [no rtree]
[2017-10-17 08:42:51.36 #6 WARNING] Found different sources with the same ID
[2017-10-17 08:42:51.37 #6 INFO] Using maxweight=200, numheavy=2, numtiles=1
[2017-10-17 08:42:51.37 #6 INFO] Sending source group #1 of 4 (Active Shallow Crust, 1 sources)
[2017-10-17 08:42:51.43 #6 INFO] Submitting  "compute_ruptures" tasks
[2017-10-17 08:42:51.44 #6 INFO] Sending source group #2 of 4 (Stable Shallow Crust, 1 sources)
[2017-10-17 08:42:51.45 #6 INFO] Sending source group #3 of 4 (Active Shallow Crust, 1 sources)
[2017-10-17 08:42:51.58 #6 INFO] Sending source group #4 of 4 (Stable Shallow Crust, 1 sources)
[2017-10-17 08:42:51.58 #6 INFO] Sent 32 sources in 28 tasks
[2017-10-17 08:42:51.61 #6 INFO] Sent 116.73 KB of data in 28 task(s)
[2017-10-17 08:42:51.64 #6 INFO] compute_ruptures   3%
[2017-10-17 08:42:51.67 #6 INFO] compute_ruptures   7%
[2017-10-17 08:42:51.68 #6 INFO] compute_ruptures  10%
[2017-10-17 08:42:51.69 #6 INFO] compute_ruptures  14%
[2017-10-17 08:42:51.70 #6 INFO] compute_ruptures  17%
[2017-10-17 08:42:51.71 #6 INFO] compute_ruptures  21%
[2017-10-17 08:42:51.75 #6 INFO] compute_ruptures  25%
[2017-10-17 08:42:51.76 #6 INFO] compute_ruptures  28%
[2017-10-17 08:42:51.77 #6 INFO] compute_ruptures  32%
[2017-10-17 08:42:51.78 #6 INFO] compute_ruptures  35%
[2017-10-17 08:42:51.79 #6 INFO] compute_ruptures  39%
[2017-10-17 08:42:51.79 #6 INFO] compute_ruptures  42%
[2017-10-17 08:42:51.80 #6 INFO] compute_ruptures  46%
[2017-10-17 08:42:51.81 #6 INFO] compute_ruptures  50%
[2017-10-17 08:42:51.82 #6 INFO] compute_ruptures  53%
[2017-10-17 08:42:51.83 #6 INFO] compute_ruptures  57%
[2017-10-17 08:42:51.84 #6 INFO] compute_ruptures  60%
[2017-10-17 08:42:51.85 #6 INFO] compute_ruptures  64%
[2017-10-17 08:42:51.86 #6 INFO] compute_ruptures  67%
[2017-10-17 08:42:51.87 #6 INFO] compute_ruptures  71%
[2017-10-17 08:42:51.87 #6 INFO] compute_ruptures  75%
[2017-10-17 08:42:51.88 #6 INFO] compute_ruptures  78%
[2017-10-17 08:42:51.89 #6 INFO] compute_ruptures  82%
[2017-10-17 08:42:51.90 #6 INFO] compute_ruptures  85%
[2017-10-17 08:42:51.91 #6 INFO] compute_ruptures  89%
[2017-10-17 08:42:51.92 #6 INFO] compute_ruptures  92%
[2017-10-17 08:42:51.93 #6 INFO] compute_ruptures  96%
[2017-10-17 08:42:51.94 #6 INFO] compute_ruptures 100%
[2017-10-17 08:42:51.94 #6 INFO] Received 712.47 KB of data, maximum per task 161.04 KB
[2017-10-17 08:42:51.95 #6 INFO] Setting 10935 event years on 555 ruptures
[2017-10-17 08:42:52.03 #6 WARNING] To store the ground motion fields change calculation_mode = event_based
[2017-10-17 08:42:52.03 #6 WARNING] To compute the hazard curves change calculation_mode = event_based
[2017-10-17 08:42:52.04 #6 INFO] minimum_intensity={'PGA': 0.0050000000000000001, 'SA(0.1)': 0.0050000000000000001, 'SA(0.3)': 0.0050000000000000001, 'SA(0.4)': 0.0050000000000000001}
[2017-10-17 08:42:52.08 #6 INFO] Submitting 2 "event_based_risk#1" tasks
[2017-10-17 08:42:52.08 #6 INFO] Sent 536.95 KB of data in 2 task(s)
[2017-10-17 08:42:52.10 #6 INFO] Submitting 2 "event_based_risk#2" tasks
[2017-10-17 08:42:52.11 #6 INFO] Sent 289.52 KB of data in 2 task(s)
[2017-10-17 08:42:52.12 #6 CRITICAL] 
[2017-10-17 08:42:52.12 #6 CRITICAL] Traceback (most recent call last):
  File "/Users/GEM/oq-engine/openquake/engine/engine.py", line 238, in run_calc
    _do_run_calc(calc, exports, hazard_calculation_id, **kw)
  File "/Users/GEM/oq-engine/openquake/engine/engine.py", line 273, in _do_run_calc
    close=False, **kw)  # don't close the datastore too soon
  File "/Users/GEM/oq-engine/openquake/calculators/base.py", line 205, in run
    self.result = self.execute()
  File "/Users/GEM/oq-engine/openquake/calculators/event_based_risk.py", line 353, in execute
    num_events = self.save_results(allres, num_rlzs)
  File "/Users/GEM/oq-engine/openquake/calculators/event_based_risk.py", line 388, in save_results
    for dic in res:
  File "/Users/GEM/oq-engine/openquake/baselib/parallel.py", line 357, in __iter__
    result = fut.result()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/concurrent/futures/_base.py", line 398, in result
    return self.__get_result()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

System log

Process:               Python [1352]
Path:                  /Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.5.3 (3.5.3)
Code Type:             X86-64 (Native)
Parent Process:        Python [1310]
Responsible:           Python [1352]
User ID:               501

Date/Time:             2017-10-18 16:54:31.802 +0200
OS Version:            Mac OS X 10.12.6 (16G29)
Report Version:        12
Anonymous UUID:        9FCC4B43-BD05-083B-D1D4-B1424E386282

Time Awake Since Boot: 2700 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000110
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [0]

VM Regions Near 0x110:
--> 
    __TEXT                 0000000100000000-0000000100001000 [    4K] r-x/rwx SM=COW  /Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python

Application Specific Information:
*** multi-threaded process forked ***
crashed on child side of fork pre-exec

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libdispatch.dylib               0x00007fffc13ea749 _dispatch_queue_push + 171
1   libBLAS.dylib                   0x00007fffa8d37b76 APL_dgemm + 1171
2   libBLAS.dylib                   0x00007fffa8da504e cblas_dgemm + 1438
3   multiarray.cpython-35m-darwin.so    0x00000001046dd933 cblas_matrixproduct + 4675
4   multiarray.cpython-35m-darwin.so    0x00000001046a7750 PyArray_MatrixProduct2 + 240
5   multiarray.cpython-35m-darwin.so    0x00000001046ac5ed array_matrixproduct + 269
6   org.python.python               0x0000000100065b19 PyCFunction_Call + 281
7   org.python.python               0x00000001000f2a81 PyEval_EvalFrameEx + 35729
8   org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
9   org.python.python               0x00000001000f33ef PyEval_EvalFrameEx + 38143
10  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
11  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
12  org.python.python               0x00000001000f33ef PyEval_EvalFrameEx + 38143
13  org.python.python               0x0000000100037a5a gen_send_ex + 154
14  org.python.python               0x00000001000ecb74 PyEval_EvalFrameEx + 11396
15  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
16  org.python.python               0x00000001000f33ef PyEval_EvalFrameEx + 38143
17  org.python.python               0x0000000100037a5a gen_send_ex + 154
18  org.python.python               0x00000001000ecb74 PyEval_EvalFrameEx + 11396
19  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
20  org.python.python               0x00000001000f33ef PyEval_EvalFrameEx + 38143
21  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
22  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
23  org.python.python               0x000000010004077a function_call + 186
24  org.python.python               0x000000010000d8d3 PyObject_Call + 99
25  org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
26  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
27  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
28  org.python.python               0x000000010004077a function_call + 186
29  org.python.python               0x000000010000d8d3 PyObject_Call + 99
30  org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
31  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
32  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
33  org.python.python               0x000000010004077a function_call + 186
34  org.python.python               0x000000010000d8d3 PyObject_Call + 99
35  org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
36  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
37  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
38  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
39  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
40  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
41  org.python.python               0x000000010004077a function_call + 186
42  org.python.python               0x000000010000d8d3 PyObject_Call + 99
43  org.python.python               0x00000001000289cc method_call + 140
44  org.python.python               0x000000010000d8d3 PyObject_Call + 99
45  org.python.python               0x00000001000834c1 slot_tp_init + 81
46  org.python.python               0x0000000100079f44 type_call + 212
47  org.python.python               0x000000010000d8d3 PyObject_Call + 99
48  org.python.python               0x00000001000edb65 PyEval_EvalFrameEx + 15477
49  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
50  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
51  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
52  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
53  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
54  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
55  org.python.python               0x00000001000f33ef PyEval_EvalFrameEx + 38143
56  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
57  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
58  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
59  org.python.python               0x000000010004077a function_call + 186
60  org.python.python               0x000000010000d8d3 PyObject_Call + 99
61  org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
62  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
63  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
64  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
65  org.python.python               0x000000010004077a function_call + 186
66  org.python.python               0x000000010000d8d3 PyObject_Call + 99
67  org.python.python               0x00000001000289cc method_call + 140
68  org.python.python               0x000000010000d8d3 PyObject_Call + 99
69  org.python.python               0x00000001000835aa slot_tp_iter + 74
70  org.python.python               0x000000010000c9cb PyObject_GetIter + 27
71  org.python.python               0x0000000100045154 listextend + 84
72  org.python.python               0x0000000100045743 list_init + 211
73  org.python.python               0x0000000100079f44 type_call + 212
74  org.python.python               0x000000010000d8d3 PyObject_Call + 99
75  org.python.python               0x00000001000edb65 PyEval_EvalFrameEx + 15477
76  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
77  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
78  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
79  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
80  org.python.python               0x00000001000f33ef PyEval_EvalFrameEx + 38143
81  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
82  org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
83  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
84  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
85  org.python.python               0x000000010004077a function_call + 186
86  org.python.python               0x000000010000d8d3 PyObject_Call + 99
87  org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
88  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
89  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
90  org.python.python               0x000000010004077a function_call + 186
91  org.python.python               0x000000010000d8d3 PyObject_Call + 99
92  org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
93  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
94  org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
95  org.python.python               0x000000010004077a function_call + 186
96  org.python.python               0x000000010000d8d3 PyObject_Call + 99
97  org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
98  org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
99  org.python.python               0x00000001000f33ef PyEval_EvalFrameEx + 38143
100 org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
101 org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
102 org.python.python               0x000000010004077a function_call + 186
103 org.python.python               0x000000010000d8d3 PyObject_Call + 99
104 org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
105 org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
106 org.python.python               0x00000001000f3f97 PyEval_EvalCodeEx + 71
107 org.python.python               0x000000010004077a function_call + 186
108 org.python.python               0x000000010000d8d3 PyObject_Call + 99
109 org.python.python               0x00000001000ec813 PyEval_EvalFrameEx + 10531
110 org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
111 org.python.python               0x00000001000f33ef PyEval_EvalFrameEx + 38143
112 org.python.python               0x00000001000f34fc PyEval_EvalFrameEx + 38412
113 org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
114 org.python.python               0x00000001000f3ff1 PyEval_EvalCode + 81
115 org.python.python               0x00000001000e73cf builtin_exec + 623
116 org.python.python               0x0000000100065ae9 PyCFunction_Call + 233
117 org.python.python               0x00000001000f2a81 PyEval_EvalFrameEx + 35729
118 org.python.python               0x00000001000f3e90 _PyEval_EvalCodeWithName + 2400
119 org.python.python               0x00000001000f3ff1 PyEval_EvalCode + 81
120 org.python.python               0x000000010012215e PyRun_FileExFlags + 206
121 org.python.python               0x00000001001223ff PyRun_SimpleFileExFlags + 447
122 org.python.python               0x000000010013b1d8 Py_Main + 3512
123 org.python.python               0x0000000100000e32 0x100000000 + 3634
124 org.python.python               0x0000000100000c84 0x100000000 + 3204

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000100  rbx: 0x0000000000000000  rcx: 0x0000000000002000  rdx: 0x0000000000002000
  rdi: 0x00007fffca2f8340  rsi: 0x0000000109c01000  rbp: 0x00007fff5bff67d0  rsp: 0x00007fff5bff6648
   r8: 0x00000000200020ff   r9: 0x00000000ffffffff  r10: 0x00000000fff00000  r11: 0xffff8001ac798db0
  r12: 0x0000000000001324  r13: 0x0000000000000000  r14: 0x0000000000000007  r15: 0x0000000000001324
  rip: 0x00007fffc13ea749  rfl: 0x0000000000010206  cr2: 0x0000000000000110

Logical CPU:     4
Error Code:      0x00000006
Trap Number:     14

Binary Images:
       0x100000000 -        0x100000ff7 +org.python.python (3.5.3 - 3.5.3) <0CBF5DD5-80D6-6F39-5B3D-BE16CC541C78> /Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
       0x100003000 -        0x100216ff7 +org.python.python (3.5.3, [c] 2001-2017 Python Software Foundation. - 3.5.3) <E51177F3-C491-7877-5818-85E1EF0E3D1E> /Library/Frameworks/Python.framework/Versions/3.5/Python
       0x1003e8000 -        0x1003e9fff +_heapq.cpython-35m-darwin.so (???) <3805F1DD-CEE4-6FCC-BA4A-1008E4FD1C02> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_heapq.cpython-35m-darwin.so
       0x1003ee000 -        0x1003f2fff +zlib.cpython-35m-darwin.so (???) <F3DA3D0C-A207-6C1E-D74D-5E496FDFBD69> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/zlib.cpython-35m-darwin.so
       0x1003f7000 -        0x1003f8fff +_bz2.cpython-35m-darwin.so (???) <89798E3B-B298-B0E7-C540-6581C22D7855> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_bz2.cpython-35m-darwin.so
       0x1003fc000 -        0x1003fcfff +grp.cpython-35m-darwin.so (???) <6F145293-2A9E-AA6D-2C6E-C9AFA0C4E6D2> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/grp.cpython-35m-darwin.so
       0x1019d1000 -        0x1019d6fff +_struct.cpython-35m-darwin.so (???) <BEB0EA88-9FD2-B91B-1179-288B4F4A497C> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_struct.cpython-35m-darwin.so
       0x1019de000 -        0x1019e1fff +binascii.cpython-35m-darwin.so (???) <23176E2E-5B33-9A17-4C7B-78DDFB6916F6> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/binascii.cpython-35m-darwin.so
       0x1019e5000 -        0x1019e6ff7 +_posixsubprocess.cpython-35m-darwin.so (???) <B185317A-931E-C7E7-7A6F-4057225B33F4> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_posixsubprocess.cpython-35m-darwin.so
       0x1019ea000 -        0x1019edfff +select.cpython-35m-darwin.so (???) <7D614C09-54B2-B722-F87B-3D7A155BC1AE> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/select.cpython-35m-darwin.so
       0x1019f3000 -        0x1019f8fef +math.cpython-35m-darwin.so (???) <5AC216BA-3732-8BBF-9BFF-5F1F84838A07> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/math.cpython-35m-darwin.so
       0x101ee1000 -        0x101f0efef +_lzma.cpython-35m-darwin.so (???) <7D90A616-0411-0A3B-C3AE-9AC128F459EE> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_lzma.cpython-35m-darwin.so
       0x101f98000 -        0x101fa5fff +_datetime.cpython-35m-darwin.so (???) <7A8D0F82-7ABA-155F-82B5-E28E71C08D3D> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_datetime.cpython-35m-darwin.so
       0x101fb0000 -        0x101fe5fff +pyexpat.cpython-35m-darwin.so (???) <281F6BFA-8814-30AC-70EB-E0A98FE5EC2B> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/pyexpat.cpython-35m-darwin.so
       0x101ff7000 -        0x101ff9fff +_hashlib.cpython-35m-darwin.so (???) <A194177C-41EE-7CBE-61D9-7A7D5BFA0867> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_hashlib.cpython-35m-darwin.so
       0x101ffd000 -        0x101ffdfff +_opcode.cpython-35m-darwin.so (???) <9AD57045-9FCB-F501-666C-ED2309CDD33D> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_opcode.cpython-35m-darwin.so
       0x104000000 -        0x104001ff7 +_random.cpython-35m-darwin.so (???) <EE260851-EE4B-EDF5-ED0C-C6516B0CFD5E> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_random.cpython-35m-darwin.so
       0x104044000 -        0x10404eff7 +_socket.cpython-35m-darwin.so (???) <ABA065F2-EAB4-0843-6EBF-72B5E1A7CD64> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_socket.cpython-35m-darwin.so
       0x1042c2000 -        0x1042d4fff +_pickle.cpython-35m-darwin.so (???) <EBCB0931-0C1D-257D-E1E9-9C091B6D4D27> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_pickle.cpython-35m-darwin.so
       0x1042e1000 -        0x1042e2ff7 +lapack_lite.cpython-35m-darwin.so (???) <EAAAB223-4E4A-3E1D-B0CE-E8A639265D3C> /Users/USER/*/lapack_lite.cpython-35m-darwin.so
       0x1042e8000 -        0x1042f1ff7 +fftpack_lite.cpython-35m-darwin.so (???) <58E02D71-415E-36AA-9D54-3F32E4786163> /Users/USER/*/fftpack_lite.cpython-35m-darwin.so
       0x1042f5000 -        0x1042f6fff +termios.cpython-35m-darwin.so (???) <8A634F5D-94F2-6E0C-5C01-CB57FFFFC70B> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/termios.cpython-35m-darwin.so
       0x1042fb000 -        0x1042fbfff +_bisect.cpython-35m-darwin.so (???) <ED4F5CE9-8926-1F4E-65EA-6145AFBF6AE1> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_bisect.cpython-35m-darwin.so
       0x1045c1000 -        0x10474dfff +multiarray.cpython-35m-darwin.so (???) <07A0BDAF-2268-3349-91E5-903DC43C2CBD> /Users/USER/*/multiarray.cpython-35m-darwin.so
       0x10480a000 -        0x1048b1fff +umath.cpython-35m-darwin.so (???) <C4FD2050-A6B9-3FDC-AE7D-35B65889CB42> /Users/USER/*/umath.cpython-35m-darwin.so
       0x10492a000 -        0x10493ffff +_ctypes.cpython-35m-darwin.so (???) <0E5BA040-E02F-258A-F736-0BDEABD9BB8A> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_ctypes.cpython-35m-darwin.so
       0x104ad0000 -        0x104ae8fff +_umath_linalg.cpython-35m-darwin.so (???) <5715001B-F5D4-39D6-95A1-3D05FFA646D3> /Users/USER/*/_umath_linalg.cpython-35m-darwin.so
       0x104bb6000 -        0x104c56ff7 +mtrand.cpython-35m-darwin.so (???) <F1E0A033-C625-3451-9F27-FF029B90473E> /Users/USER/*/mtrand.cpython-35m-darwin.so
       0x104e80000 -        0x104e87ff7 +_errors.cpython-35m-darwin.so (???) <A20BA715-27D4-313D-B87E-4184B77B70D2> /Users/USER/*/_errors.cpython-35m-darwin.so
       0x104e8d000 -        0x10527dfff +libhdf5.8.dylib (0) <2ABD0D54-717A-3B8F-8ACD-67A65E83CCB8> /Users/USER/*/libhdf5.8.dylib
       0x1052b9000 -        0x1052d6ff7 +libhdf5_hl.8.dylib (0) <ED9737D4-4A40-391F-B8EB-5091AFA53CD8> /Users/USER/*/libhdf5_hl.8.dylib
       0x1052ef000 -        0x105305fff +libsz.2.0.0.dylib (0) <0CF91DAD-4C26-3073-9516-DE79E1408248> /Users/USER/*/libsz.2.0.0.dylib
       0x105310000 -        0x10531cff7 +_conv.cpython-35m-darwin.so (???) <38E2DFF3-AC63-34DE-8B47-6E74A7E4C4C2> /Users/USER/*/_conv.cpython-35m-darwin.so
       0x105324000 -        0x10532dff7 +h5r.cpython-35m-darwin.so (???) <CFE98DAF-784D-36BD-8C70-A21071F5D578> /Users/USER/*/h5r.cpython-35m-darwin.so
       0x105335000 -        0x10534dfff +_objects.cpython-35m-darwin.so (???) <A1712EEF-E3EA-3CEF-B763-17020FEDF9C9> /Users/USER/*/_objects.cpython-35m-darwin.so
       0x10535a000 -        0x105383ff7 +defs.cpython-35m-darwin.so (???) <68981EFD-4EFF-3B57-9CA2-44AC0377D978> /Users/USER/*/defs.cpython-35m-darwin.so
       0x10539a000 -        0x1053f1fff +h5t.cpython-35m-darwin.so (???) <47AAE53B-EC2D-3B01-8485-BBA20F0BAE75> /Users/USER/*/h5t.cpython-35m-darwin.so
       0x105452000 -        0x105459ff7 +utils.cpython-35m-darwin.so (???) <554FEDF0-7751-3D10-B846-E3673B35E832> /Users/USER/*/utils.cpython-35m-darwin.so
       0x10545f000 -        0x10546ffff +h5.cpython-35m-darwin.so (???) <26CA540D-F937-3E9B-94DF-28BAED6E3F17> /Users/USER/*/h5.cpython-35m-darwin.so
       0x10547a000 -        0x10547ffff +h5z.cpython-35m-darwin.so (???) <B702FC04-1613-3819-ADCF-7D4C70B19F82> /Users/USER/*/h5z.cpython-35m-darwin.so
       0x105485000 -        0x10549eff7 +h5a.cpython-35m-darwin.so (???) <1F435E6B-855B-3B91-AC4B-06A5F8848193> /Users/USER/*/h5a.cpython-35m-darwin.so
       0x1054ab000 -        0x1054c3fff +h5s.cpython-35m-darwin.so (???) <AF29CE68-E7D0-3C0A-BD42-F2EBFE8D50B3> /Users/USER/*/h5s.cpython-35m-darwin.so
       0x1054d0000 -        0x1054fefff +h5p.cpython-35m-darwin.so (???) <30666491-12C7-38C5-BEB6-4CE0DB41E090> /Users/USER/*/h5p.cpython-35m-darwin.so
       0x105519000 -        0x105521fff +h5ac.cpython-35m-darwin.so (???) <1D446028-35E8-3FC4-9196-6D1B6F350021> /Users/USER/*/h5ac.cpython-35m-darwin.so
       0x105528000 -        0x10552fff7 +_proxy.cpython-35m-darwin.so (???) <604C313B-E48C-3C1E-A3D7-BCF232C1E90F> /Users/USER/*/_proxy.cpython-35m-darwin.so
       0x105535000 -        0x10554aff7 +h5d.cpython-35m-darwin.so (???) <0A4DA8D2-527A-3DDC-93E2-E0CBD65D0EBE> /Users/USER/*/h5d.cpython-35m-darwin.so
       0x105556000 -        0x105562ff7 +h5ds.cpython-35m-darwin.so (???) <3DE87DC5-BD2B-3323-900B-99C7A462368A> /Users/USER/*/h5ds.cpython-35m-darwin.so
       0x10556a000 -        0x105581fff +h5f.cpython-35m-darwin.so (???) <B7E5539D-BD2F-3ED7-91C6-7C1F0A0CD570> /Users/USER/*/h5f.cpython-35m-darwin.so
       0x10558f000 -        0x1055aafff +h5g.cpython-35m-darwin.so (???) <7409622D-117D-3F84-8A3C-5AB7C8CEC376> /Users/USER/*/h5g.cpython-35m-darwin.so
       0x1055b9000 -        0x1055c3ff7 +h5i.cpython-35m-darwin.so (???) <5256BFF5-37BD-303D-8E17-35C3F3B343D9> /Users/USER/*/h5i.cpython-35m-darwin.so
       0x1055cb000 -        0x1055cdff7 +h5fd.cpython-35m-darwin.so (???) <5F9CA246-7D9D-3FA4-9C0B-D5ABD7D7CBBD> /Users/USER/*/h5fd.cpython-35m-darwin.so
       0x105612000 -        0x105623fff +h5o.cpython-35m-darwin.so (???) <E0408E78-54BD-3351-B0EC-09C3996A1AE9> /Users/USER/*/h5o.cpython-35m-darwin.so
       0x105630000 -        0x10563ffff +h5l.cpython-35m-darwin.so (???) <FEEB34DB-D06D-3F14-A584-2CAC29CF99B0> /Users/USER/*/h5l.cpython-35m-darwin.so
       0x105749000 -        0x10574dff7 +_csv.cpython-35m-darwin.so (???) <87CF99BC-2D6E-10A9-EA53-2CC89E7FCAFA> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_csv.cpython-35m-darwin.so
       0x105753000 -        0x105783ff7 +libgeos_c.1.dylib (10.2) <AE7FF747-12C5-3CAB-BBB7-A315997B20C9> /Users/USER/*/libgeos_c.1.dylib
       0x1057db000 -        0x1057e2fff +array.cpython-35m-darwin.so (???) <C349684C-D7A1-CF74-316C-0A9EF54B045B> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/array.cpython-35m-darwin.so
       0x1057eb000 -        0x1057f4ff7 +_elementtree.cpython-35m-darwin.so (???) <74840180-0783-D474-2CC6-DC737911D00C> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_elementtree.cpython-35m-darwin.so
       0x1057fd000 -        0x1057fdff7 +_scproxy.cpython-35m-darwin.so (???) <D748A07C-91A8-6025-2DF4-2F28248663BB> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_scproxy.cpython-35m-darwin.so
       0x105900000 -        0x105af1fff +libgeos-3.4.2.dylib (0) <D20221E7-A93B-3B7B-9821-A2AE1829F5A0> /Users/USER/*/libgeos-3.4.2.dylib
       0x105d76000 -        0x105d83ff7 +_ssl.cpython-35m-darwin.so (???) <94A8AB08-FA67-EE9C-6145-5532BFC40E8F> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_ssl.cpython-35m-darwin.so
       0x105dd2000 -        0x105efcfef +_ufuncs.cpython-35m-darwin.so (???) <95E1ECE6-F829-3ADC-9961-9405B6AAAD21> /Users/USER/*/_ufuncs.cpython-35m-darwin.so
       0x105f50000 -        0x106006fcf +libgfortran.2.0.0.dylib (3) <ACF1816E-EA8A-4DB9-A893-A4EF0CEFFFBA> /Users/USER/*/libgfortran.2.0.0.dylib
       0x10604b000 -        0x106057fb1 +libgcc_s.1.dylib (1) <BBB4EAFF-D7EC-4B7A-9AED-C5BB3003BDE0> /Users/USER/*/libgcc_s.1.dylib
       0x106060000 -        0x106071fff +_ufuncs_cxx.cpython-35m-darwin.so (???) <73108F1C-9375-36A9-B2F1-A8EDC5420823> /Users/USER/*/_ufuncs_cxx.cpython-35m-darwin.so
       0x10607c000 -        0x10612efef +specfun.cpython-35m-darwin.so (???) <46A017DA-CD57-3D25-96AD-1CC48991FF26> /Users/USER/*/specfun.cpython-35m-darwin.so
       0x10617f000 -        0x1061b8ff7 +_fblas.cpython-35m-darwin.so (???) <0CB45320-FFA2-39B2-AA86-550B6A9DD2AF> /Users/USER/*/_fblas.cpython-35m-darwin.so
       0x1061de000 -        0x106260fff +_flapack.cpython-35m-darwin.so (???) <6723F2ED-1C5E-31C3-8170-D9DFB687407E> /Users/USER/*/_flapack.cpython-35m-darwin.so
       0x1062f7000 -        0x106301fff +_flinalg.cpython-35m-darwin.so (???) <CB2BDE62-03C1-356F-8927-8B457FBCA4FC> /Users/USER/*/_flinalg.cpython-35m-darwin.so
       0x106309000 -        0x10632dff7 +_solve_toeplitz.cpython-35m-darwin.so (???) <2398F903-33D9-36DB-A6C7-0176046144F8> /Users/USER/*/_solve_toeplitz.cpython-35m-darwin.so
       0x106340000 -        0x106380ff7 +_decomp_update.cpython-35m-darwin.so (???) <C5AC40AC-D545-3FAD-B4A1-3603A877EBB0> /Users/USER/*/_decomp_update.cpython-35m-darwin.so
       0x106396000 -        0x1063befff +cython_blas.cpython-35m-darwin.so (???) <863B5E16-BD41-35A3-B917-7239861962BC> /Users/USER/*/cython_blas.cpython-35m-darwin.so
       0x1063d8000 -        0x106453fff +cython_lapack.cpython-35m-darwin.so (???) <0B1A07A3-0DA0-3A92-B00F-EA4FA4C4C363> /Users/USER/*/cython_lapack.cpython-35m-darwin.so
       0x106662000 -        0x10666fff7 +_ellip_harm_2.cpython-35m-darwin.so (???) <AE464E11-4038-3FC1-B58C-771F88C3C291> /Users/USER/*/_ellip_harm_2.cpython-35m-darwin.so
       0x106678000 -        0x106699fe7 +_odepack.cpython-35m-darwin.so (???) <ABD1E507-18C6-339B-AECC-7CC0F2581158> /Users/USER/*/_odepack.cpython-35m-darwin.so
       0x10669e000 -        0x1066b7fe7 +_quadpack.cpython-35m-darwin.so (???) <91B4C548-B06D-3280-A35D-6B7553B30019> /Users/USER/*/_quadpack.cpython-35m-darwin.so
       0x1066bb000 -        0x1066e3ff7 +vode.cpython-35m-darwin.so (???) <368A0DB3-C3F7-3723-B95F-D25F49E8C56D> /Users/USER/*/vode.cpython-35m-darwin.so
       0x1066eb000 -        0x1066fdfff +_dop.cpython-35m-darwin.so (???) <1511F0FA-056D-3E62-8187-F8AF759EE730> /Users/USER/*/_dop.cpython-35m-darwin.so
       0x106703000 -        0x106726fe7 +lsoda.cpython-35m-darwin.so (???) <0FBD7567-47BB-3C18-812F-793E4F950B21> /Users/USER/*/lsoda.cpython-35m-darwin.so
       0x10676c000 -        0x106772ff7 +minpack2.cpython-35m-darwin.so (???) <0CCDD6CF-7421-3B2F-AE38-4E6D084D4304> /Users/USER/*/minpack2.cpython-35m-darwin.so
       0x1067b7000 -        0x1067d0fff +_lbfgsb.cpython-35m-darwin.so (???) <D5CA996A-EF42-3803-8F4B-07C69BD899E3> /Users/USER/*/_lbfgsb.cpython-35m-darwin.so
       0x1067d5000 -        0x106a72ff7 +_sparsetools.cpython-35m-darwin.so (???) <74379AF2-7D95-3668-8A29-D15AD751C328> /Users/USER/*/_sparsetools.cpython-35m-darwin.so
       0x106b9d000 -        0x106c01fff +_csparsetools.cpython-35m-darwin.so (???) <6A44ABE2-2F1A-3810-A4D8-15AA51AB4DD8> /Users/USER/*/_csparsetools.cpython-35m-darwin.so
       0x106c1b000 -        0x106c52fff +_shortest_path.cpython-35m-darwin.so (???) <855A6D63-6644-3552-8F5E-4F71F83D7909> /Users/USER/*/_shortest_path.cpython-35m-darwin.so
       0x106c63000 -        0x106c7efff +_tools.cpython-35m-darwin.so (???) <F13E78A8-9A21-3406-AAC8-37856ED8AA21> /Users/USER/*/_tools.cpython-35m-darwin.so
       0x106c8b000 -        0x106cacff7 +_traversal.cpython-35m-darwin.so (???) <DFD34AB4-5804-3690-A283-D9927E27AA25> /Users/USER/*/_traversal.cpython-35m-darwin.so
       0x106cb8000 -        0x106cd2ff7 +_min_spanning_tree.cpython-35m-darwin.so (???) <30E12FC8-4520-3770-B7AE-E012ABF3DD44> /Users/USER/*/_min_spanning_tree.cpython-35m-darwin.so
       0x106ce3000 -        0x106d1bff7 +_reordering.cpython-35m-darwin.so (???) <AA8C1325-3FEB-3E68-8DB2-144D0BB5EADD> /Users/USER/*/_reordering.cpython-35m-darwin.so
       0x106d72000 -        0x106d9eff7 +_iterative.cpython-35m-darwin.so (???) <7DE36845-3908-3222-AE9F-38E62B99C57E> /Users/USER/*/_iterative.cpython-35m-darwin.so
       0x106db3000 -        0x106dfdff7 +_superlu.cpython-35m-darwin.so (???) <AB530BC6-0DDD-36B1-A2D7-1E17BA056F5A> /Users/USER/*/_superlu.cpython-35m-darwin.so
       0x106e18000 -        0x106e9bff7 +_arpack.cpython-35m-darwin.so (???) <B5C4515D-EA09-3AD6-B605-C9D735C9BC62> /Users/USER/*/_arpack.cpython-35m-darwin.so
       0x106ef3000 -        0x106efbfff +moduleTNC.cpython-35m-darwin.so (???) <15F88686-FC2C-3A6E-9E95-53CEA8082425> /Users/USER/*/moduleTNC.cpython-35m-darwin.so
       0x106efe000 -        0x106f14ff7 +_cobyla.cpython-35m-darwin.so (???) <6BB0B74A-B131-3658-9A96-911498624BCB> /Users/USER/*/_cobyla.cpython-35m-darwin.so
       0x106f19000 -        0x106f2bff7 +_slsqp.cpython-35m-darwin.so (???) <84B88F3E-A09A-38ED-95E4-F009F41D04BB> /Users/USER/*/_slsqp.cpython-35m-darwin.so
       0x106f30000 -        0x106f48ff7 +_minpack.cpython-35m-darwin.so (???) <7C25E2FB-F4CA-3B25-B37E-785FF63F5A7D> /Users/USER/*/_minpack.cpython-35m-darwin.so
       0x106f4c000 -        0x106f69ff7 +_group_columns.cpython-35m-darwin.so (???) <5E780CF2-C775-37BC-8D22-9B13AA32B8D0> /Users/USER/*/_group_columns.cpython-35m-darwin.so
       0x106fb9000 -        0x106fcfff7 +givens_elimination.cpython-35m-darwin.so (???) <115520B7-CAE5-36D2-88B4-25A56D93C010> /Users/USER/*/givens_elimination.cpython-35m-darwin.so
       0x106fde000 -        0x106fdfff7 +_zeros.cpython-35m-darwin.so (???) <5D0DA3A6-2B93-3C4E-BA89-FF2F8F1E3DAE> /Users/USER/*/_zeros.cpython-35m-darwin.so
       0x106fe2000 -        0x106feafff +_nnls.cpython-35m-darwin.so (???) <3482ECEF-2AD5-3518-8E14-5078285726B1> /Users/USER/*/_nnls.cpython-35m-darwin.so
       0x10706e000 -        0x107078ff7 +vonmises_cython.cpython-35m-darwin.so (???) <068B04A0-6408-3B52-BA0A-7EBFFB23D032> /Users/USER/*/vonmises_cython.cpython-35m-darwin.so
       0x10717f000 -        0x107185ff7 +_rank.cpython-35m-darwin.so (???) <919C1655-10DE-3D08-BBF2-068C05E26382> /Users/USER/*/_rank.cpython-35m-darwin.so
       0x10718b000 -        0x107192ff7 +statlib.cpython-35m-darwin.so (???) <95457675-B460-317E-A216-D7943E993EBF> /Users/USER/*/statlib.cpython-35m-darwin.so
       0x1071d7000 -        0x1071e1ff7 +_distance_wrap.cpython-35m-darwin.so (???) <8D7622D1-507E-3435-9F1B-EFD12BA298A6> /Users/USER/*/_distance_wrap.cpython-35m-darwin.so
       0x1071ee000 -        0x1071f4ff7 +_psutil_osx.cpython-35m-darwin.so (???) <DF900C7D-0BF1-3D90-BF52-D96FF7D83926> /Users/USER/*/_psutil_osx.cpython-35m-darwin.so
       0x1071fa000 -        0x1071fbff7 +_psutil_posix.cpython-35m-darwin.so (???) <37012A66-29CB-3174-851C-EBD338805728> /Users/USER/*/_psutil_posix.cpython-35m-darwin.so
       0x107300000 -        0x10730eff7 +mvn.cpython-35m-darwin.so (???) <315AA8AC-2972-386B-9707-C95238F96794> /Users/USER/*/mvn.cpython-35m-darwin.so
       0x10744b000 -        0x10747aff7 +_fitpack.cpython-35m-darwin.so (???) <6CD0D35D-2FB0-3E2B-B484-076BBE15DCA3> /Users/USER/*/_fitpack.cpython-35m-darwin.so
       0x10747e000 -        0x1074d2fff +dfitpack.cpython-35m-darwin.so (???) <BB314E80-F41F-30C7-AAB5-E9824D4E6254> /Users/USER/*/dfitpack.cpython-35m-darwin.so
       0x1074df000 -        0x107518ff7 +_ppoly.cpython-35m-darwin.so (???) <5D29C4A5-16F4-32E8-BD71-7E169CA1034E> /Users/USER/*/_ppoly.cpython-35m-darwin.so
       0x107570000 -        0x1075affff +interpnd.cpython-35m-darwin.so (???) <21B31A83-7A05-3A29-942E-E7BCD65484E5> /Users/USER/*/interpnd.cpython-35m-darwin.so
       0x1075ca000 -        0x107641ff7 +ckdtree.cpython-35m-darwin.so (???) <0585C4DB-5D0A-3285-A497-93F886B70EF5> /Users/USER/*/ckdtree.cpython-35m-darwin.so
       0x107662000 -        0x107719ff7 +qhull.cpython-35m-darwin.so (???) <56F451B1-1473-3AED-AEA9-0CD89B308AF1> /Users/USER/*/qhull.cpython-35m-darwin.so
       0x10788b000 -        0x1078d3ff7 +_decimal.cpython-35m-darwin.so (???) <23499295-DC6F-10A0-03F2-582C57075BE1> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_decimal.cpython-35m-darwin.so
       0x1079f0000 -        0x1079f1ff7 +_multiprocessing.cpython-35m-darwin.so (???) <621E872D-0B45-F6D0-A6CE-ED238D0228F5> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_multiprocessing.cpython-35m-darwin.so
       0x1079f5000 -        0x1079f8ff7 +error.cpython-35m-darwin.so (???) <B58504EB-5276-33B8-9947-8E6E6AD58CE5> /Users/USER/*/error.cpython-35m-darwin.so
       0x1082ca000 -        0x1082d3ff7 +constants.cpython-35m-darwin.so (???) <7AC480D5-78CB-3CB6-91A8-D945665E513B> /Users/USER/*/constants.cpython-35m-darwin.so
       0x1082df000 -        0x1082ebff7 +message.cpython-35m-darwin.so (???) <11A47000-A896-3CD7-9F0C-C2AA2B4C8B96> /Users/USER/*/message.cpython-35m-darwin.so
       0x1082f8000 -        0x1082faff7 +_version.cpython-35m-darwin.so (???) <11702E85-87C9-3F47-8FD9-C9BD26615DD8> /Users/USER/*/_version.cpython-35m-darwin.so
       0x108600000 -        0x10865fff7 +libzmq.cpython-35m-darwin.so (???) <187D74FD-9A97-3172-B8F1-DCFBCE14319C> /Users/USER/*/libzmq.cpython-35m-darwin.so
       0x1086ee000 -        0x1086f2fff +utils.cpython-35m-darwin.so (???) <3039938A-5548-3015-90FA-B07762C618B4> /Users/USER/*/utils.cpython-35m-darwin.so
       0x1086f8000 -        0x1086faff7 +_lsprof.cpython-35m-darwin.so (???) <EEFEFA33-3818-A1D7-D106-D6B45917CF12> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_lsprof.cpython-35m-darwin.so
       0x108800000 -        0x108809fff +context.cpython-35m-darwin.so (???) <DE2901DD-2FFF-342E-85B0-A3DC3CABFBB9> /Users/USER/*/context.cpython-35m-darwin.so
       0x108813000 -        0x10882bff7 +socket.cpython-35m-darwin.so (???) <84422136-5855-3733-A306-48E49F155F97> /Users/USER/*/socket.cpython-35m-darwin.so
       0x10883d000 -        0x108845fff +_poll.cpython-35m-darwin.so (???) <59A57DF6-617C-3ED2-B14B-C768069D5BF1> /Users/USER/*/_poll.cpython-35m-darwin.so
       0x10884e000 -        0x108855ff7 +_device.cpython-35m-darwin.so (???) <0F9454AC-533E-31EB-B1D4-60EFE1EE9CCE> /Users/USER/*/_device.cpython-35m-darwin.so
       0x10895d000 -        0x108963fff +_json.cpython-35m-darwin.so (???) <CC189DA1-AAD4-DD9C-FED2-F3E1BD593AC1> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_json.cpython-35m-darwin.so
       0x108b28000 -        0x108be1fe7 +_sqlite3.cpython-35m-darwin.so (???) <319D7117-B790-F43C-E06E-2AE255A999C9> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_sqlite3.cpython-35m-darwin.so
       0x108d00000 -        0x108db7fff +unicodedata.cpython-35m-darwin.so (???) <26739E32-86E2-96AA-ED69-96D894449EF3> /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/unicodedata.cpython-35m-darwin.so
    0x7fff690d2000 -     0x7fff6910fdc7  dyld (433.5) <322C06B7-8878-311D-888C-C8FD2CA96FF3> /usr/lib/dyld
    0x7fffa8758000 -     0x7fffa8758fff  com.apple.Accelerate (1.11 - Accelerate 1.11) <916E360F-323C-3AE1-AB3D-D1F3B284AEE9> /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate
    0x7fffa8771000 -     0x7fffa8c8afeb  com.apple.vImage (8.1 - ???) <B58A7937-BEE2-38FE-87F4-5D5F40D31DC9> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImage.framework/Versions/A/vImage
    0x7fffa8c8b000 -     0x7fffa8dfcff3  libBLAS.dylib (1185.50.4) <4087FFE0-627E-3623-96B4-F0A9A1991E09> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
    0x7fffa8dfd000 -     0x7fffa8e11ffb  libBNNS.dylib (15) <254698C7-7D36-3FFF-864E-ADEEEE543076> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBNNS.dylib
    0x7fffa8e12000 -     0x7fffa9208fef  libLAPACK.dylib (1185.50.4) <C35FFB2F-A0E6-3903-8A3C-113A74BCBCA2> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
    0x7fffa9209000 -     0x7fffa921ffff  libLinearAlgebra.dylib (1185.50.4) <345CAACF-7263-36EF-B69B-793EA8B390AF> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLinearAlgebra.dylib
    0x7fffa9220000 -     0x7fffa9226fff  libQuadrature.dylib (3) <EF56C8E6-DE22-3C69-B543-A8648F335FDD> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libQuadrature.dylib
    0x7fffa9227000 -     0x7fffa923bff7  libSparseBLAS.dylib (1185.50.4) <67BA432E-FB59-3C78-A8BE-ED4274CBC359> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libSparseBLAS.dylib
    0x7fffa923c000 -     0x7fffa93c3fe7  libvDSP.dylib (600.60.1) <4155F45B-41CD-3782-AE8F-7AE740FD83C3> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvDSP.dylib
    0x7fffa93c4000 -     0x7fffa9476fff  libvMisc.dylib (600.60.1) <E18365D7-DCC4-3304-A8D1-395E656D7B99> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvMisc.dylib
    0x7fffa9477000 -     0x7fffa9477fff  com.apple.Accelerate.vecLib (3.11 - vecLib 3.11) <7C5733E7-0568-3E7D-AF61-160F19FED544> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib
    0x7fffabbfc000 -     0x7fffac095ff7  com.apple.CoreFoundation (6.9 - 1349.8) <09ED473E-5DE8-307F-B55C-16F6419236D5> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fffadbdb000 -     0x7fffadc71ff7  com.apple.framework.IOKit (2.0.2 - 1324.60.3) <7CE4C98B-107C-3AAA-B49A-F2ACFCBBF526> /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
    0x7fffb1f09000 -     0x7fffb220bff7  com.apple.security (7.0 - 57740.60.18) <005E8C96-40B6-35E3-B58B-888A5F5957C2> /System/Library/Frameworks/Security.framework/Versions/A/Security
    0x7fffb2636000 -     0x7fffb26a5ff7  com.apple.SystemConfiguration (1.14 - 1.14) <3FFD3930-B307-3E31-9FA6-12221EB0F969> /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
    0x7fffbe8cf000 -     0x7fffbe8d0fff  com.apple.TrustEvaluationAgent (2.0 - 28.50.1) <EBE65DD5-1732-3747-8C6C-7BECEBF089A4> /System/Library/PrivateFrameworks/TrustEvaluationAgent.framework/Versions/A/TrustEvaluationAgent
    0x7fffbfc22000 -     0x7fffbfc23ff3  libDiagnosticMessagesClient.dylib (102) <84A04D24-0E60-3810-A8C0-90A65E2DF61A> /usr/lib/libDiagnosticMessagesClient.dylib
    0x7fffbfe5b000 -     0x7fffbfe5bfff  libOpenScriptingUtil.dylib (172) <90743888-C1E8-34E3-924E-1A754B2B63B9> /usr/lib/libOpenScriptingUtil.dylib
    0x7fffbfe61000 -     0x7fffbfe62ffb  libSystem.B.dylib (1238.60.2) <F18AC1E7-C6F1-34B1-8069-BE571B3231D4> /usr/lib/libSystem.B.dylib
    0x7fffbff7a000 -     0x7fffbff7aff3  libauto.dylib (187) <34388D0B-C539-3C1B-9408-2BC152162E43> /usr/lib/libauto.dylib
    0x7fffbff7b000 -     0x7fffbff8bff3  libbsm.0.dylib (34) <20084796-B04D-3B35-A003-EA11459557A9> /usr/lib/libbsm.0.dylib
    0x7fffbff8c000 -     0x7fffbff9aff7  libbz2.1.0.dylib (38) <ADFA329A-DCE7-356D-8F09-A3168DFC6610> /usr/lib/libbz2.1.0.dylib
    0x7fffbff9b000 -     0x7fffbfff1ff7  libc++.1.dylib (307.5) <0B43BB5D-E6EB-3464-8DE9-B41AC8ED9D1C> /usr/lib/libc++.1.dylib
    0x7fffbfff2000 -     0x7fffc001bff7  libc++abi.dylib (307.4) <BC271AD3-831B-362A-9DA7-E8C51F285FE4> /usr/lib/libc++abi.dylib
    0x7fffc0044000 -     0x7fffc0044ff7  libcoretls.dylib (121.50.4) <64B1001E-10F6-3542-A3B2-C4B49F51817F> /usr/lib/libcoretls.dylib
    0x7fffc0045000 -     0x7fffc0046ff3  libcoretls_cfhelpers.dylib (121.50.4) <1A10303E-5EB0-3C7C-9165-021FCDFD934D> /usr/lib/libcoretls_cfhelpers.dylib
    0x7fffc0100000 -     0x7fffc01e5ff7  libcrypto.0.9.8.dylib (64.50.6) <D34E16A7-990A-37A9-933A-DFAA46554EAA> /usr/lib/libcrypto.0.9.8.dylib
    0x7fffc0453000 -     0x7fffc0453fff  libenergytrace.dylib (15) <A1B040A2-7977-3097-9ADF-34FF181EB970> /usr/lib/libenergytrace.dylib
    0x7fffc055c000 -     0x7fffc0781ffb  libicucore.A.dylib (57166.0.1) <CCD2ED24-3071-383B-925D-8D763BB12A6F> /usr/lib/libicucore.A.dylib
    0x7fffc0789000 -     0x7fffc07a2ffb  liblzma.5.dylib (10) <44BD0279-99DD-36B5-8A6E-C11432E2098D> /usr/lib/liblzma.5.dylib
    0x7fffc0a95000 -     0x7fffc0b0fff3  libnetwork.dylib (856.60.1) <191E99F5-4723-3180-8013-02AF2F9AE4B8> /usr/lib/libnetwork.dylib
    0x7fffc0b10000 -     0x7fffc0ee2047  libobjc.A.dylib (709.1) <70614861-0340-32E2-85ED-FE65759CDFFA> /usr/lib/libobjc.A.dylib
    0x7fffc0ee5000 -     0x7fffc0ee9fff  libpam.2.dylib (21.30.1) <71EB0D88-DE84-3C8D-A2C5-58AA282BC5BC> /usr/lib/libpam.2.dylib
    0x7fffc0eea000 -     0x7fffc0f1bfff  libpcap.A.dylib (67.60.1) <F6BC6ED6-AEE4-3520-B248-0C342636E2B0> /usr/lib/libpcap.A.dylib
    0x7fffc0fa4000 -     0x7fffc10f2ff7  libsqlite3.dylib (254.8) <1ECF7DF7-7A07-3B4B-A63B-F4EFF6BC7ACF> /usr/lib/libsqlite3.dylib
    0x7fffc1115000 -     0x7fffc114dff3  libssl.0.9.8.dylib (64.50.6) <9A9C9D9A-7948-3412-ABE4-01FCC4A72CD2> /usr/lib/libssl.0.9.8.dylib
    0x7fffc114e000 -     0x7fffc119efff  libstdc++.6.dylib (104.1) <A980E08C-A511-3D19-9881-1D79B7CFF2BA> /usr/lib/libstdc++.6.dylib
    0x7fffc11e7000 -     0x7fffc11f4fff  libxar.1.dylib (357) <69547C64-E811-326F-BBED-490C6361BDCB> /usr/lib/libxar.1.dylib
    0x7fffc11f5000 -     0x7fffc12e4fff  libxml2.2.dylib (30.17) <D4C3E467-37C6-34DE-8A1B-E7BFC29985E9> /usr/lib/libxml2.2.dylib
    0x7fffc130f000 -     0x7fffc1320ff3  libz.1.dylib (67) <46E3FFA2-4328-327A-8D34-A03E20BFFB8E> /usr/lib/libz.1.dylib
    0x7fffc132f000 -     0x7fffc1333ff7  libcache.dylib (79) <093A4DAB-8385-3D47-A350-E20CB7CCF7BF> /usr/lib/system/libcache.dylib
    0x7fffc1334000 -     0x7fffc133efff  libcommonCrypto.dylib (60092.50.5) <8A64D1B0-C70E-385C-92F0-E669079FDA90> /usr/lib/system/libcommonCrypto.dylib
    0x7fffc133f000 -     0x7fffc1346fff  libcompiler_rt.dylib (62) <55D47421-772A-32AB-B529-1A46C2F43B4D> /usr/lib/system/libcompiler_rt.dylib
    0x7fffc1347000 -     0x7fffc134ffff  libcopyfile.dylib (138) <819BEA3C-DF11-3E3D-A1A1-5A51C5BF1961> /usr/lib/system/libcopyfile.dylib
    0x7fffc1350000 -     0x7fffc13d3fdf  libcorecrypto.dylib (442.50.19) <65D7165E-2E71-335D-A2D6-33F78E2DF0C1> /usr/lib/system/libcorecrypto.dylib
    0x7fffc13d4000 -     0x7fffc1405fff  libdispatch.dylib (703.50.37) <6582BAD6-ED27-3B30-B620-90B1C5A4AE3C> /usr/lib/system/libdispatch.dylib
    0x7fffc1406000 -     0x7fffc140bffb  libdyld.dylib (433.5) <9B2AC56D-107C-3541-A127-9094A751F2C9> /usr/lib/system/libdyld.dylib
    0x7fffc140c000 -     0x7fffc140cffb  libkeymgr.dylib (28) <7AA011A9-DC21-3488-BF73-3B5B14D1FDD6> /usr/lib/system/libkeymgr.dylib
    0x7fffc140d000 -     0x7fffc1419fff  libkxld.dylib (3789.70.16) <BD02EF3F-6E09-3A64-AB21-553D5B1D2859> /usr/lib/system/libkxld.dylib
    0x7fffc141a000 -     0x7fffc141afff  liblaunch.dylib (972.70.1) <B856ABD2-896E-3DE0-B2C8-146A6AF8E2A7> /usr/lib/system/liblaunch.dylib
    0x7fffc141b000 -     0x7fffc1420ff3  libmacho.dylib (898) <17D5D855-F6C3-3B04-B680-E9BF02EF8AED> /usr/lib/system/libmacho.dylib
    0x7fffc1421000 -     0x7fffc1423ff3  libquarantine.dylib (85.50.1) <12448CC2-378E-35F3-BE33-9DC395A5B970> /usr/lib/system/libquarantine.dylib
    0x7fffc1424000 -     0x7fffc1425ffb  libremovefile.dylib (45) <38D4CB9C-10CD-30D3-8B7B-A515EC75FE85> /usr/lib/system/libremovefile.dylib
    0x7fffc1426000 -     0x7fffc143eff7  libsystem_asl.dylib (349.50.5) <096E4228-3B7C-30A6-8B13-EC909A64499A> /usr/lib/system/libsystem_asl.dylib
    0x7fffc143f000 -     0x7fffc143fff7  libsystem_blocks.dylib (67) <10DC5404-73AB-35B3-A277-A8AFECB476EB> /usr/lib/system/libsystem_blocks.dylib
    0x7fffc1440000 -     0x7fffc14cdfef  libsystem_c.dylib (1158.50.2) <E5AE5244-7D0C-36AC-8BB6-C7AE7EA52A4B> /usr/lib/system/libsystem_c.dylib
    0x7fffc14ce000 -     0x7fffc14d1ffb  libsystem_configuration.dylib (888.60.2) <BECC01A2-CA8D-31E6-BCDF-D452965FA976> /usr/lib/system/libsystem_configuration.dylib
    0x7fffc14d2000 -     0x7fffc14d5fff  libsystem_coreservices.dylib (41.4) <7D26DE79-B424-3450-85E1-F7FAB32714AB> /usr/lib/system/libsystem_coreservices.dylib
    0x7fffc14d6000 -     0x7fffc14eefff  libsystem_coretls.dylib (121.50.4) <EC6FCF07-DCFB-3A03-9CC9-6DD3709974C6> /usr/lib/system/libsystem_coretls.dylib
    0x7fffc14ef000 -     0x7fffc14f5fff  libsystem_dnssd.dylib (765.50.9) <CC960215-0B1B-3822-A13A-3DDE96FA796F> /usr/lib/system/libsystem_dnssd.dylib
    0x7fffc14f6000 -     0x7fffc151fff7  libsystem_info.dylib (503.50.4) <611DB84C-BF70-3F92-8702-B9F28A900920> /usr/lib/system/libsystem_info.dylib
    0x7fffc1520000 -     0x7fffc1542ff7  libsystem_kernel.dylib (3789.70.16) <34B1F16C-BC9C-3C5F-9045-0CAE91CB5914> /usr/lib/system/libsystem_kernel.dylib
    0x7fffc1543000 -     0x7fffc158afe7  libsystem_m.dylib (3121.6) <86D499B5-BBDC-3D3B-8A4E-97AE8E6672A4> /usr/lib/system/libsystem_m.dylib
    0x7fffc158b000 -     0x7fffc15a9ff7  libsystem_malloc.dylib (116.50.8) <A3D15F17-99A6-3367-8C7E-4280E8619C95> /usr/lib/system/libsystem_malloc.dylib
    0x7fffc15aa000 -     0x7fffc1603ffb  libsystem_network.dylib (856.60.1) <369D0221-56CA-3C3E-9EDE-94B41CAE77B7> /usr/lib/system/libsystem_network.dylib
    0x7fffc1604000 -     0x7fffc160dff3  libsystem_networkextension.dylib (563.60.2) <B021F2B3-8A75-3633-ABB0-FC012B8E9B0C> /usr/lib/system/libsystem_networkextension.dylib
    0x7fffc160e000 -     0x7fffc1617ff3  libsystem_notify.dylib (165.20.1) <B8160190-A069-3B3A-BDF6-2AA408221FAE> /usr/lib/system/libsystem_notify.dylib
    0x7fffc1618000 -     0x7fffc1620fe7  libsystem_platform.dylib (126.50.8) <897462FD-B318-321B-A554-E61982630F7E> /usr/lib/system/libsystem_platform.dylib
    0x7fffc1621000 -     0x7fffc162bff7  libsystem_pthread.dylib (218.60.3) <B8FB5E20-3295-39E2-B5EB-B464D1D4B104> /usr/lib/system/libsystem_pthread.dylib
    0x7fffc162c000 -     0x7fffc162fff7  libsystem_sandbox.dylib (592.70.1) <4B92EC49-ACD0-36AE-B07A-A2B8152EAF9D> /usr/lib/system/libsystem_sandbox.dylib
    0x7fffc1630000 -     0x7fffc1631ff3  libsystem_secinit.dylib (24.50.4) <F78B847B-3565-3E4B-98A6-F7AD40392E2D> /usr/lib/system/libsystem_secinit.dylib
    0x7fffc1632000 -     0x7fffc1639ffb  libsystem_symptoms.dylib (532.50.47) <3390E07C-C1CE-348F-ADBD-2C5440B45EAA> /usr/lib/system/libsystem_symptoms.dylib
    0x7fffc163a000 -     0x7fffc164dff7  libsystem_trace.dylib (518.70.1) <AC63A7FE-50D9-3A30-96E6-F6B7FF16E465> /usr/lib/system/libsystem_trace.dylib
    0x7fffc164e000 -     0x7fffc1653ffb  libunwind.dylib (35.3) <3D50D8A8-C460-334D-A519-2DA841102C6B> /usr/lib/system/libunwind.dylib
    0x7fffc1654000 -     0x7fffc167dff7  libxpc.dylib (972.70.1) <BF896DF0-D8E9-31A8-A4B3-01120BFEEE52> /usr/lib/system/libxpc.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 1189
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=188.9M resident=0K(0%) swapped_out_or_unallocated=188.9M(100%)
Writable regions: Total=164.1M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=164.1M(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Dispatch continuations            16.0M        2 
Kernel Alloc Once                    8K        2 
MALLOC                           127.3M       80 
MALLOC guard page                   16K        4 
MALLOC_LARGE (reserved)           1956K        4         reserved VM address space (unallocated)
Stack                             1560K        4 
Stack Guard                         12K        4 
VM_ALLOCATE                          4K        2 
__DATA                            7464K      233 
__LINKEDIT                       122.7M      130 
__TEXT                            66.2M      204 
__UNICODE                          556K        2 
__UNIXSTACK                       16.0M        2 
mapped file                       24.7M        2 
shared memory                       44K        5 
===========                     =======  ======= 
TOTAL                            384.2M      665 
TOTAL, minus reserved VM space   382.3M      665 

Model: MacBookPro11,2, BootROM MBP112.0138.B40, 4 processors, Intel Core i7, 2 GHz, 16 GB, SMC 2.18f15
Graphics: Intel Iris Pro, Intel Iris Pro, Built-In
Memory Module: BANK 0/DIMM0, 8 GB, DDR3, 1600 MHz, 0x80AD, 0x484D54343147533641465238412D50422020
Memory Module: BANK 1/DIMM0, 8 GB, DDR3, 1600 MHz, 0x80AD, 0x484D54343147533641465238412D50422020
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x134), Broadcom BCM43xx 1.0 (7.21.171.130.1a1)
Bluetooth: Version 5.0.5f1, 3 services, 27 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en0
Serial ATA Device: APPLE SSD SM0512G, 500.28 GB
USB Device: USB 3.0 Bus
USB Device: Apple Internal Keyboard / Trackpad
USB Device: BRCM20702 Hub
USB Device: Bluetooth USB Host Controller
Thunderbolt Bus: MacBook Pro, Apple Inc., 17.1

daniviga commented 7 years ago

How did you installed oq? Using:

our package
virtualenv
- using the provide requirements-py*-macos.txt
- just running setup.py
other (macPort, brew...) ?

It looks like a fork in a non fork-safe area of the code. We had similar issues in the past caused by Apple's customized libraries.

daniviga commented 7 years ago

I can confirm the issue with both Python 2.7 and 3.5 using either dependencies from pypi and our wheels.

@micheles it works fine in all cases using oq run. It should use futures in the same way, am I wrong?

$ oq run test/event_based_risk/inputs/case_master/job.ini
[cut]
[2017-10-18 21:03:48,222 #41 INFO] Sent 545.79 KB of data in 2 task(s)
[2017-10-18 21:03:48,247 #41 INFO] Submitting 2 "event_based_risk#2" tasks
[2017-10-18 21:03:48,249 #41 INFO] Sent 297.74 KB of data in 2 task(s)
[2017-10-18 21:03:55,500 #41 INFO] event_based_risk#1  50%
[2017-10-18 21:04:06,112 #41 INFO] event_based_risk#1 100%
[2017-10-18 21:04:06,271 #41 INFO] Received 11.86 MB of data, maximum per task 11.26 MB
[2017-10-18 21:04:06,273 #41 INFO] event_based_risk#2  50%
[2017-10-18 21:04:06,295 #41 INFO] event_based_risk#2 100%
[2017-10-18 21:04:06,318 #41 INFO] Received 605.08 KB of data, maximum per task 547.94 KB
[2017-10-18 21:04:06,321 #41 INFO] Generated 8.19 MB of GMFs
[2017-10-18 21:04:15,245 #41 INFO] Instantiating LossRatiosGetters
[2017-10-18 21:04:15,275 #41 INFO] Submitting 7 "build_curves_maps" tasks
[2017-10-18 21:04:15,288 #41 INFO] Sent 10.34 MB of data in 7 task(s)
[2017-10-18 21:04:15,361 #41 INFO] build_curves_maps  14%
[2017-10-18 21:04:15,473 #41 INFO] build_curves_maps  28%
[2017-10-18 21:04:15,520 #41 INFO] build_curves_maps  42%
[2017-10-18 21:04:15,545 #41 INFO] build_curves_maps  57%
[2017-10-18 21:04:15,598 #41 INFO] build_curves_maps  71%
[2017-10-18 21:04:15,680 #41 INFO] build_curves_maps  85%
[2017-10-18 21:04:15,690 #41 INFO] build_curves_maps 100%
[2017-10-18 21:04:15,696 #41 INFO] Received 29.16 KB of data, maximum per task 4.17 KB
[2017-10-18 21:04:15,724 #41 INFO] Total time spent: 28.684799909591675 s
[2017-10-18 21:04:15,724 #41 INFO] Memory allocated: 53.34 MB
See the output with hdfview /Users/jenkins/oqdata/calc_41.hdf5

micheles commented 7 years ago

The only difference between oq run and oq engine on a single machine is the logging (oq engine uses the DbServer and logs on the db). Could this be an issue with sqlite? Remember that we had an issue on macOS. Perhaps is still there but less visible than before. My hint come from this line in the segfault report:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libdispatch.dylib               0x00007fffc13ea749 _dispatch_queue_push + 171

The issue with sqlite was in libdispatch.dylib if I remember correctly.

Anirudh, can you run the same test with engine 2.6? My bet is the that the problem is there too. Engine 2.5 should be safe, though.

daniviga commented 7 years ago

Yes @micheles that was my suspect too. I'm now compiling the linux standalone package on macOS which embeds its own copy of Sqlite and Python compiled against it, to see it something changes.

daniviga commented 7 years ago

@micheles @raoanirudh Engine 2.6.0 works for me (Python 3.5 + our wheels, macOS 10.11). Should we considering this a regression blocking the release?

daniviga commented 7 years ago

I've been able to bisect code and after some trials this is the PR that breaks the code: https://github.com/gem/oq-engine/pull/3093:

up to 9e46030d76779af41c396d3517061acf981bf73c (merge of #3088) it works from 8a5f6376e4261f75c95ff87656a95c644bcf1b43 (merge of #3093) it breaks

daniviga commented 7 years ago

@raoanirudh could you try this branch please? https://github.com/gem/oq-engine/tree/revert-3093

raoanirudh commented 7 years ago

@daniviga Works fine with the branch revert-3093

micheles commented 7 years ago

I confirm that the issue is in check_obsolete_version. The only thing done there is a call to requests. However by replacing requests with the standard library

        req = Request(OQ_API + '/engine/latest', headers=headers)
        data = urlopen(req, timeout=1).read()  # bytes
        tag_name = json.loads(decode(data))['tag_name']

the segfault is still there :-( The problem is the call to urlopen even if it seems absurd.

micheles commented 7 years ago

The workaround I suggest is to change oq_distribute from futures to zmq in the file openquake.cfg and everything will work. I would not block the release for this issue. It would be interesting to see if there are other calculations affected by this issue, especially real examples.

gem / oq-engine

Segmentation fault for event_based_risk on macOS #3161

Engine log

System log