search

NNPDF / nnpdf

An open-source machine learning framework for global analyses of parton distributions.
https://docs.nnpdf.science/
GNU General Public License v3.0
28 stars 6 forks source link

Sometimes test_pseudodata errors on Mac #931

Closed wilsonmr closed 1 year ago

wilsonmr commented 3 years ago

Welcome back sports fans, we have another exciting round of Mac errors which make no sense.

This time in the blue corner we have test_pseudodata which can throw the following error on travis:

==================================== ERRORS ====================================

______________________ ERROR at setup of test_pseudodata _______________________

    @pytest.fixture(scope="session", autouse=True)

    def setup_dicts():

        exp_infos_bytes = read_binary(validphys.tests.regressions, "test_exp_infos.pickle")

        ns = yaml.safe_load(EXAMPLE_RUNCARD)

        # This is what all the fitted replicas saw

        exp_infos = pickle.loads(exp_infos_bytes)

        # We now need to convert these to postfit replicas

        fitted_indices = API.fitted_replica_indexes(**ns)

        fit_postfit_mapping = dict(enumerate(exp_infos, 1))

        exp_infos = [fit_postfit_mapping[i] for i in fitted_indices]

>       pseudodata_info = API.get_pseudodata(**ns)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.7/site-packages/validphys/tests/test_pseudodata.py:57: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.7/site-packages/reportengine/api.py:43: in closure

    return self.__call__(name, **kwargs)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.7/site-packages/reportengine/api.py:37: in __call__

    builder.execute_sequential()

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.7/site-packages/reportengine/resourcebuilder.py:168: in execute_sequential

    perform_final=self.perform_final)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.7/site-packages/reportengine/resourcebuilder.py:175: in get_result

    fres =  function(**kwdict)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.7/site-packages/validphys/pseudodata.py:156: in get_pseudodata

    return [fitted_pseudodata[i] for i in fitted_replica_indexes]

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

.0 = <list_iterator object at 0x1a1dc943d0>

>   return [fitted_pseudodata[i] for i in fitted_replica_indexes]

E   KeyError: 7

I'll see if I can understand this problem locally. In a dev env I find the tests pass just fine, but I seem to remember being able to reproduce this with conda-build so hopefully I can get some insight

wilsonmr commented 3 years ago

some kind of segfaul

#Python tests for the installed validphys package
pytest --pyargs --mpl validphys -k test_pseudodata -s
============================= test session starts ==============================
platform darwin -- Python 3.7.9, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
Matplotlib: 3.3.1
Freetype: 2.10.2
rootdir: $SRC_DIR
plugins: mpl-0.11, hypothesis-5.35.4
collected 54 items / 52 deselected / 2 selected

[INFO]: Could not find a resource (pdf): pseudodata_test_fit. Attempting to download it.
[INFO]: Downloading https://data.nnpdf.science/fits/pseudodata_test_fit.tar.gz.
[INFO]: Extracting archive to /Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/NNPDF/results/fit_download_deleteme_vcca_xc6
[INFO]: Could not find a resource (theoryID): Could not find theory 162. Folder '/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/NNPDF/data/theory_162' not found. Attempting to download it.
[INFO]: Downloading https://nnpdf.web.cern.ch/nnpdf/tables/theory_162.tgz.
[INFO]: Extracting archive to /Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/NNPDF/data
[WARNING]: old fit found, falling back to old behaviour
[WARNING]: `experiments` has been deprecated, specify data using `dataset_inputs`. Any grouping defined by `experiments` is being ignored.
[INFO]: Could not find a resource (pdf): NNPDF31_nnlo_as_0118. Attempting to download it.
[INFO]: Downloading https://lhapdf.hepforge.org/downloads/pdfsets/6.1/NNPDF31_nnlo_as_0118.tar.gz.
[INFO]: Extracting archive to /Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF
[ERROR]: The original archive at /Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF/tmpxa5ka2zqNNPDF31_nnlo_as_0118.tar.gz was only extracted partially at 
/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF
[ERROR]: /Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF/tmpxa5ka2zqNNPDF31_nnlo_as_0118.tar.gz is not a compressed or uncompressed tar file. It seems the LHAPDF URLs aren't behaving, attempting to find resource in other repositories
[INFO]: Downloading https://data.nnpdf.science/pdfs/NNPDF31_nnlo_as_0118.tar.gz.
[INFO]: Extracting archive to /Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF
tests/test_pseudodata.py PDF: NNPDF31_nnlo_as_0118  ErrorType: Monte Carlo t0 booked
LHAPDF 6.2.3 loading $PREFIX/share/LHAPDF/NNPDF31_nnlo_as_0118/NNPDF31_nnlo_as_0118_0000.dat
NNPDF31_nnlo_as_0118 PDF set, member #0, version 1; LHAPDF ID = 303600
NNPDF31_nnlo_as_0118 Initialised with 1 members and errorType replicas

[WARNING]: Using all 4 cores available, this may be dangerous especially for use on a cluster. Consider setting the NPROC variable to something sensible.

-- Reading COMMONDATA for Dataset: NMCPD
nData: 260 nSys: 5

-- Reading COMMONDATA for Dataset: NMCPD
nData: 260 nSys: 5
-- COMMONDATA Files for NMCPD successfully read.

-- COMMONDATA Files for NMCPD successfully read.

_NMCPD______________________________________________________
NMCPD_D
260 Data Points 10 X points 4 active flavours
_NMCPD______________________________________________________
NMCPD_D
260 Data Points 10 X points 4 active flavours
_NMCPD______________________________________________________
NMCPD_P
260 Data Points 10 X points 5 active flavours

-- Reading COMMONDATA for Dataset: NMCPD
nData: 260 nSys: 5
_NMCPD______________________________________________________
NMCPD_P
260 Data Points 10 X points 5 active flavours
-- COMMONDATA Files for NMCPD successfully read.

_NMCPD______________________________________________________
NMCPD_D
260 Data Points 10 X points 4 active flavours

-- Reading COMMONDATA for Dataset: NMCPD
nData: 260 nSys: 5
_NMCPD______________________________________________________
NMCPD_P
260 Data Points 10 X points 5 active flavours
-- COMMONDATA Files for NMCPD successfully read.

_NMCPD______________________________________________________
NMCPD_D
260 Data Points 10 X points 4 active flavours

-- Reading COMMONDATA for Dataset: NMC
nData: 292 nSys: 16
-- Reading COMMONDATA for Dataset: NMC
nData: 292 nSys: 16
_NMCPD______________________________________________________
NMCPD_P
260 Data Points 10 X points 5 active flavours

-- Reading COMMONDATA for Dataset: NMC
nData: 292 nSys: 16

-- Reading COMMONDATA for Dataset: NMC
nData: 292 nSys: 16
-- COMMONDATA Files for NMC successfully read.

-- COMMONDATA Files for NMC successfully read.

-- COMMONDATA Files for NMC successfully read.

-- COMMONDATA Files for NMC successfully read.

_NMC________________________________________________________
NMC
292 Data Points 10 X points 9 active flavours
_NMC________________________________________________________
NMC
292 Data Points 10 X points 9 active flavours
_NMC________________________________________________________
NMC
292 Data Points 10 X points 9 active flavours
_NMC________________________________________________________
NMC
292 Data Points 10 X points 9 active flavours
Fatal Python error: Segmentation fault

Current thread 0x000000011764e5c0 (most recent call first):
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/NNPDF/nnpdf.py", line 6201 in __Fatal Python error: init__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/core.py", line 517 in load
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/n3fit/io/reader.py", line 223 in common_data_reader
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py", line 106 in task
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/process.py", line 99 in run
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/process.py", line 297 in _bootstrap
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/popen_fork.py", line 74 in _launch
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/popen_fork.py", line 20 in __init__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/context.py", line 277 in _Popen
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/context.py", line 223 in _Popen
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/process.py", line 112 in start
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py", line 141 in fitted_pseudodata_internal
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pSegmentation fault

Current thread 0x000000011764e5c0 (most recent call first):
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_lacehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/pythonp3l.a7c/eshiotled-_ppalcakcaegheosl/dr_epploarcteehnoglidn_ep/lraecseohuorlcde_bpulialcdeehro.lpdy_"placehold, line _175p in laceholgde_tp_lraecseuhlotl
d_placeho  File l"d/_Upslearcse/hmoilcdh_apella/cceohnodld_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/NNPDF/nnpdfa/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py", line 168 in execute_sequential
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py", line 37 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py", line 43 in closure
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/tests/test_pseudodata.py", line 57 in setup_dicts
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 924 in call_fixture_func
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 1117 in pytest_fixture_setup
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 1062 in execute
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 683 in _compute_fixture_value
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 601 in _get_active_fixturedef
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 581 in getfixturevalue
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 568 in _fillfixtures
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/python.py", line 1576 in setup
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 432 in prepare
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_plac.py"ehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 141 in pytest_runtest_setup
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 247 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 294 in from_call
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 247 in call_runtest_hook
  File "/Users/micha, line e6201l in /_c_oinndiat/_c_o
nd  File a-bld/"n/nUpsdefr_s1/6m0i0c7h8a9e6l7/7c1o5n1d/a_/tceosntd_ae-nbvl_dp/lnancpdefh_o1l6d0_0p7l8a9c6e7h7o1l5d1_/p_latceeshto_ledn_vp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalaccehold_placehold_placehold_placehold_placehold_placehold_placehoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_/plliabc/ephyotlhdo_np3l.a7c/eshiotled-_ppalcakcaegheosl/dv_aplli/dlpihby/sp/yctohroen.3p.y7"/site-, line p517a in clkoaagde
s  File /_pyt"e/sUts/errusn/nmeirc.hpaye"l, line /207c in ocnadlal/_caonndd_ar-ebplodr/tn
n  File pdf_1"6/0U0s7e8r9s6/7m7i1c5h1a/e_lt/ecsotn_dean/vc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/anc3efhiotl/di_op/lr/elaidbe/rp.yptyh"o, line n2233 in .c7o/msmiotne_-dpaatcak_argeeasd/e_rp
y  File tes"t//Ursuenrnse/rm.ipcyh"a, line e111l in /crounndtae/sctopnrdoat-obclodl/
nn  File pdf_"1/6U0s0e7r8s9/6m7i7c1h5a1e/l_/tceosntd_ae/ncvo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7a7c1e5h1o/l_dt_epslta_ceenhvo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_spiltaec-ephaoclkda_gpelsa/cveahloilddp_hpyls//lpisbe/updyotdhaotna3..p7y/"s, line i106t in e-pactkaasgke
s  File /_pyte"s/tU/sreurnsn/emri.cphya"e, line l100/ in copnydtae/scto_nrduan-tbelsdt/_npnrpodtfo_c1o6l0
0  File 7"8/9U6s7e7r1s5/1m/i_ctheasetl_/ecnovn_dpal/accoenhdoal-db_lpdl/ancnephdofl_d1_6p0l0a7c8e9h6o7l7d1_5p1l/a_cteehsotl_de_npvl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_/plliabc/ephyotlhdo_np3l.a7c/emhuolltdi_pprloacceeshsoilndg_/pplraocceehsosl.dp_yp"l/lib/, line p99y in trhuonn
3  File ."7//Ussietres-/pmaicckhaageels//cpolnudgag/yc/ocnadlal-ebrlsd./pnyn"p, line d187f in __1m6u0l0t7i8c9a6l7l7
151/_tes  File t"_/eUnsve_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocehlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/dm_uplltaicperhoocleds_spilnagc/ephroolcde_spsl.apcye"h, line o297l in d__bpoloatcsethroalpd
_  File pl/li"b//Upsyetrhso/nm3i.c7h/aseilt/ec-opnadcak/acgoensd/ap-lbulgdg/yn/nmpadnfa_g1e6r0.0p7y8"9, line 6877 in 7<1l5a1m/b_dtae>s
t_env_p  File l"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaa/cceohnodlad-_bpllda/cnenhpodlfd__1p6l0a0c7e8h9o6l7d7_1p5l1a/c_etheosltd__epnlva_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cmeuhlotlidp_rpolcaecseshionlgd/_ppolpaecne_hfoolrdk_.pplya"c, line e74h in ol_dl_apulnacche
h  File old_p"l//Ulsiebr/sp/ymtihcohna3e.l7//csointdea-/pcaocnkdaag-ebsl/dp/lnungpgdyf/_m1a6n0a0g7e8r9.6p7y7"1, line 5931 in /__htoeoskte_xeencv
_place  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7a7c1e5h1o/l_dt_epslta_ceenhvo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_mpullatciephroolcde_spsliancge/hpoolpde_np_lfaocrekh.oplyd"_, line p20l in a_c_eihnoiltd___p
l  File aceh"o/lUds_eprls//lmiibc/hpayetlh/ocno3n.d7a//sciotned-ap-abclkda/gnensp/dpfl_u1g6g0y0/7h8o9o6k7s7.1p5y1"/, line _286t in e_s_tc_aelnlv___p
laceh  File old_p"l/aUcseehrosl/dm_ipclhaaceelh/oclodn_dpal/accoenhdoal-db_lpdl/ancnephdofl_d1_6p0l0a7c8e9h6o7l7d1_5p1l/a_cteehsotl_de_npvl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_/plliabc/ephyotlhdo_np3l.a7c/emhuolltdi_pprloacceeshsoilndg_/pcloancteehxotl.dp_yp"lace, line h277o in l_dP_oppleanc
e  File h"o/lUds_eprlsa/cmeihcohlade_lp/lc/olnidba//pcyotnhdoan-3b.l7d//snintped-fp_a1c6k0a0g7e8s9/6_7p7y1t5e1s/t_/tmeasitn_.epnyv"_plac, line e321h in oplydt_epslta_creuhnotleds_tplloaocpe
h  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/dm_uplltaicperhoocleds_spilnagc/echoonltde_xptl.apcye"h, line o223l in d__Ppolpaecne
h  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppld/fl_i1b6/0p0y7t8h9o6n737.175/1s/i_ttee-spta_ceknavg_epsl/apcleuhgoglyd/_cpallalceerhso.lpdy_"p, line l187a in c_emhuolltdi_cpallalc
e  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7a7c1e5h1o/l_dt_epslta_ceenhvo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_mpullatciephroolcde_spsliancge/hporlodc_epslsa.cpeyh"o, line l112d in _sptlaarcte
h  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6/7l7i1b5/1p/y_ttheosnt3_.e7n/vs_iptlea-cpeahcokladg_epsl/apcleuhgoglyd/_mpalnaacgeehro.lpdy_"p, line l87a in c<elhaomlbdd_ap>l
acehol  File d"_/pUlsaecresh/omlidc_hpalealc/echoonldda_/pcloancdeah-oblldd_/pnlnapcdefh_o1l6d0_0p7l8a9c6e7h7o1l5d1_/p_ltaecseth_oelndv__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppll/alciebh/oplydt_hpolna3c.e7h/oslidt_ep-lpaaccekhaogleds_/pvlaalciedhpohlyds_/pplsaecuedhoodladt_ap.lpayc"e, line h141o in lfdi_tptleadc_ephsoeludd_opdlaatcae_hionltde_rpnlaalc
ehold_pl  File acehold_"p/lUasceerhso/lmdi_cphla/elli/bc/opnydtah/ocno3n.d7a/-sbiltde/-npnapcdkfa_g1e6s0/0p7l8u9g6g7y7/1m5a1n/a_gteers.tp_ye"n, line v93_ in p_lhaocoekheoxledc_
placeh  File o"l/dU_splacehold_placehold_placehold_plers/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_palcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lp/ll/ilbi/bp/yptyhtohno3n.37./7s/istiet-ep-apcakcakgaegse/sr/e_ppoyrtteesntg/imnaei/nr.epsyo"ur, line c296e in b_umialidne
r.py"  File "/Us, line e175r in s/micgheate_lr/ecsounldta
/conda-b  File l"d//Unsneprdsf/_m1i6c0h0a7e8l9/6c7o7n1d5a1//c_otnedsat-_belndv/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptleascte_heonlvd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpdy_tphloanc3e.h7o/lsdi_tpel-apcaechkoalgde_sp/l_/plyitbe/spty/tmhaoinn3..p7y/"s, line i240t in ew-rpaapc_ksaegsessi/orne
porte  File n"g/iUnsee/rrse/smoiucrhcaeeblu/iclodnedra./pcyo"nda-, line b168l in d/nenxpedcfu_t1e6_0s0e7q8u9e6n7t7i1a5l1
/_test  File _env_pl"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaa/cceohnodlad-_bpllda/cnenhpodlfd__1p6l0a0c7e8h9o6l7d7_1p5l1a/c_etheosltd__epnlva_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpehlold_pl/lib/python3.7/site-packages/_pytest/main.py", line 289 in pytest_cmdline_main
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelh/ollidb_/pplyatcheohno3l.d7_/psliatcee-hpoalcdk_apgleasc/erheoplodr_tpelnagcienheo/ladp_ip.lpayc"ehold_, line p37l in a_c_echaollld___p
laceho  File ld_"p/lUasceerhso/lmdi_cphlaaecle/hcoolndd_ap/lc/olnidba/-pbyltdh/onnn3p.d7f/_s1i6t0e0-7p8a9c6k7a7g1e5s1//p_ltuegsgty_/ecnavl_lpelrasc.ephyo"l, line d187_ in p_lmauclethiocladl_lp
laceh  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lapcle/hloilbd/_ppyltahcoenh3o.l7d/_spiltaec-ephaoclkda_gpelsa/creehpoolrdt_epnlgaicneeh/oalpdi_.pplya"c, line e43h in old_placcleohsoulrde_
placeh  File old_plac"e/hUosledr_sp/lm/ilcihba/eply/tchoonnd3a./7/sicte-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plaFatal Python error: ceSegmentation faulth

old_pCurrent thread 0xlFatal Python error: acSegmentation fault000000011764e5c0e

 (most recent call first):
ho  File lCurrent thread 0xd_p"000000011764e5c0l (most recent call first):
/a  File Ucseehr"os/l/Udsm_iepcrhlsaae/cmle/ihccoohlnadde_alp//lcc/oonnlddiaab/-/cbpolyndt/hdnoann-p3bd.fl7d_//1nsn6i0pt0de7f-8_p19a66c07k70a71g85e916/s7_/t7_e1ps5y1tt/_ee_snttve/s_cptol_neafcnievg_h/po_ll_daic_nepihltoa_lc_de.h_popyllda"_cpelhaocledh_, line op158ll in adc_mephlaoaiclned
_h  File opllda_cpelhaocle"dh/o_Uplsdle_arcpsle/ahmcoilecdhh_oapleldla_/cpcleohancodleahd/_copollndad_cape-lhbaolcldde/_hnponllpdadc_fpe_lh1ao6lc0ed0h_7po8ll9ad6c_7ep7lh1oa5lc1de/h__potlleadsc_tep_hleaonlcvde__hpopllladacc_eephlhoaolclde_dhpo_llpadlc_aepchleoahlocdel_hdpo_llpadlc_aepchleohalocdel_dhp_olplaldca_ecpheloahlcodel_hdpo_lplaldca_epchleoahlcodle_dhp_olpladlc_aepchleohalocdel_hdpo_llpadlc_aepcheloahlcodel_hdpo_llpadlc_aepcheloahlcodel_hdp_olplladacc_eephlho/olllddi_b_p/pllpay/cltehihboon/lp3dy_.tph7l/oasnc3iet.h7eo-/lspdai_ctpekla-apgceaesch/koaNgleNdsP_/DpFNlN/anPcnDepFhd/ofnln.dpp_ydp"fl.apcye, line "h6201o in l_, line d_6201_ in ip_nl_iaitcn_ei_ht
o_  File l_d
_  File p"l/aU"csee/hUrossle/drm_sip/clmhaiacceehlha/oecloldn/_cdpoal/nadccoaen/hdcooal-nddb_alp-dlb/anlcndep/hdnonfl_pdd1_f6p_0l10/76b08i90n67/78p97y61t75e17s/1t5_"t1e/s_tt_e, line se11nt in _ve<_nmpvo_ldapuclleaech>oe
hlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohold_plld/_plli/bl/ipby/tphyotohnn3od.n7a3/-s.bi7lt/dse/-inptnaepc-dpkfaa_cg1ke6sa0/g0ev7as8l/9iv6ad7lp7ih1dy5ps1h//yc_sot/receso.trp_eye."npvy_"pl, line a517c in e, line lh517oo in aldld
o_a  File pdl
a  File c"e/h"oU/lsUdse_reprsls/a/mcmieichchoahleadel_l/p/clocanocndedaha/oc/locdon_dnpadla-ab-cbleldh/don/lnndnp_pdpdfl_fa_1c61e06h00o70l78d89_96p76l77a71c51e15h1/o_/l_tdte_espstl_ta_ecenenvhv_o_pllpdal_acpcelheaohcloeldhd_op_llpdla_acpcelheaohcoleldh_dop_lpldla_acpcelheaohcoledlh_dop_llpdal_capeclheahocoleldhd__popllladca_ecpelhhaooclleddh__oppllldaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla_capeclheaohcloedlh_dop_llpdal_capeclheaohcloeldhd_o_plpldla_acpeclehahocoleldh_dop_lpldla_acpcelehahocloedlh_dop_llpdal_capeclehahocloedlhd_o_pllpdal_capeclheahocoleldhd_o_plpldlaa_ccpeelhh/oollliddb__/pppllyaatccheeohnho3ol.dl7_d/p_slipatlceae-chpeoalhcdokl_apdgl_e/psll//ivlbai/lpbiy/dtpphyhotynhso3/.nt73e/.ssti7st//est-ieptsaet-_cppkasacegkueadgsoe/dnsa3/tfnai3.tfp/iyit"o//iroe/arde, line ea57rd in e.rp.ys"pey, line t"223u, line p in 223_cd in oicmcomtmosmn
_odna_tdaa_tra  File e_ardeeard
er  File 
"  File /"U/"s/UesUressre/rsms/im/cimhciahceahlea/elcl/o/cncodonand/dac/aoc/nocdnoadn-adb-al-bdbl/lnddn//pnndnnfpp_dd1ff6__0101676080090767878979617657717115/15_/1t/_e_tstetes_stet_n_even_nvpv_l_palpclaeachceohelhodlo_dlpd_lp_alpcaleachceohelhodol_dlp_dl_paplcalecahecohelhodol_dlp_dlp_alpcalecahceohelohdlo_ldpd_lp_alpcaelchaeohcleodhl_dopl_ldpa_lcapelchaeohcloedhl_odpl_ldpa_lcpaelchaeohcloedlh_dopl_ldpa_lcpaelchaeochelohdol_ldp_dlp_alpclaeachceohelohdlo_dlp_dlp_alpcalecahecohelhodol_ldpd_l_paplcalecahceoehlhodlo_dlp_dlp_alpcleaahccoeelhhdoo_llpddl__appclleaahccoeelhhdoo_llpddl__appllcaaeccheeohhloodll_ddp__lppallcaaeccheeohhloodll_ddp__lpalpcalecahecohelohdlo_dlpd_l_paplclaeachceoehlhodol_ldpd_l_paplclaeachceohelohdol_ldpd_l_paplclaeachceoehlhodol_ldpd_l_p/pll/l/liilbbi//bpp/ypytyththohonon3n3.3.7.7//7s/sisititee-t-pepa-acpckakacagkgeaessg//e_vspa/yvltaildeipsdhtpy/hsfy/ispx/stpeusuredeuosdd.aoptdyaa".tpay., line "p924y, line  in "106 in , line 106c in allt_aftsiakx
stk  File u
r  File e_f"u/n"Uc/s
Uesresr/sm/i  File mc"hi/caUhesalee/rlcs/o/cnmodinacd/hcaa/oencldo/anc-dobanl-ddbal//dncn/opnndndfpad_-1fb6_l01d06/07n08n97p68d79f76_171576110/501_7/t8_e9ts6te7_s7et1n_5ve1n_/pv_l_tpaelcseatch_eoehlnodv_l_pdpl_lpaaclceaechheoohllodd_l_pdp_llpaalccaeehchoeohlldodl__pdpl_lpaalcceaehcheoolhldod_l_pdpl_lapalccaeehcheoohlloddl__dpp_llpaaclceaechheoohlldod_l_pdp_llapalccaeechheoohlloddl__pdpl_lapaccleeahcohleodhl_odlp_dlp_alpcalecahechoeohlloddl__dpp_lpalclaeachceoehlhoodll_dpd_l_paplclaeahccoeehlhodol_ldpd_l_paplclaeachceoehlhodo_lldpdl__appclleaahccoeelhhdoo_llpddl__appclleaahccoeelhdho_olpldlda__cpeplhlaoaclcedeh_hoopllldda__cppellhaaoccleedhh_opollldda_c_peplhlaaoccleedh_hoopllldad_c_peplhlaoaccleedh_hopollldad__cpeplhlaoaclcedeh_hoopllldd/__lpiplbla//cpleyihtbo/hlpodny_3tp.hl7o//nlm3iu.bl7/t/pmiyputrlhotoicnpe3rs.os7ci/ensgsi/stipenr-gop/capecrskosa.cgpeeyss"s/._ppyy", line t99e in s, line tr99/u in fnri
ux  File nt
"u/  File rU"e/ssUe.srpesyr/"sm/imc, line ih1117ca in heale/lcp/oynctdoeans/dtca_o/fcnidoxant-dubarl-edb_/lsnden/tpnundppf
_d1f6_01  File 06708097687976"17/57U1s15e1/r/_st_/etmseitsc_htea_neevl_n/pvcl_oapncldeaahc/oeclhoodn_ldpdal_-apblclaedch/eonhlndop_ldpdfl__pa1lc6ea0ch0eo7hl8od9l_6dp7_l7pa1lc5ae1hc/oe_hltode_lspdtl__aepcnlevah_copellhadocl_edph_lopallcdae_hcpoellhadoc_ledph_loaplcldea_chpeolhlaodcl_edph_lopalcldae_hcpoelhladocl_epdh_lopalcldea_hcpoellhadoc_ledph_lopallcdae_hcpoellhadoc_lepdh_lopallcdae_chpoellhadocl_edphl_oaplcldea_hcpoellhadoc_lepdhl_oaplcldea_hcpoellahdc_oeplhldoa_lcpdelh_aopclledah_copellhdao_clpedlh_oaplcldea_hcpoellhdao_clpedlh_aopclldeah_cpoellhdao_clpedlh_aopclledah_copellhdao_clpedlh_apoclleadhc_oelphdlo_alpcdel_hapoclleadhc_oelphdlo_/lpldli/_blp/ilbpay/ctpehyohtnoh3l.od7n_/3p.mlu7al/ctmeiuhlpotrliodpc_reposlcsaeicsnesghi/onplrgdo/_pcprelosacsce.epshyso".l, line pd297y_ in "p_l, line b/297olo in it_bsb/toproyattspht
or  File na3p.
7  File /"s/iUt"se/e-rUpssae/crmksia/cgmheiasec/lhp/alceuolgn/gdcyao//ncdcaaoln/lcdeaor-nsbd.lapd-yb/"lndn/pnd, line nf187p_ in 1d6f0__10m76u08l09t67i78c97a61l57l17
/1_5t1  File /e_stt"e_/esUnts_vee_rnpsvl/_ampcilechahcoaelehdlo/_lcpdol_npadlcaaec/ehchooolnlddd_a_p-plblalacdce/ehnohnlpodd_lfdp__l1pa6lc0ea0hc7oe8lh9do6_l7pd7l_1ap5cl1ea/hc_oethleodsl_tpd_l_eapnclvae_hcpoelhlaodc_lepdhl_opallcdae_chpeolhaolcdle_dhp_olplladac_cepehlhoaolclded_hp_olplladac_cephelohaloclde_dh_popllladca_ecphelohalocdledh__oppllldaa_ccpeelhhaooclleddh__oppllldaa_ccpeelhhaooclleddh__oppllldaa_ccpeelhhaooclleddh__oppllldaa_ccpeelhhaooclleddh__oppllldaa_ccpeelhhaoocllded_h_popllladac_cepehlohaloclded_h_popllldaa_ccpeelhahocoleldhd_o_plpldla_acpcelehahocoleldhd_op_lplaldca_ecphelohaolclde_dhp_olpl/ldl/_ilpbil/bap/cypetyhthoholondn3_3.p7.l/7am/cumeulhtloitlpidpr_ropcolceaescsseshiionnlggd//_pppoolppaeecnne__hffooorlrkd.k_p.pyply"a", line c, line 74e74 in h in _o_llladau_nupcnlch/h
l
  File i  File b/p"y/t"Uh/soUensr3es./r7ms/i/scmihitaceeh-lape/alcco/kncadoganed/sac//opcnloduang-dgbayl-/dmb/anlndna/pgndenfrp_.1dp6fy0_"10670809, line 768787 in 791<65l17a/7m1_b5tde1a/s>_t
_t  File ee"ns/tvU__speelnravsc_/pemlhiaocclhedah_eoplll/dacc_oepnlhdaoacl/dech_oopnldldaa_-cpbelhlaodlc/den_hnpoplldadf_c_pe1lh6ao0lc0de7_h8po9ll6da7_c7pe1lh5ao1cl/ed_h_topellsadt_c_peelhnaovcl_edp_hlpoallcade_chpeolhloadlc_dep_hlopallcdae_chpeolhloadlc_dep_hlopallcdae_chpeolhlaodlc_dep_hlpoallcadec_hepohlloadcl_edph_loalpcdlea_hcpolelahdco_elphdlo_lpadlc_aecphelohalocdel_hdpo_llpadlca_ecphleoahlcodel_hdpo_lplaldca_ecphelohalocde_lhpdoll_apdcl_eaphcoellhadoc_lepdhl_oalpcldea_hpcolelahdco_elphdl_oaplcldea_hpcoellhadoc_lepdhl_aopclledah_copelhldao_clpdel_hapoclledahc_opellhdao_clpeldhao_clpedlh_/oplllidb/_/lpiplybat/chpeoyhnto3hlo.dn7_3/p.ml7ua/lcmteuihlotplirdpo_rcpoelscasecisenshigo/nlpgdo/_pppeolnpae_cnfe_ohfrooklr.dkp_.ypp"l, line ay20c" in e, line _h20_o in il_ndi__tp_il_n/i
lt  File i_b_/"
p/  File yUtsh"eor/nUs3s/.me7ir/css/himatieelc-/hpcaaeoclnkd/aacg/oecnsdo/anpd/lacu-ogbngldyda/-/mnbanlndpa/dgnfe_nrp1.d6pf0y_0"17, line 869390 in 60777_81h95o61o7/k7_e1txe5es1c/t
__  File teensv"_t/p_Uelsnaevcr_eshp/olmalicdce_hhpaoleallcd/e_chpoolnladcdae_/phclooalncdd_eaph-olblaldcd_e/hpnolnalpcdde_fhp_l1ao6cl0de0_h7po8ll9da6c_7ep7hl1oa5lc1de/_h_potlledasc_tep_hleonalvcd_e_phplolalacdcee_hhpololaldcd_e_phplloaalccdee_hphololalddc_e_phplolalacdce_ehhpoollladdc_e_phplolalacdce_ephhlooallcdde__hpplolalacdc_eephhloolaldcde__hppolllaadcc_eehphololalcdd_e_phplolalacdce_ehpholoalldcd_e_hppolllaadcc_eephholoalldcd_e_phplolaladcc_eephhloolaldcde__hppolllaadcc_epehlhoaolclded__hppollladac_cepehhlooallcdde__hppolllaadc_cepehlhoaolclded__hppolllaadcc_eephlhoaolclded_h_poplllad/cl_eiphlbo//lpldyi_btp/hlpoaynct3e.hho7on/l3md.u_7lpt/limapucrleothcioeplrdso_scpielsnagsci/enchogon/ltcdeo_xnpttl.eaxpctye."hpoyl, line "d277_ in , line p277_lP in aoc_pePehonop
le  File dn"_
/p  File Ul"s//elUrissb/e/rpsmy/itmchihocanhe3al.e/7lc//oscniodtae/n-cdpoaan/cdckaoan-gdbeals-d/b/plnldnup/gdngnfy_p/1dh6fo_0o01k76s80.90p76y87"79, line 1628657 in 17_/1__5c1at/l_elst_te__s
etn_ve_n  File pv"l_/apUlcsaeechreoshl/odml_idpc_lhpaalceaelch/oechloodnl_ddpa_l/paccloaenchdeoalh-dob_llpdd_l/panlcneaphcdoeflh_do1l_6pd0l_0ap7lc8ae9ch6oe7lh7od1l_5dp1l_/ap_clteaechseothl_odel_npdvl__appclleaahccoeelhhdo_olplldda__cppellahacocleedhho_olplldda__cppellhaaoccleedhho_olplldda__cppellahaoccleedhh_oopllldad_c_pelphaolcleadh_copellhdaoc_lephdl_oalpcdle_ahpcoellhadoc_lepdhl_oaplcldea_hcpoellhadoc_lephdl_oaplcldea_hcpolelahdco_elphdlo_alpcdle_ahpcoellhadco_elphdlo_aplcldea_hpcolelahdco_elphdlo_alpcdle_ahpcolelahdco_elphdlo_alpdcl_epahcloealchdeo_hlpdol_lapdcl_eaphcloe/lhldoi_blp/dpl_yaptclheo/hnlo3il.bd7/_/ppymltualhctoeinh3po.rl7od/c_empusllsaticinegph/rocolocdne_tspelsxaitcn.egph/yo"cl, line odn223_t in pe_lPxaotc.peephyno"
l, line   File d223"_ in /pU_lPsaoecpeerhnso
/l  File md"i_c/phUlas/eelrlis/b//cmpioycnthdhaaoe/nlc3o/.nc7do/ans-diabt/ledc-o/pnnandcapkd-abfg_le1ds6//0n_nppyte0s7t8/9f6i7x7t1u5r1e/s_.tpeys"t, line _1062e in nevx_epclacehold_placehold_placehold_pdlafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalaceholdu_tpel
a  File cehold"_/pUls/elrisb//mpiycthhaoenl3/.c7o/nmdual/tciopnrdoac-ebslsdi/nngn/ppdrfo_c1e6s0s0.7p8y9"6, line 71127 in 15s1t/a_rtte
s  File t"_/enUvs_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlf_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpehloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hploalcde_hpolladc_ephloalcde_hpolladc_ephloalcdeh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpehloalcde_hpolladc_ephloalcde_hpolladc_ephloalcdeh_opllda_cpelhaocledh_opll/dl_ipbl/apcyethhoolnd_3p.l7a/cseihtoel-dp_apclkaacgeehso/l_dp_yptleascte/hfoilxdt_uprleasc.ephyo"ld_pl, line /683l in ib/p_yctohmopnu3t.e7_/fsiixttuer-ep_avcaklaugee
s  File /"v/aUlsiedrpsh/ymsi/cphsaeeuld/ocdoantdaa./pcyo"n, line d141a in -fbiltdt/endn_ppdsfe_u1d6o0d0a7t8a9_6i7n7t1e5r1n/a_lt
est_en  File v_placeho"l/dU_spelrasc/emhioclhda_epll/caocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e60h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaoclde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpollda_cpelhaocledh_opllda_cephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-opladc_kpalgaceesh/o_lpdy_tpelsatc/efhioxltdu_rpelsa.cpeyh"old, line _601p in l_agceeth_oalcdt_ipvle/_lfiibx/tpuyrtehdone3f.
7/site-pa  File c"k/aUgseesr/sr/empiocrhtaeenlg/icnoen/drae/scoounrdcae-bbulidl/dnenrp.dpfy_"160, line 01757 in 896771g5e1t/__rteessutl_te
nv_p  File l"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaa/cceohnodlad-_bpllda/cnenhpodlfd__1p6l0a0c7e8h9o6l7d7_1p5l1a/c_etheosltd__epnlva_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_polladc_ephloalcde_hpolladc_ephloalcde_hpolladc_eplhaocledh_opllda_cpelhaoclde_hpolladc_ephloalcde_hpollda_cpelhaocledh_opllda_cpleahcoelhdo_lplda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cseihtoel-dp_apclkaacgeehso/l_d_ppyltaecseth/oflidx_tpurleasc.ephyo"ld_p, line l581a in cgeehtoflidx_tpulr/elviabl/upey
thon3  File ."7//Ussietres-/pmaicckhaageels//croenpdoar/tceonngdian-eb/lrde/snonuprdcfe_bu1i6l0d0e7r8.9p6y7"715, line 1/168_ in tesetx_eecnuvt_ep_sleaqcueehnotlida_lp
laceho  File ld_plac"e/hoUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_oplld_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_acehold_placehold_placehocledh_opllda_cpelhaocledh_opllda_cpelhaoldc_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py", line 37 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py", line 43 in closure
  File "/Users/meihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7a7c1e5h1o/l_dt_epslta_ceenhvo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_mpullatciephroolcde_spsliancge/hporlodc_epslsa.cpeyh"o, line l112d in _sptlaarcte
ho  File l"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d_/p_ltaecseth_oelndv__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppll/alciebh/oplydt_hpolna3c.e7h/osiltde_-pplaacckeahgoelsd/_vpallaicdephhoylsd_/ptleasctesh/otleds_tp_lpasceeuhdooldda_tpal.apcy"ehol, line d57_ in plasceethoulpd__dpilcatcse
hold_  File placehol"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpoln/dlai-bb/ldp/yntnhpodnf3_.176/0s0i7t8e9-6p7a7c1k5a1g/e_st/evsta_leindvp_hpylsa/cpesheuodlodd_aptlaa.cpeyh"o, line l141d in _fpiltatceedh_oplsde_puldaocdeahtoal_di_nptleacrenhaoll
d_placeho  File ld_placeho"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_89p6l7a7c1e5h1o/l_dt_epslta_ecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_opldl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apclehaocledh_opllda_cpelhaocledh_opllda_cpelh/ollidb_/pplyatcheohno3l.d7_/psliatcee-hpoalcdk_apgleasc/e_hpoyltde_sptl/afciexhtoulrde_sp.lpayc"ehol, line d924_ in placcaelhlo_lfdi_xptluarcee_hfoulndc_
placeh  File ol"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9/6li7b7/1p5y1t/h_otne3s.t7_/esnivt_ep-lpaaccekhaogleds_/prleapcoerhtoelndg_ipnle/arceeshooulrdc_epbluaicledheorl.dp_yp"lac, line e175ho in ld_plgaecte_hroelds_upllta
ceho  File l"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hboldl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelascte_heonlvd__ppllaacceehhoolldd__ppllaacceehhoolld_dp_lpalcaecheohlodl_dp_lplaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpd_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/a_cpeyhtoelsdt_/pfliaxcteuhroelsd._ppyl"ace, line h1117o in ld_pplyatceeshto_lfdi_xptluarcee_hsoeltdu_pp
lace  File hold_pl"a/cUesehrosl/dm_ipclhaaceelh/oclodn_dpal/accoenhdoal-db_lpdl/ancnephdofl_d1_6p0l0/7l8i9b6/7p7y1t5h1o/n_3.t7e/ssti_teen-vp_apclkacaegheosl/dr_epploarcteheonlgdi_npel/arceeshooulrdc_epbluaicledhero.lpdy_"pla, line c168e in holedx_epcluatec_esheoqlude_nptliaacleh
old_p  File laceho"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpdy_tphloanc3e.h7o/lsdi_tpel-apcaechkoagleds_/ppllaucgeghyo/lcda_lpllearcse.hpoyl"d_pl, line 187a in ceho_lmdu_lptliaccaellh
old_  File plac"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1a5c1e/h_otleds_tp_leancve_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-opladc_kpalgaecse/hroelpdo_rptleancgeihnoel/da_ppil.apcye"hol, line d37_ in p_l_accaelhlo_l_d
_pl  File ace"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelh/ollidb_/pplyatcheohno3l.d7_/psliatcee-hpoalcdk_apgleasc/ephloulgdg_yp/lmaacneahgoelrd._ppyl"ace, line h87o in ld<l_apmlbadcae>h
o  File ld"_/pUlsaecresh/omlidc_hpalealc/echoonldda_/pcloancdeah-oblldd_/npnlpadcfe_h1o6l0d0_7pl8a9c6e7h7o1l5d1_/p_ltaecseth_oelndv__ppllaacceehhoolldd__ppll/alciebh/oplydt_hpolna3c.e7h/oslidt_ep-lpaaccekhaogleds_p/lraecpeohrotledn_gpilnaec/eahpoil.dp_yp"l, line a43c in ehold_cplloascuerheo
ld_pla  File cehold_p"l/aUcseehrosl/dm_ipclhaaceelh/oclodn_dpal/accoenhdoal-db_lpdl/ancnephdofl_d1_6p0l0a7c8e9h6o7l7d1_5pl1a/c_etheosltd__epnlva_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cseihtoel-dp_apclkaacgeehso/lpdl_upglgayc/emhaonladg_eprl.apcye"ho, line l93d in _plac_ehhooolkde_xpelca
c  File ehol"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptleascte_heonlv_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/avcaelhiodlpdh_ypsl/atceeshtosl/dt_epslta_cephsoeludd_opdlaatcae.hpoyl"d_pl, line a57c in ehosledt_uppl_adciechtosl
d_pla  File cehold_p"l/aUcseehrosl/dm_ipclhaaceelh/oclodn_dpal/accoenhdoal-db_lpdl/ancnephdofl_d1_6p0l0a7c8e9h6o7l7d1_5p1l/a_cteehsotl_de_npvl_apcleahcoelhdo_lpd_lp/llaicbe/hpoyltdh_polna3c.e7h/oslidt_ep-lpaaccekhaogleds_/ppllaucgeghyo/lhdo_opklsa.cpeyh"o, line l286d in __p_lcaaclelh_o_l
d_pla  File c"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1a5c1e/h_otleds_tp_leancve_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-opladc_kpalgaecse/h_oplydt_epslta/cfeihxotludr_epsl.apcye"hol, line d924_ in placcaelhlo_lfdi_xptluarcee_hfoulndc_
placeho  File l"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptle/slti_be/npvy_tphloanc3e.h7o/lsdi_tpel-apcaechkoagleds_/p_lpaycteehsotld/_fpilxatcuerheosl.dp_yp"l, line a1062c in eehxoelcdu_tpel
a  File ceho"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohloldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpdy_tphloanc3e.h7o/lsdi_tpel-apcaechkoalgde_sp/l_apcyetheosltd/_fpilxatcuerheosl.dp_yp"lac, line e1117h in old_ppyltaecseth_oflidx_tpulraec_esheotludp_
place  File hold_p"l/aUcseehrosl/dm_ipclhaaceelh/oclodn_dpal/accoenhdoal-db_lpdl//nlnipbd/fp_y1t6h0o0n738.976/7s7it1e5-1p/a_ctkeasgte_se/n_vp_yptleascte/hfoilxdtu_rpelsa.cpeyh"old_p, line l683a in ceh_oclodm_ppultaec_efhioxltdu_rpel_avcaelhuoel
d  File _"p/lUasceerhso/lmdi_cphlaaecelh/oclodn_dpal/accoenhdoal-db_lpdl/ancnephdofl_d1_6p0l0a7c8e9h6o7l7d1_5p1l/a_cteehsotl_de_npvl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_aplcaecheohlodl_dp_lpalceahcoelhdo_lpdl_/plliabc/ephyotlhdo_np3l.a7c/eshiotled-_ppalcakcaegheosl/dp_lpulgagcye/hcoalldl_eprlsa.cpeyh"old, line _187p in lac_emhuolltdi_cpallalc
ehol  File d_p"l/aUcseehrosl/dm_ipclhaaceelh/oclodn_dpal/accoenhdoal-db_lpdl/ancnephdofl_d1_6p0l0/7l8i9b6/7p7y1t5h1o/n_3.t7e/ssti_teen-vp_apclkaacgeehso/l_dp_yptleascte/hfoilxdt_uprleasc.ephyo"ld_, line p601l in a_cgeehto_ladc_tpilvaec_efhioxltdu_rpeldaecfe
hold_p  File l"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaa/cceohnodlad-b_lpdl/ancnephdofl_d1_6p0l0a7c8e9h6o7l7d1_5p1l/a_cteehsotl_de_npvla_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cseihtoel-dp_apclkaacgeehso/lpdl_upglgayc/emhaonladg_eprl.apcye"hol, line d87_ in pl<alcaemhbodlad>
_  File p"l/aUcseehrosl/dm_pilcahcaeehlo/lcdo_npldaac/echoonldda_-pblladc/enhnoplddf__p1l6a0c0e7h8o9l6d7_7p1l5a1c/e_htoelsdt__peln/vl_ipbl/apcyethhoolnd3_.p7/lsaicteeh-opladc_kpalgaecse/h_oplydt_epsltac/efhioxltdu_rpelsa.cephyo"ld_p, line l581a in cgeehtoflidx_tpulraecveahlouled
_pla  File c"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_f_p1l6a0c0e7h8o9l6d7_7p1l5a1c/e_htoelsdt__pelnavc_ephloalcedh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cseihtoel-dp_apclkaacgeehso/lpdl_upglgayc/emhaonladg_eprl.apcye"h, line o93l in d_p_lhaocoekheoxledc_
p  File lac"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1a5c1e/h_otleds_tp_leancve_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-olpda_cpklaagceesh/o_lpdy_tpelsatc/efhioxltdu_rpelsa.cpeyh"o, line l568d in _pl_afcielhloflidx_tpulraecse
hold_  File placehol"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptleascte_heonlvd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpdy_tphloanc3e.h7o/lsdi_tpel-apcaechkoalgde_sp/lpalcueghgoyl/dh_opolkasc.ephyo"l, line d286_ in p_l_accaelhlo_l_d
_plac  File e"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelh/ollidb_/pplyatcheohno3l.d7_/psliatcee-hpoalcdk_apgleasc/e_hpoyltde_sptl/apcyethhoolnd._ppyl"aceho, line ld1576_ in placseehtoulpd
_  File plac"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1a5c1e/h_otleds_tp_leancve_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-opladc_kpalgaecse/h_oplydt_epslta/cfeihxotludr_epsl.apcye"h, line o1062l in de_xpelcaucteeh
o  File ld_"p/lUasceerhso/lmdi_cphlaaecle/hcoolndd_ap/lcaocnedhao-lbdl_dp/lnancpedhfo_l1d6_0p0l7a8c9e6h7o7l1d5_1p/l_atceeshto_ledn_vp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lp/llaicbe/hpoyltdh_opnl3a.c7e/hsoiltde_-pplacakcaegheosl/d__ppyltaecseth/orludn_npelra.cpeyh"old_, line p432l in apcreehpoalrde_
place  File hold"_/pUlsaecresh/omlidc_hpalealc/echoonldda_/pcloancdeah-oblldd_/pnlnapcdefh_o1l6d0_0p7l8a9c6e7h7o1l5d1_/p_ltaecseth_oelndv__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppll/alciebh/oplydt_hpolna3c.e7h/oslidt_ep-lpaaccekhaogleds_/p_lpaycteehsotl/df_ipxltaucreehso.lpdy_"plac, line e683h in old__cpolmapcuetheo_lfdi_xptluarcee_hvoalldu_ep
l  File a"c/eUhsoelrds_/pmliaccheaheoll/dc_opnldaac/echoonldda_-pblladc/enhnoplddf__p1l6a0c0e7h8o9l6d7_7p1l5a1c/e_htoelsdt__pelnavc_ephloalcde_hpolladc_ephloalcde_hpoll/dl_ipbl/apcyethhoolnd3_.p7l/ascietheo-lpda_cpklaagceesh/o_lpdy_tpelsatc/erhuonlnde_rp.lpayc"eho, line l141d in _ppyltaecseth_orludn_tpelsatc_esheotludp_
plac  File e"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelh/ollidb_/pplyatcheohno3l.d7_/psliatcee-hpoalcdk_apgleasc/e_hpoyltde_sptl/afciexhtoulrde_sp.lpayc"eho, line l601d in __pgleatc_eahcotilvde__pfliaxcteuhroeldde_fp
laceh  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/apcleuhgoglyd/_cpallalceerhso.lpdy_"p, line l187a in c_emhuolltdi_cpallalc
e  File h"o/ldU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelascte_heonlvd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppll/alciebh/oplydt_hpolna3c.e7h/oslidt_ep-lpaaccekhaogleds_/p_lpaycteehsotl/df_ipxltaucreehso.lpdy_"plac, line e581h in ogledt_fpilxatcuerheovladl_upel
aceh  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/apcleuhgoglyd/_mpalnaacgeehro.lpdy_"p, line l87a in c<elhaomlbdd_ap>l
aceho  File l"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptleascte_heonlvd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpdy_tphloanc3e.h7o/lsdi_tpel-apcaechkoalgde_sp/l_apcyetheosltd/_fpilxatcuerheosl.dp_yp"l, line a568c in eho_lfdi_lpllfaicxethuorleds_
plac  File ehold_"p/lUasceerhso/lmdi_cphlaaecle/hcoolndd_ap/lcaocnedhao-lbdl_dp/lnancpehdofl_d1_6p0l0a7c8e9h6o7l7d1_5p1l//_ltiebs/tp_yetnhvo_np3l.a7c/eshiotled-_ppalcakcaegheosl/dp_lpulgagcye/hold_mplacehold_plaacneahgoelrd._ppyl"a, line c93e in h_ohlodo_kpelxaecce
h  File o"l/dU_spelrasc/emhioclhda_elp/colnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1a5c1e/h_otleds_tp_leancve_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-opladc_kpalgaecse/h_oplydt_epslta/cpeyhtohlodn_.pplya"cehold_p, line l1576a in ceholsde_tpulpa
c  File eho"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpels/tl_iebn/vp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/apcleuhgoglyd/_hpoloakcse.hpoyl"d, line _286p in l_a_cceahloll_d__
placeh  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/a_cpeyhtoelsdt_/prluancneehro.lpdy_"place, line h432o in lpdr_epplaarcee
hold_pl  File ace"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7/1l5i1b//_ptyetshto_ne3n.v7_/psliatcee-hpoalcdk_apgleasc/e_hpoyltde_sptl/arcuenhnoelrd._ppyl"a, line c247e in h<olladm_bpdlaa>c
ehol  File d"_/pUlsaecresh/omlidc_hpalealc/echoonldda_/pcloancdeah-oblldd_/pnlnapcdefh_o1l6d0_0p7l8a9c6e7h7o1l5d1_/p_ltaecseth_oelndv__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppll/alciebh/oplydt_hpolna3c.e7h/oslidt_ep-lpaaccekhaogleds_/p_lpaycteehsotl/dr_upnlnaecre.hpoyl"d_pla, line c141e in hpoyltde_sptl_arcuenhtoelsdt__psleatcueph
old_p  File l"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaa/cceohnodlad-_bplld//lninbp/dpfy_t1h6o0n037.879/6s7i7t1e5-1p/a_ctkeasgte_se/n_vp_yptleascte/hroulndn_eprl.apcye"hold, line _294p in lfarcoemh_ocladl_lp
lace  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7a7c1e5h1o/l_dt_epslta_ceenhvo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_spiltaec-ephaoclkda_gpelsa/cpelhuoglgdy_/pclaalcleehrosl.dp_yp"l, line a187c in e_hmoulldt_ipclaalcle
ho  File l"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlp/dlfi_b1/6p0y0t7h8o9n637.771/5s1i/t_et-epsatc_keangve_sp/l_apcyetheosltd/_rpulnanceerh.oplyd"_, line p247l in acechaollld__rpulnatceeshto_lhdo_opkl
a  File c"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1a5c1e/h_otleds_tp_leancve_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-opladc_kpalgaecse/hpolludg_gpyl/amcaenhaogledr_.pplya"c, line e87h in o<llda_mpbldaac>e
hold_p  File l"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaa/cceohnodlad-_bpllda/cnenhpodlfd__1p6l0/0l7i8b9/6p7y7t1h5o1n/3_.t7e/ssti_teen-vp_apclkaacgeehso/l_dp_yptleascte/hroulndn_eprl.apcye"h, line o207l in dc_apllla_caenhdo_lrde_ppolratc
e  File hol"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptleascte_heonlvd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpdy_tphloanc3e.h7o/lsdi_tpel-apcaechkoalgde_sp/lpalcueghgoyl/dm_apnlaagceerh.oplyd"_, line p93l in a_cheohookledx_epcl
a  File c"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1/5l1i/b_/tpeystth_oenn3v._7p/lsaicteeh-opladc_kpalgaecse/h_oplydt_epslta/creuhnonledr_.pplya"c, line e111h in orludn_tpelsatcperhootlodc_opll
a  File ceh"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7a7c1e5h1o/l_dt_epslta_ceenhvo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_spiltaec-ephaoclkda_gpelsa/cpelhuoglgdy_/phloaockesh.oplyd"_, line p286l in a_c_echaollld___p
laceho  File l"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptle/slti_be/npvy_tphloanc3e.h7o/lsdi_tpel-apcaechkoalgde_sp/l_apcyetheosltd/_rpulnanceerh.oplyd"_, line p100l in apcyetheosltd__rpulnatceeshto_lpdr_optloaccoelh
o  File l"d/_Upslearcse/hmoilcdh_apella/cceohnodlad/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptleascte_heonlvd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpdy_tphloanc3e.h7o/lsdi_tpel-apcaechkoalgde_sp/l_apcyetheosltd/_rpulnanceerh.oplyd"_, line p247l in a<cleahmobldda_>p
laceh  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/d_spiltaec-ephaoclkda_gpelsa/cpelhuoglgdy_/pclaalcleehrosl.dp_yp"l, line a187c in e_hmoulldt_ipclaalcle
hold_plac  File e"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelh/ollidb_/pplyatcheohno3l.d7_/psliatcee-hpoalcdk_apgleasc/e_hpoyltde_sptl/arcuenhnoelrd._ppyl"acehol, line d294_ in pflraocme_hcoalldl_
placeh  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpestl_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 247 in call_runtest_hook
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeholadc_ephloalcde_hpoll/dl_ipbl/apcyethhoolnd3_.p7l/ascietheo-lpda_cpkla/gleisb//pplyutghgoyn/3m.a7n/asgietre.-ppya"c, line k87a in g<elsa/m_bpdyat>e
st/runn  File e"r/.Upsye"r, line s207/ in mciaclhla_ealn/dc_ornedpao/rcto
n  File da-b"l/dU/snenrpsd/fm_i1c6h0a0e7l8/9c6o7n7d1a5/1c/o_ntdeas-tb_ledn/vn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheolhdo_lpdl_apcleahceohlodl_dp_lpalcaehcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_spiltaec-ephaoclkda_gpels//lpilbu/gpgyyt/hmoann3a.g7e/rs.iptye"-, line p93a in c_khaogoekse/x_epcy
test/r  File u"n/nUesre.rpsy/"m, line i111c in hrauenlt/ecsotnpdrao/tcoocnodla
-  File bld"//nUnspedrfs_/1m6i0c0h7a8e9l6/7c7o1n5d1a//_ctoensdta_-ebnlvd_/pnlnapcdefh_o1l6d0_0p7l8a9c6e7h7o1l5d1_/p_ltaecseth_oelndv__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__plpalcaecheohlodl_dp_lpalcaecheohlodld__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppll/laicbe/hpoyltdh_opnl3a.c7e/hsoiltde_-pplaacckeahgoelsd/_ppllu/glgiyb//hpoyotkhso.np3y."7, line /286s in i_t_ec-aplalc_k_a
ges/  File _pyte"s/tU/sreurnsn/emri.cphya"e, line l100/ in cpoyntdeas/tc_ornudnat-ebsltd_/pnrnoptdofc_o1l6
0  File 0"7/8U9s6e7r7s1/5m1i/c_htaeeslt/_ceonnvd_ap/lcaocnedhao-lbdl_dp/lnancpedhfo_l1d6_0p0l7a8c9e6ho7l7d1_5p1l/a_cteehsotl_de_npvl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_polladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpoll/dl_ipbl/apcyethhoolnd3_.p7l/ascietheo-lpda_cpklaagceesh/o_lpdy_tpelsatc/emhaoilnd._ppyl"/lib/, line p321y in tphyotne3s.t7_/rsuinttee-sptalcokoapg
e  File s"//pUlsuegrgsy//mciaclhlaeerls/.cpoyn"d, line a187/ in c_omnudlat-ibcladl/ln
npdf_1  File 6"0/0U7s8e9r6s7/7m1i5c1h/a_etle/scto_nednav/_cpolnadcae-hbolldd/_npnlpadcfe_h1o6l0d0_7p8l9a6c7e7h1o5l1d/__ptleascte_heonlvd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__pplla/cleihbo/lpdy_tphloanc3e.h7o/lsdi_tpel-apcaeckages/pplluagcgeyh/ocladl_lpelrasc.ephyo"l, line d187_ in p_lmauclethiocladl_lp
l  File a"c/eUhsoelrds_/pmli/clhiabe/lp/yctohnodna3/.c7o/nsdiat-eb-lpda/cnknapgdefs_/1_6p0y0t7e8s9t6/7f7i1x5t1u/r_etse.spty_"e, line n568v in _place_fhiollldf_ipxltaucreehso
ld_pla  File cehold"_/pUlsaecresh/omlidc_hpalealc/echoonldda_/pcloancdeah-oblldd_/pnlnapcdefh_o1l6d0_0p7l8a9c6e7h7o1l5d1_/p_ltaecseth_oelndv__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lp/llaicbe/hpoyltdh_opnl3a.c7e/hsoiltde_-pplaacckeahgoelsd/_pplluagcgeyh/omladn_apglearc.ephyo"l, line d87_ in p<llaacmebhdoal>d
_plac  File e"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-/bllidb//npnyptdhfo_n136.070/7s8i9t6e7-7p1a5c1k/a_gteess/t__peyntve_sptl/apcyethhoolnd._ppyl"ace, line h1576o in ld_psleatcueph
o  File ld_"p/lUasceerhso/lmdi_cphlaaecle/hcoolndd_ap/lcaocnedhao-lbdl_dp/lnancpedhfo_l1d6_0p0l7a8c9e6h7o7l1d5_1p/l_atceeshto_ledn_vp_lpalcaecehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppllaacceehhoolldd__ppll/alciebh/oplydt_hpolna3c.e7h/oslidt_ep-lpaaccekhaogleds_/ppllaucgeghyo/lmda_npalgaecre.hpoyl"d, line _93p in l_ahcoeohkoelxde_cp
lace  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdla/-lbilbd//pnyntphdofn_31.670/0s7i8te9-6p7a7c1k5a1g/e_st/e_spty_teensvt_/prluancneehro.lpdy_"pla, line c432e in hporledp_aprlea
cehold  File _pl"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaa/cceohnodlad-_bpllda/cnenhpodlfd__1p6l00a7c8e9h6o7l7d1_5p1l/a_cteehsotl_de_npvl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_/plliabc/ephyotlhdo_np3l.a7c/eshiotled-_ppalcakcaegheosl/dp_lpulgagcye/hhoolodk_sp.lpayc"e, line h286o in l_d__cpallalc_e_h
old_p  File l"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaac/echoonldda_-pbll/dl/inbn/ppdyft_h1o6n030.778/9s6i7t7e1-5p1a/c_ktaegsets_/e_npvy_tpelsatc/erhuonlnde_rp.lpayc"eho, line l141d in _ppyltaecseth_orludn_tpelsatc_esheotludp_
place  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7a7c1e5h1o/l_dt_epslta_ceenhvo_lpdl_apcleahcoelhdo_lpdl_acpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelh/ollidb_/pplyathcoenh3o.l7d/_spilatcee-hpoalcdk_apgleasc/e_hpoyltde_sptl/amcaeihno.lpdy_"pl, line a296c in e_hmoalidn_
place  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7/7l1i5b1//p_yttehsotn_3e.n7v/_spiltaec-ephaoclkda_gpelsa/cpelhuoglgdy_/pclaalcleehrosl.dp_yp"l, line a187c in e_hmoulldt_ipclaalcle
h  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/a_cpeyhtoelsdt_/pmlaaicne.hpoyl"d, line _240p in lwarcaeph_osleds_spiloanc
ehold  File _"p/lUasceerhso/lmdi_cphlaaecle/hcoolndd_ap/lcaocnedhao-lbdl_dp/lnancpedhfo_l1d6_0p0l7/8l9i6b7/7p1y5t1h/o_nt3e.s7t/_seintve_-pplaacckeahgoelsd/_pplluagcgeyh/omladn_apglearc.ephyo"l, line d87_ in p<llaacmebhdoal>d
_plac  File e"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_pcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_spiltaec-ephaoclkda_gpelsa/c_ephyotleds_t/pmlaaicne.hpoyl"d, line _289p in lacephyotleds_tp_lcamcdelhionled__mpalianc
ehol  File d"_/pUlsaecresh/omlidc_hpalealc/echoonldda_/pcloancdeah-oblldd_/pnln/pldifb_/1p6y0t0h7o8n936.777/1s5i1t/e_-tpeasctk_aegnevs_/ppllaucgeghyo/lmda_npalgaecre.hpoyl"d, line _93p in l_ahcoeohkoelxde_cp
l  File a"c/eUhsoelrds_/pmliaccheaheoll/dc_opnldaac/echoonldda_-pblladc/enhnoplddf__p1l6a0c0e7h8o9l6d7_7p1l5a1c/e_htoelsdt__pelnva_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_/plliabc/ephyotlhdo_np3l.a7c/eshiotled-_ppalcakcaegheosl/dp_lpulgagcye/hcoalldl_eprlsa.cpeyh"o, line l187d in __pmlualcteihcoalldl_
place  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdla/-lbilbd//pnyntphdofn_31.670/0s7i8t9e6-7p7a1c5k1a/g_etse/sptl_uegngvy_/phloaockesh.oplyd"_, line p286l in a_c_echaollld___p
lace  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/clodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelh/ollidb_/pplyatcheohno3l.d7_/psliatcee-hpoalcdk_apgleasc/ephloulgdg_yp/lmaacneahgoelrd._ppyl"a, line c87e in h<olladm_bpdlaa>c
ehol  File d"_/pUlsaecresh/omlidc_hpalealc/echoonldda_/pclo/nldiab-/bplydt/hnonnp3d.f7_/1s6i0t0e7-8p9a6c7k7a1g5e1s//__tpeystte_setn/vr_upnlnaecre.hpoyl"d, line _247p in l<alcaemhbodlad>_
plac  File e"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7a1c5e1h/o_ltde_sptl_aecnevh_opllda_cpelhaocledh_opllda_cpelhaocledh_polladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-opladc_kpalgaecse/hpolludg_gpyl/amcaenhaogledr_.pplya"c, line e93h in o_lhdo_opkleaxceech
old_  File p"l/aUcseehrosl/dm_ipclhaaceelh/oclodn_dpal//cloinbd/ap-ybtlhdo/nn3n.p7d/fs_i1t6e0-0p7a8c9k6a7g7e1s5/1_/p_ytteesstt_/ernuvn_npelra.cpeyh"old, line _294p in lfarcoemh_ocladl_lp
lace  File h"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlfa_c1e6h0o0l7d8_9p6l7a7c1e5h1o/l_dt_epslta_ceenhvo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hloilbd/_ppyltahcoenh3o.l7d/_spiltaec-ephaoclkda_gpelsa/cpelhuoglgdy_/phloaockesh.oplyd"_, line p286l in a_c_echaollld___p
laceh  File old_place"h/oUlsde_rpsl/amciechhoaledl_/pclo/nldiab//cpoyntdhao-nb3l.d7//nsniptdef-_p1a6c0k0a7g8e9s6/7_7p1y5t1e/s_tt/ersutn_neenrv._ppyl"a, line c247e in holcda_lpll_arcuenhtoelsdt__phloaocke
h  File o"l/dU_spelrasc/emhioclhda_epll/accoenhdoal/dc_opnldaac-ebhlodl/dn_nppldafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalc/elhiobl/dp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_epsl/a_cpeyhtoelsdt_/pcloancfeihgo/l_d__ipnliatc_e_h.oplyd"_placeh, line o158l in dm_apilna
c  File ehold_pla"c/eUhsoelrds_/pmli/clhiabe/lp/yctohnodna3/.c7o/nsdiat-eb-lpda/cnknapgdefs_/1_6p0y0t7e8s9t6/7r7u1n5n1e/r_.tpeys"t, line _207e in ncva_lpll_aacnedh_orelpdo_rptl
a  File ceh"o/lUds_eprlsa/cmeihcohlade_lp/lcaocnedhao/lcdo_npdlaa-cbelhdo/lndn_ppdlafc_e1h6o0l0d7_8p9l6a7c7e1h5o1l/d__tpelsatc_eehnovl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_dp_lpalcaecheohlodl_pdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcle/hbionl/dp_yptleascte"hold, line _11p in l<amcoedhuolled>_
placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 111 in runtestprotocol
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 100 in pytest_runtest_protocol
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plahcoelhdo_lpdl_apcleahcoelhdo_lpdl_/plliabc/ephyotlhdo_np3l.a7c/eshiotled-_ppalcakcaegheosl/dp_lpulgagcye/hmoalnda_gperl./plyi"b, line /87p in y<tlhaomnb3d.a7>/
site-pa  File c"k/aUgseesr/sp/lmuigcgyh/aceall/lceornsd.ap/yc"o, line n187d in a_-mbulldt/incnapldlf
_16007  File 8"9/6U7s7e1r5s1//m_itcehsate_le/ncvo_npdlaa/cceohnodlad-_bpllda/cnenhpodlfd__1p6l0a0c7e8h9o6l7d7_1p5l1a/c_etheosltd__epnlva_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_placehold_placehold_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cseihtoel-dp_apclk/algiebs//ppyltuhgogny3/.m7a/nsaigteer-.ppayc"k, line a93g in e_sh/opolkuegxgeyc/
manage  File r"./pUys"e, line r87s in /<mliacmhbadeal>/
conda/c  File o"n/dUas-ebrlsd//mnincphdafe_l1/6c0o0n7d8a9/6c7o7n1d5a1-/b_ltde/sntn_pednfv__1p6l0a0c7e8h9o6l7d7_1p5l1a/c_etheosltd__epnlva_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cseihtoel-dp_apclk/algiebs//ppyltuhgogny3/.h7o/oskist.ep-yp"a, line c286k in a_g_ecsa/lpll_u_g
gy/man  File ager."p/yU"s, line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeeholrds_/pmliaccheaheoll/dc_opnldaac/echoonldda_-pblladc/enhnoplddf__1600789677placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/main.py", line 321 in pytest_runtestloop
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho1l51d/_test_env_placehold_placehold_place_hpolladc_ephloalcde_hpoll/dl_ipbl/apcyethhoolnd3_.p7l/ascietheo-lpda_cpklaagceesh/o_lpdy_tpelsatc/emhaoilnd._ppyl"ac, line e296h in ol_dm_apilna
cehold_  File p"l/aUcseehrosl/dm_ipclhaaceelh/oclodn_dpal/accoenhdoal-db_lpdl/ancnephdofl_d1_6p0l0a7c8e9h6o7l7d1_5p1l/a_cteehsotl_de_npvl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_apcleahcoelhdo_lpdl_/plliabc/ephyotlhdo_np3l.a7c/eshiotled-_ppalcakcaegheosl/d__ppyltaecseth/omladi_np.lpayc"ehold_p, line l321a in cpeyhtoelsdt__prluancteehsotlldo_oppl
a  File c"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1a5c1e/h_otleds_tp_leancve_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephlo/lldi_bp/lpayctehold_placehold_placehhoonl3d._7p/lsaicteeh-opladc_kpalgaecse/h_oplydt_epslta/cmeahionl.dp_yp"l, line a240c in ewhroalpd__spelsascieohno
ld_pla  File c"e/hUosledr_sp/lmaiccehhaoelld/_cpolnadcae/hcoolndd_ap-lbalcde/hnonlpdd_fp_l1a6c0e0h7o8l9d6_7p7l1a5c1e/h_otleds_tp_leancve_hpolladc_ephloalcde_hpolladc_ephlo/lldi_bp/lpayctehhoonl3d._7p/lsaicteeh-opladc_kpalgaecse/hpolludg_gpyl/accaelhloelrds_.pplya"c, line e187h in o_lmdu_lptliaccaelhlo
l  File d"_/pUlsaecresh/omlidc_hpalealc/echoonldda_/pclonda-blad/nnpdf_160078967c7e1h5o1l/d__tpelsatc_eenhv_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/liobl/dp_yptlhaocne3h.o7l/ds_iptlea-cpeahcokladg_es/ppllaucgeghyo/lhdo_opklsa.cpeyh"o, line l286d in _p_l_accaelhlo_l_d
_pl  File "/Users/michael/conda/conda-bld/nnapcdefh_o1l6d0_0p7l8a9c6e7h7o1l5d1_/p_lt/elsitb_/epnyvt_hpolna3c.e7h/oslidt_ep-lpaaccekhaogleds_/p_lpaycteehsotl/dm_apilna.cpeyh"o, line l289d in _placehpoyltde_sptl_accmedhloilnde__pmlaaicne
hold_p  File l"a/cUesheorlsd/_mpilcahcaeehlo/lcdo_npdlaa/cceohnodlad-_bpllda/cnenhpodlfd__1pl6a0c0e7h8o9l6d7_7p1l5a1c/e_htoelsdt__pelnavc_ephloalcde_hpolladc_ephloacehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/main.py", line 296 in _main
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/main.py", line 240 in wrap_session
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/main.py", line 289 in ld_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_ppyltaecseth_oclmd_placehold_placehold_placdelhionled__mpalianc
ehold_p  File la"c/eUhsoelrds_/pmilcahcaeehlo/lcdo_npdlaa/cceohnodlad-_bpllda/cnenhpodlfd__1p6l0a0c7e8h9o6l7d7_1p5l1a/c_etheosltd__epnlva_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cseihtoel-dp_apclkaacgeehso/lpdl_upglgayc/emhaonladg_eprl.apcye"h, line o93l in d__hpoloakceexheocl
d_plac  File e"h/oUlsde_rpsl/amciechhoaledl_/pcloancdeah/oclodn_dpal-abcledh/onlndp_dpfl_a1c6e0h0o7l8d9_6p7l7/1l5i1b//_ptyetshton_3e.n7v/_spiltaece-hpoalcdk_apgleasc/ephloulgdg_yp/lcaaclelheorlsd._ppyl"a, line c187e in h_omludl_tpilcaaclelh
old_pl  File a"c/eUhsoelrds_/pmliaccheaheoll/dc_opnldaac/echoonldda_-pblladc/enhnoplddf__p1l6a0c0e7h8o9l6d7_7p1l5a1c/e_htoelsdt__pelnavc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloalcde_hpolladc_ephloacledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_opllda_cpelhaocledh_oplld/_lpilba/cpeyhtohlodn_3p.l7a/cseihtoel-dp_apclkaacgeehso/lpdl_upglgayc/ehhooolkds_.pplya"c, line e286h in o_l_dc_apllla_c_e
hold_p  File lacehold_p"l/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/config/__init__.py", line 158 in main
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/bin/pytest", line 11 in <module>
acehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/config/__init__.py", line 158 in main
  File "/Users/michael/conda/conda-bld/nnpdf_1600789677151/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/bin/pytest", line 11 in <module>
{}
dict_keys([])
EE

==================================== ERRORS ====================================
______________________ ERROR at setup of test_pseudodata _______________________

    @pytest.fixture(scope="session", autouse=True)
    def setup_dicts():
        exp_infos_bytes = read_binary(validphys.tests.regressions, "test_exp_infos.pickle")
        ns = yaml.safe_load(EXAMPLE_RUNCARD)
        # This is what all the fitted replicas saw
        exp_infos = pickle.loads(exp_infos_bytes)

        # We now need to convert these to postfit replicas
        fitted_indices = API.fitted_replica_indexes(**ns)
        fit_postfit_mapping = dict(enumerate(exp_infos, 1))
        exp_infos = [fit_postfit_mapping[i] for i in fitted_indices]

>       pseudodata_info = API.get_pseudodata(**ns)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/tests/test_pseudodata.py:57: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py:43: in closure
    return self.__call__(name, **kwargs)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py:37: in __call__
    builder.execute_sequential()
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py:168: in execute_sequential
    perform_final=self.perform_final)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py:175: in get_result
    fres =  function(**kwdict)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py:158: in get_pseudodata
    return [fitted_pseudodata[i] for i in fitted_replica_indexes]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

.0 = <list_iterator object at 0x1a2c39b110>

>   return [fitted_pseudodata[i] for i in fitted_replica_indexes]
E   KeyError: 7

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py:158: KeyError
------------------------------ Captured log setup ------------------------------
INFO     validphys.loader:loader.py:905 Could not find a resource (pdf): pseudodata_test_fit. Attempting to download it.
INFO     validphys.loader:loader.py:585 Downloading https://data.nnpdf.science/fits/pseudodata_test_fit.tar.gz.
INFO     validphys.loader:loader.py:599 Extracting archive to $PREFIX/share/NNPDF/results/fit_download_deleteme_vcca_xc6
INFO     validphys.loader:loader.py:905 Could not find a resource (theoryID): Could not find theory 162. Folder '$PREFIX/share/NNPDF/data/theory_162' not found. Attempting to download it.
INFO     validphys.loader:loader.py:585 Downloading https://nnpdf.web.cern.ch/nnpdf/tables/theory_162.tgz.
INFO     validphys.loader:loader.py:599 Extracting archive to $PREFIX/share/NNPDF/data
WARNING  validphys.config:config.py:241 old fit found, falling back to old behaviour
WARNING  validphys.config:config.py:1229 `experiments` has been deprecated, specify data using `dataset_inputs`. Any grouping defined by `experiments` is being ignored.
INFO     validphys.loader:loader.py:905 Could not find a resource (pdf): NNPDF31_nnlo_as_0118. Attempting to download it.
INFO     validphys.loader:loader.py:585 Downloading https://lhapdf.hepforge.org/downloads/pdfsets/6.1/NNPDF31_nnlo_as_0118.tar.gz.
INFO     validphys.loader:loader.py:599 Extracting archive to $PREFIX/share/LHAPDF
ERROR    validphys.loader:loader.py:603 The original archive at $PREFIX/share/LHAPDF/tmpxa5ka2zqNNPDF31_nnlo_as_0118.tar.gz was only extracted partially at 
$PREFIX/share/LHAPDF
ERROR    validphys.loader:loader.py:817 $PREFIX/share/LHAPDF/tmpxa5ka2zqNNPDF31_nnlo_as_0118.tar.gz is not a compressed or uncompressed tar file. It seems the LHAPDF URLs aren't behaving, attempting to find resource in other repositories
INFO     validphys.loader:loader.py:585 Downloading https://data.nnpdf.science/pdfs/NNPDF31_nnlo_as_0118.tar.gz.
INFO     validphys.loader:loader.py:599 Extracting archive to $PREFIX/share/LHAPDF
WARNING  validphys.pseudodata:pseudodata.py:119 Using all 4 cores available, this may be dangerous especially for use on a cluster. Consider setting the NPROC variable to something sensible.
_________________ ERROR at setup of test_pseudodata_generator __________________

    @pytest.fixture(scope="session", autouse=True)
    def setup_dicts():
        exp_infos_bytes = read_binary(validphys.tests.regressions, "test_exp_infos.pickle")
        ns = yaml.safe_load(EXAMPLE_RUNCARD)
        # This is what all the fitted replicas saw
        exp_infos = pickle.loads(exp_infos_bytes)

        # We now need to convert these to postfit replicas
        fitted_indices = API.fitted_replica_indexes(**ns)
        fit_postfit_mapping = dict(enumerate(exp_infos, 1))
        exp_infos = [fit_postfit_mapping[i] for i in fitted_indices]

>       pseudodata_info = API.get_pseudodata(**ns)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/tests/test_pseudodata.py:57: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py:43: in closure
    return self.__call__(name, **kwargs)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py:37: in __call__
    builder.execute_sequential()
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py:168: in execute_sequential
    perform_final=self.perform_final)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py:175: in get_result
    fres =  function(**kwdict)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py:158: in get_pseudodata
    return [fitted_pseudodata[i] for i in fitted_replica_indexes]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

.0 = <list_iterator object at 0x1a2c39b110>

>   return [fitted_pseudodata[i] for i in fitted_replica_indexes]
E   KeyError: 7

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py:158: KeyError
=============================== warnings summary ===============================
$PREFIX/lib/python3.7/site-packages/validphys/tests/test_closuretest.py:12
  $PREFIX/lib/python3.7/site-packages/validphys/tests/test_closuretest.py:12: PytestCollectionWarning: cannot collect test class 'TestResult' because it has a __init__ constructor (from: tests/test_closuretest.py)
    class TestResult:

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
ERROR tests/test_pseudodata.py::test_pseudodata - KeyError: 7
ERROR tests/test_pseudodata.py::test_pseudodata_generator - KeyError: 7
=========== 52 deselected, 1 warning, 2 errors in 219.46s (0:03:39) ============
Thanks for using LHAPDF 6.2.3. Please make sure to cite the paper:
  Eur.Phys.J. C75 (2015) 3, 132  (http://arxiv.org/abs/1412.7420)
Tests failed for nnpdf-3.5.1031+gd735f163-py37h6c726b0_0.tar.bz2 - moving package to /Users/michael/conda/conda-bld/broken
WARNING:conda_build.build:Tests failed for nnpdf-3.5.1031+gd735f163-py37h6c726b0_0.tar.bz2 - moving package to /Users/michael/conda/conda-bld/broken
WARNING conda_build.build:tests_failed(2230): Tests failed for nnpdf-3.5.1031+gd735f163-py37h6c726b0_0.tar.bz2 - moving package to /Users/michael/conda/conda-bld/broken
TESTS FAILED: nnpdf-3.5.1031+gd735f163-py37h6c726b0_0.tar.bz2
wilsonmr commented 3 years ago

@Zaharid @siranipour could it be because we have an RandomGenerator.InitRNG(0, mcseed) inside of the thread? I notice a comment on #880 which mentions this isn't thread safe?

siranipour commented 3 years ago

Perhaps we can set the NPROC variable to 1 to see if that fixes the problem. This isn't truly sequential as it would still use threads.

old_pCurrent thread 0xlFatal Python error: acSegmentation fault000000011764e5c0e

@Zaharid @siranipour could it be because we have an RandomGenerator.InitRNG(0, mcseed) inside of the thread? I notice a comment on #920 which mentions this isn't thread safe?

Possibly, however, in n3fit.io.reader the monte carlo seed is read in explicitly for each replica. I think in this sense it would be thread safe. I'd be more worried if the RNG was initalized and then MakeReplica was performed on different experiments parallelly, for the same replica.

wilsonmr commented 3 years ago

Yeah but it calls RandomGenerator.InitRNG and RandomGenerator has a global state, right?

siranipour commented 3 years ago

Oh wait, I used multiprocessing for this code, not threading. The RNG is surely process safe right? There is no shared memory there

wilsonmr commented 3 years ago

I have basically no idea how this works, it actually seems like it might be segfaulting at the point of loading the data anyway. I switched on NPROC: 1 and get:

pytest --pyargs --mpl validphys -k test_pseudodata -s
============================= test session starts ==============================
platform darwin -- Python 3.7.9, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
Matplotlib: 3.3.1
Freetype: 2.10.2
rootdir: $SRC_DIR
plugins: mpl-0.11, hypothesis-5.35.4
collected 54 items / 52 deselected / 2 selected

[INFO]: Could not find a resource (pdf): pseudodata_test_fit. Attempting to download it.
[INFO]: Downloading https://data.nnpdf.science/fits/pseudodata_test_fit.tar.gz.
[INFO]: Extracting archive to /Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/NNPDF/results/fit_download_deleteme_p0da5tzz
[INFO]: Could not find a resource (theoryID): Could not find theory 162. Folder '/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/NNPDF/data/theory_162' not found. Attempting to download it.
[INFO]: Downloading https://nnpdf.web.cern.ch/nnpdf/tables/theory_162.tgz.
[INFO]: Extracting archive to /Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/NNPDF/data
[WARNING]: old fit found, falling back to old behaviour
[WARNING]: `experiments` has been deprecated, specify data using `dataset_inputs`. Any grouping defined by `experiments` is being ignored.
[INFO]: Could not find a resource (pdf): NNPDF31_nnlo_as_0118. Attempting to download it.
[INFO]: Downloading https://lhapdf.hepforge.org/downloads/pdfsets/6.1/NNPDF31_nnlo_as_0118.tar.gz.
[INFO]: Extracting archive to /Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF
[ERROR]: The original archive at /Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF/tmpf4_3mzwhNNPDF31_nnlo_as_0118.tar.gz was only extracted partially at 
/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF
[ERROR]: /Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF/tmpf4_3mzwhNNPDF31_nnlo_as_0118.tar.gz is not a compressed or uncompressed tar file. It seems the LHAPDF URLs aren't behaving, attempting to find resource in other repositories
[INFO]: Downloading https://data.nnpdf.science/pdfs/NNPDF31_nnlo_as_0118.tar.gz.
[INFO]: Extracting archive to /Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/share/LHAPDF
tests/test_pseudodata.py PDF: NNPDF31_nnlo_as_0118  ErrorType: Monte Carlo t0 booked
LHAPDF 6.2.3 loading $PREFIX/share/LHAPDF/NNPDF31_nnlo_as_0118/NNPDF31_nnlo_as_0118_0000.dat
NNPDF31_nnlo_as_0118 PDF set, member #0, version 1; LHAPDF ID = 303600
NNPDF31_nnlo_as_0118 Initialised with 1 members and errorType replicas

-- Reading COMMONDATA for Dataset: NMCPD
nData: 260 nSys: 5
-- COMMONDATA Files for NMCPD successfully read.

_NMCPD______________________________________________________
NMCPD_D
260 Data Points 10 X points 4 active flavours
_NMCPD______________________________________________________
NMCPD_P
260 Data Points 10 X points 5 active flavours

-- Reading COMMONDATA for Dataset: NMC
nData: 292 nSys: 16
-- COMMONDATA Files for NMC successfully read.

_NMC________________________________________________________
NMC
292 Data Points 10 X points 9 active flavours
Fatal Python error: Segmentation fault

Current thread 0x000000010d5ee5c0 (most recent call first):
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/NNPDF/nnpdf.py", line 6201 in __init__
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/core.py", line 517 in load
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/n3fit/io/reader.py", line 223 in common_data_reader
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py", line 106 in task
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/process.py", line 99 in run
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/process.py", line 297 in _bootstrap
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/popen_fork.py", line 74 in _launch
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/popen_fork.py", line 20 in __init__
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/context.py", line 277 in _Popen
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/context.py", line 223 in _Popen
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/multiprocessing/process.py", line 112 in start
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py", line 141 in fitted_pseudodata_internal
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py", line 175 in get_result
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py", line 168 in execute_sequential
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py", line 37 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py", line 43 in closure
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/tests/test_pseudodata.py", line 59 in setup_dicts
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 924 in call_fixture_func
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 1117 in pytest_fixture_setup
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 1062 in execute
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 683 in _compute_fixture_value
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 601 in _get_active_fixturedef
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 581 in getfixturevalue
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/fixtures.py", line 568 in _fillfixtures
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/python.py", line 1576 in setup
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 432 in prepare
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 141 in pytest_runtest_setup
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 247 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 294 in from_call
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 247 in call_runtest_hook
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 207 in call_and_report
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 111 in runtestprotocol
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/runner.py", line 100 in pytest_runtest_protocol
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/main.py", line 321 in pytest_runtestloop
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/main.py", line 296 in _main
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/main.py", line 240 in wrap_session
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/main.py", line 289 in pytest_cmdline_main
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/_pytest/config/__init__.py", line 158 in main
  File "/Users/michael/conda/conda-bld/nnpdf_1600791212535/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/bin/pytest", line 11 in <module>
{}
dict_keys([])
EE

==================================== ERRORS ====================================
______________________ ERROR at setup of test_pseudodata _______________________

    @pytest.fixture(scope="session", autouse=True)
    def setup_dicts():
        exp_infos_bytes = read_binary(validphys.tests.regressions, "test_exp_infos.pickle")
        ns = yaml.safe_load(EXAMPLE_RUNCARD)
        # This is what all the fitted replicas saw
        exp_infos = pickle.loads(exp_infos_bytes)

        # We now need to convert these to postfit replicas
        fitted_indices = API.fitted_replica_indexes(**ns)
        fit_postfit_mapping = dict(enumerate(exp_infos, 1))
        exp_infos = [fit_postfit_mapping[i] for i in fitted_indices]

>       pseudodata_info = API.get_pseudodata(**ns)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/tests/test_pseudodata.py:59: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py:43: in closure
    return self.__call__(name, **kwargs)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py:37: in __call__
    builder.execute_sequential()
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py:168: in execute_sequential
    perform_final=self.perform_final)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py:175: in get_result
    fres =  function(**kwdict)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py:158: in get_pseudodata
    return [fitted_pseudodata[i] for i in fitted_replica_indexes]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

.0 = <list_iterator object at 0x1a2d60b9d0>

>   return [fitted_pseudodata[i] for i in fitted_replica_indexes]
E   KeyError: 7

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py:158: KeyError
------------------------------ Captured log setup ------------------------------
INFO     validphys.loader:loader.py:905 Could not find a resource (pdf): pseudodata_test_fit. Attempting to download it.
INFO     validphys.loader:loader.py:585 Downloading https://data.nnpdf.science/fits/pseudodata_test_fit.tar.gz.
INFO     validphys.loader:loader.py:599 Extracting archive to $PREFIX/share/NNPDF/results/fit_download_deleteme_p0da5tzz
INFO     validphys.loader:loader.py:905 Could not find a resource (theoryID): Could not find theory 162. Folder '$PREFIX/share/NNPDF/data/theory_162' not found. Attempting to download it.
INFO     validphys.loader:loader.py:585 Downloading https://nnpdf.web.cern.ch/nnpdf/tables/theory_162.tgz.
INFO     validphys.loader:loader.py:599 Extracting archive to $PREFIX/share/NNPDF/data
WARNING  validphys.config:config.py:241 old fit found, falling back to old behaviour
WARNING  validphys.config:config.py:1229 `experiments` has been deprecated, specify data using `dataset_inputs`. Any grouping defined by `experiments` is being ignored.
INFO     validphys.loader:loader.py:905 Could not find a resource (pdf): NNPDF31_nnlo_as_0118. Attempting to download it.
INFO     validphys.loader:loader.py:585 Downloading https://lhapdf.hepforge.org/downloads/pdfsets/6.1/NNPDF31_nnlo_as_0118.tar.gz.
INFO     validphys.loader:loader.py:599 Extracting archive to $PREFIX/share/LHAPDF
ERROR    validphys.loader:loader.py:603 The original archive at $PREFIX/share/LHAPDF/tmpf4_3mzwhNNPDF31_nnlo_as_0118.tar.gz was only extracted partially at 
$PREFIX/share/LHAPDF
ERROR    validphys.loader:loader.py:817 $PREFIX/share/LHAPDF/tmpf4_3mzwhNNPDF31_nnlo_as_0118.tar.gz is not a compressed or uncompressed tar file. It seems the LHAPDF URLs aren't behaving, attempting to find resource in other repositories
INFO     validphys.loader:loader.py:585 Downloading https://data.nnpdf.science/pdfs/NNPDF31_nnlo_as_0118.tar.gz.
INFO     validphys.loader:loader.py:599 Extracting archive to $PREFIX/share/LHAPDF
_________________ ERROR at setup of test_pseudodata_generator __________________

    @pytest.fixture(scope="session", autouse=True)
    def setup_dicts():
        exp_infos_bytes = read_binary(validphys.tests.regressions, "test_exp_infos.pickle")
        ns = yaml.safe_load(EXAMPLE_RUNCARD)
        # This is what all the fitted replicas saw
        exp_infos = pickle.loads(exp_infos_bytes)

        # We now need to convert these to postfit replicas
        fitted_indices = API.fitted_replica_indexes(**ns)
        fit_postfit_mapping = dict(enumerate(exp_infos, 1))
        exp_infos = [fit_postfit_mapping[i] for i in fitted_indices]

>       pseudodata_info = API.get_pseudodata(**ns)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/tests/test_pseudodata.py:59: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py:43: in closure
    return self.__call__(name, **kwargs)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/api.py:37: in __call__
    builder.execute_sequential()
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py:168: in execute_sequential
    perform_final=self.perform_final)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/reportengine/resourcebuilder.py:175: in get_result
    fres =  function(**kwdict)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py:158: in get_pseudodata
    return [fitted_pseudodata[i] for i in fitted_replica_indexes]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

.0 = <list_iterator object at 0x1a2d60b9d0>

>   return [fitted_pseudodata[i] for i in fitted_replica_indexes]
E   KeyError: 7

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.7/site-packages/validphys/pseudodata.py:158: KeyError
=============================== warnings summary ===============================
$PREFIX/lib/python3.7/site-packages/validphys/tests/test_closuretest.py:12
  $PREFIX/lib/python3.7/site-packages/validphys/tests/test_closuretest.py:12: PytestCollectionWarning: cannot collect test class 'TestResult' because it has a __init__ constructor (from: tests/test_closuretest.py)
    class TestResult:

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
ERROR tests/test_pseudodata.py::test_pseudodata - KeyError: 7
ERROR tests/test_pseudodata.py::test_pseudodata_generator - KeyError: 7
=========== 52 deselected, 1 warning, 2 errors in 219.07s (0:03:39) ============
Thanks for using LHAPDF 6.2.3. Please make sure to cite the paper:
  Eur.Phys.J. C75 (2015) 3, 132  (http://arxiv.org/abs/1412.7420)
Tests failed for nnpdf-3.5.1032+gd8c24e88-py37h6c726b0_0.tar.bz2 - moving package to /Users/michael/conda/conda-bld/broken
WARNING:conda_build.build:Tests failed for nnpdf-3.5.1032+gd8c24e88-py37h6c726b0_0.tar.bz2 - moving package to /Users/michael/conda/conda-bld/broken
WARNING conda_build.build:tests_failed(2230): Tests failed for nnpdf-3.5.1032+gd8c24e88-py37h6c726b0_0.tar.bz2 - moving package to /Users/michael/conda/conda-bld/broken
TESTS FAILED: nnpdf-3.5.1032+gd8c24e88-py37h6c726b0_0.tar.bz2
Zaharid commented 3 years ago

Could you try initializing multiprocessing with the spawn method?

https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

Zaharid commented 3 years ago

Btw, it should be possible to run something like 

$ pytest --pyargs validphys -k test_pseudodata

without having to build the entire package.

wilsonmr commented 3 years ago

Btw, it should be possible to run something like

$ pytest --pyargs validphys -k test_pseudodata

without having to build the entire package.

it is! But it only errors when teh tests run in conda build for me!

siranipour commented 3 years ago

Note that #866 is now failing for this reason too

wilsonmr commented 3 years ago

if this spawn thing doesn't work.

My proposed fix would be to make a function which is truly sequential and call that if NPROC is 1 and then force tests to have NPROC at 1. But we can talk about it at the code call

wilsonmr commented 3 years ago

hmm using spawn method threw a new error

=========================================================================================== test session starts ============================================================================================
platform darwin -- Python 3.7.9, pytest-6.0.2, py-1.9.0, pluggy-0.13.1
rootdir: /Users/michael
plugins: hypothesis-5.35.3
collected 54 items / 52 deselected / 2 selected                                                                                                                                                            

nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_pseudodata.py EE                                                                                                                                  [100%]

================================================================================================== ERRORS ==================================================================================================
____________________________________________________________________________________ ERROR at setup of test_pseudodata _____________________________________________________________________________________

    @pytest.fixture(scope="session", autouse=True)
    def setup_dicts():
        exp_infos_bytes = read_binary(validphys.tests.regressions, "test_exp_infos.pickle")
        ns = yaml.safe_load(EXAMPLE_RUNCARD)
        # This is what all the fitted replicas saw
        exp_infos = pickle.loads(exp_infos_bytes)

        # We now need to convert these to postfit replicas
        fitted_indices = API.fitted_replica_indexes(**ns)
        fit_postfit_mapping = dict(enumerate(exp_infos, 1))
        exp_infos = [fit_postfit_mapping[i] for i in fitted_indices]

>       pseudodata_info = API.get_pseudodata(**ns)

nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_pseudodata.py:59: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
conda/envs/nnpdf/lib/python3.7/site-packages/reportengine/api.py:43: in closure
    return self.__call__(name, **kwargs)
conda/envs/nnpdf/lib/python3.7/site-packages/reportengine/api.py:37: in __call__
    builder.execute_sequential()
conda/envs/nnpdf/lib/python3.7/site-packages/reportengine/resourcebuilder.py:168: in execute_sequential
    perform_final=self.perform_final)
conda/envs/nnpdf/lib/python3.7/site-packages/reportengine/resourcebuilder.py:175: in get_result
    fres =  function(**kwdict)
nnpdfgit/nnpdf/validphys2/src/validphys/pseudodata.py:143: in fitted_pseudodata_internal
    p.start()
conda/envs/nnpdf/lib/python3.7/multiprocessing/process.py:112: in start
    self._popen = self._Popen(self)
conda/envs/nnpdf/lib/python3.7/multiprocessing/context.py:223: in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
conda/envs/nnpdf/lib/python3.7/multiprocessing/context.py:284: in _Popen
    return Popen(process_obj)
conda/envs/nnpdf/lib/python3.7/multiprocessing/popen_spawn_posix.py:32: in __init__
    super().__init__(process_obj)
conda/envs/nnpdf/lib/python3.7/multiprocessing/popen_fork.py:20: in __init__
    self._launch(process_obj)
conda/envs/nnpdf/lib/python3.7/multiprocessing/popen_spawn_posix.py:47: in _launch
    reduction.dump(process_obj, fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <Process(Process-2, initial)>, file = <_io.BytesIO object at 0x1a2dc12e30>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       TypeError: can't pickle SwigPyObject objects

conda/envs/nnpdf/lib/python3.7/multiprocessing/reduction.py:60: TypeError
------------------------------------------------------------------------------------------ Captured stdout setup -------------------------------------------------------------------------------------------
PDF: NNPDF31_nnlo_as_0118  ErrorType: Monte Carlo t0 booked
LHAPDF 6.2.3 loading /Users/michael/conda/envs/nnpdf/share/LHAPDF/NNPDF31_nnlo_as_0118/NNPDF31_nnlo_as_0118_0000.dat
NNPDF31_nnlo_as_0118 PDF set, member #0, version 1; LHAPDF ID = 303600
NNPDF31_nnlo_as_0118 Initialised with 1 members and errorType replicas
------------------------------------------------------------------------------------------ Captured stderr setup -------------------------------------------------------------------------------------------
[WARNING]: old fit found, falling back to old behaviour
[WARNING]: `experiments` has been deprecated, specify data using `dataset_inputs`. Any grouping defined by `experiments` is being ignored.
-------------------------------------------------------------------------------------------- Captured log setup --------------------------------------------------------------------------------------------
WARNING  validphys.config:config.py:241 old fit found, falling back to old behaviour
WARNING  validphys.config:config.py:1229 `experiments` has been deprecated, specify data using `dataset_inputs`. Any grouping defined by `experiments` is being ignored.
_______________________________________________________________________________ ERROR at setup of test_pseudodata_generator ________________________________________________________________________________

    @pytest.fixture(scope="session", autouse=True)
    def setup_dicts():
        exp_infos_bytes = read_binary(validphys.tests.regressions, "test_exp_infos.pickle")
        ns = yaml.safe_load(EXAMPLE_RUNCARD)
        # This is what all the fitted replicas saw
        exp_infos = pickle.loads(exp_infos_bytes)

        # We now need to convert these to postfit replicas
        fitted_indices = API.fitted_replica_indexes(**ns)
        fit_postfit_mapping = dict(enumerate(exp_infos, 1))
        exp_infos = [fit_postfit_mapping[i] for i in fitted_indices]

>       pseudodata_info = API.get_pseudodata(**ns)

nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_pseudodata.py:59: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
conda/envs/nnpdf/lib/python3.7/site-packages/reportengine/api.py:43: in closure
    return self.__call__(name, **kwargs)
conda/envs/nnpdf/lib/python3.7/site-packages/reportengine/api.py:37: in __call__
    builder.execute_sequential()
conda/envs/nnpdf/lib/python3.7/site-packages/reportengine/resourcebuilder.py:168: in execute_sequential
    perform_final=self.perform_final)
conda/envs/nnpdf/lib/python3.7/site-packages/reportengine/resourcebuilder.py:175: in get_result
    fres =  function(**kwdict)
nnpdfgit/nnpdf/validphys2/src/validphys/pseudodata.py:143: in fitted_pseudodata_internal
    p.start()
conda/envs/nnpdf/lib/python3.7/multiprocessing/process.py:112: in start
    self._popen = self._Popen(self)
conda/envs/nnpdf/lib/python3.7/multiprocessing/context.py:223: in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
conda/envs/nnpdf/lib/python3.7/multiprocessing/context.py:284: in _Popen
    return Popen(process_obj)
conda/envs/nnpdf/lib/python3.7/multiprocessing/popen_spawn_posix.py:32: in __init__
    super().__init__(process_obj)
conda/envs/nnpdf/lib/python3.7/multiprocessing/popen_fork.py:20: in __init__
    self._launch(process_obj)
conda/envs/nnpdf/lib/python3.7/multiprocessing/popen_spawn_posix.py:47: in _launch
    reduction.dump(process_obj, fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <Process(Process-2, initial)>, file = <_io.BytesIO object at 0x1a2dc12e30>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       TypeError: can't pickle SwigPyObject objects

conda/envs/nnpdf/lib/python3.7/multiprocessing/reduction.py:60: TypeError
============================================================================================= warnings summary =============================================================================================
nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_closuretest.py:12
  /Users/michael/nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_closuretest.py:12: PytestCollectionWarning: cannot collect test class 'TestResult' because it has a __init__ constructor (from: nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_closuretest.py)
    class TestResult:

nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_plots.py:11
  /Users/michael/nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_plots.py:11: PytestUnknownMarkWarning: Unknown pytest.mark.mpl_image_compare - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.mpl_image_compare

nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_plots.py:20
  /Users/michael/nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_plots.py:20: PytestUnknownMarkWarning: Unknown pytest.mark.mpl_image_compare - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.mpl_image_compare

nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_plots.py:41
  /Users/michael/nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_plots.py:41: PytestUnknownMarkWarning: Unknown pytest.mark.mpl_image_compare - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.mpl_image_compare

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================================= short test summary info ==========================================================================================
ERROR nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_pseudodata.py::test_pseudodata - TypeError: can't pickle SwigPyObject objects
ERROR nnpdfgit/nnpdf/validphys2/src/validphys/tests/test_pseudodata.py::test_pseudodata_generator - TypeError: can't pickle SwigPyObject objects
=============================================================================== 52 deselected, 4 warnings, 2 errors in 3.06s ===============================================================================
Thanks for using LHAPDF 6.2.3. Please make sure to cite the paper:
  Eur.Phys.J. C75 (2015) 3, 132  (http://arxiv.org/abs/1412.7420)
wilsonmr commented 3 years ago

@siranipour I think that actually having a truly sequential mode might be quite useful in the testing suite, even for linux! I think that ideally we would want them to produce the same numbers right?

In that case we can skip the parallel tests on mac and put appropriate exceptions letting people know it isn't supported until somebody is prepared to work out which objects aren't pickling and what to do about it.

wilsonmr commented 3 years ago

plus perhaps I'm being naive but I'm pretty sure that the difference between sequential mode and using mp is not that big (famous last words)

Zaharid commented 3 years ago

Just for the record, the way to do this properly may be to load the datasets in the initializer argument of the Pool.