Error running PE example combining models

[cmbiwer]

In this demo: https://pygwb.docs.ligo.org/pygwb/run_pe.html#Combining-models

When I run this cell:

gw_schu = type('gw_schu', (PowerLawModel,SchumannModel), dict(c='c'))
kwargs={"baselines":[HL, HV, LV], "model_name":'PL+Schu', "polarizations":['tensor'],"fref":25}
model = gw_schu(**kwargs)

priors= {'kappa_H': bilby.core.prior.LogUniform(0.01, 10, '$\\kappa_H$'),
            'kappa_L': bilby.core.prior.LogUniform(0.01, 10, '$\\kappa_L$'),
            'kappa_V': bilby.core.prior.LogUniform(0.01, 10, '$\\kappa_V$'),
            'beta_H': bilby.core.prior.Uniform(0.0, 12.0, '$\\beta_H$'),
            'beta_L': bilby.core.prior.Uniform(1.0, 10.0, '$\\beta_L$'),
           'beta_V': bilby.core.prior.Uniform(0.0, 7.0, '$\\beta_V$'),
            'omega_ref': bilby.core.prior.LogUniform(1e-13, 1e-5, '$\\Omega_{\\rm ref}$'),
             'alpha': bilby.core.prior.Gaussian(0, 3.5, '$\\alpha$')}
hlv_gw_schu=bilby.run_sampler(likelihood=model,priors=priors,sampler='dynesty', npoints=1000, walks=10,npool=10,outdir='./pe/',label= 'hlv_gw_schu', resume=False)

I get an error:

07:51 bilby INFO    : Running for label 'hlv_gw_schu', output will be saved to './pe/'
07:52 bilby INFO    : Analysis priors:
07:52 bilby INFO    : kappa_H=LogUniform(minimum=0.01, maximum=10, name='$\\kappa_H$', latex_label='$\\kappa_H$', unit=None, boundary=None)
07:52 bilby INFO    : kappa_L=LogUniform(minimum=0.01, maximum=10, name='$\\kappa_L$', latex_label='$\\kappa_L$', unit=None, boundary=None)
07:52 bilby INFO    : kappa_V=LogUniform(minimum=0.01, maximum=10, name='$\\kappa_V$', latex_label='$\\kappa_V$', unit=None, boundary=None)
07:52 bilby INFO    : beta_H=Uniform(minimum=0.0, maximum=12.0, name='$\\beta_H$', latex_label='$\\beta_H$', unit=None, boundary=None)
07:52 bilby INFO    : beta_L=Uniform(minimum=1.0, maximum=10.0, name='$\\beta_L$', latex_label='$\\beta_L$', unit=None, boundary=None)
07:52 bilby INFO    : beta_V=Uniform(minimum=0.0, maximum=7.0, name='$\\beta_V$', latex_label='$\\beta_V$', unit=None, boundary=None)
07:52 bilby INFO    : omega_ref=LogUniform(minimum=1e-13, maximum=1e-05, name='$\\Omega_{\\rm ref}$', latex_label='$\\Omega_{\\rm ref}$', unit=None, boundary=None)
07:52 bilby INFO    : alpha=Gaussian(mu=0, sigma=3.5, name='$\\alpha$', latex_label='$\\alpha$', unit=None, boundary=None)
07:52 bilby INFO    : Analysis likelihood class: <class '__main__.gw_schu'>
07:52 bilby INFO    : Analysis likelihood noise evidence: 68225.8137823414
07:52 bilby INFO    : Single likelihood evaluation took 5.967e-04 s
07:52 bilby INFO    : Using sampler Dynesty with kwargs {'nlive': 1000, 'bound': 'live', 'sample': 'act-walk', 'periodic': None, 'reflective': None, 'update_interval': 600, 'first_update': None, 'npdim': None, 'rstate': None, 'queue_size': 10, 'pool': None, 'use_pool': None, 'live_points': None, 'logl_args': None, 'logl_kwargs': None, 'ptform_args': None, 'ptform_kwargs': None, 'gradient': None, 'grad_args': None, 'grad_kwargs': None, 'compute_jac': False, 'enlarge': None, 'bootstrap': None, 'walks': 10, 'facc': 0.2, 'slices': None, 'fmove': 0.9, 'max_move': 100, 'update_func': None, 'ncdim': None, 'blob': False, 'save_history': False, 'history_filename': None, 'maxiter': None, 'maxcall': None, 'dlogz': 0.1, 'logl_max': inf, 'n_effective': None, 'add_live': True, 'print_progress': True, 'print_func': <bound method Dynesty._print_func of <bilby.core.sampler.dynesty.Dynesty object at 0x7fae10514730>>, 'save_bounds': False, 'checkpoint_file': None, 'checkpoint_every': 60, 'resume': False}
07:52 bilby INFO    : Checkpoint every check_point_delta_t = 600s
07:52 bilby INFO    : Using dynesty version 2.1.3
07:52 bilby INFO    : Using the bilby-implemented act-walk sampling tracking the autocorrelation function and thinning by 2 with maximum length 10000
07:52 bilby INFO    : Setting up multiproccesing pool with 10 processes
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/cmbiwer/anaconda3/envs/pygwb/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Users/cmbiwer/anaconda3/envs/pygwb/lib/python3.9/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'gw_schu' on <module '__main__' (built-in)>

[kevinturbang]

I ran the code on my end and things seem to work fine for me. It seems like this might be a multiprocessing issues. In the last line of the cell, could you try setting the npool parameter equal to 1, instead of 10 to see if that fixes things? Thank you.

[cmbiwer]

How do you recommend installing jupyter or whatever app to run the notebook? It seems some of these issues stem from me just installing the latest jupyter off conda.

[a-renzini]

How do you recommend installing jupyter or whatever app to run the notebook? It seems some of these issues stem from me just installing the latest jupyter off conda.

Hi @cmbiwer - I've now looked into this, and indeed, the issue arises when using a jupyter notebook whereas it runs fine on jupyter lab. I've reproduced your error by running:

conda create -n pygwb_for_joss python=3.9
git clone git@git.ligo.org:pygwb/pygwb.git
cd pygwb
pip install .
conda install jupyter
pip install ipykernel ipython==8.12.0 jupyter
ipython kernel install --name "pygwb_for_joss" --user

and using the pygwb_for_joss environment. Whereas, if I run using jupyter lab as we usually do, I get the same results as on the tutorial in the docs.

I think this is coming from a known bug/issue with multiprocessing on jupyter, see

• https://stackoverflow.com/questions/41385708/multiprocessing-example-giving-attributeerror
• https://bugs.python.org/issue25053

I have now fixed the issue by just removing the multiprocessing call in the notebook, npool=10. The example runs fast and doesn't actually need multiprocessing. I don't think it's worth having the example break when using notebooks, but we can add a note about it in the tutorials.

[cmbiwer]

Thanks for looking into that. That is what I was doing.

but we can add a note about it in the tutorials.

I think that would be helpful and I'd be happy to close the issue and approve the JOSS submission.

[a-renzini]

Great - I've added a tip, see https://git.ligo.org/pygwb/pygwb/-/blob/master/tutorials/run_pe.ipynb

All merged.