search

pymc-devs / pymc

Bayesian Modeling and Probabilistic Programming in Python
https://docs.pymc.io/
Other
8.69k stars 2.01k forks source link

BrokenPipeError: [Errno 32] Broken pipe #4985

Closed karanotsingyu closed 3 years ago

karanotsingyu commented 3 years ago

Hi, I’m a newbie of PyMC3 and Bayesian Analysis and currently learning them through Martin’s Bayesian Analysis with Python - First Edition.

When I tried to run tutorial codes in section 2.2.1 (Flipping coins the PyMC3 way), some error occurred:

import pymc3 as pm
import numpy as np
import scipy.stats as stats

np.random.seed(123)
n_experiments = 4
theta_real = 0.35
data = stats.bernoulli.rvs(p=theta_real, size=n_experiments)

with pm.Model() as our_first_model:
    theta = pm.Beta('theta', alpha=1, beta=1)
    y = pm.Bernoulli('y', p=theta, observed=data)
    start = pm.find_MAP()
    step = pm.Metropolis()
    trace = pm.sample(1000, step=step, start=start)
 |███████████████████████████| 100.00% [5/5 00:00<00:00 logp = -2.7726, ||grad|| = 1]]
/Users/maiyunfei/maiyunfei2000.com/code/just-coding/bayes/martin-2-2.py:47: FutureWarning: In v4.0, pm.sample will return an `arviz.InferenceData` object instead of a `MultiTrace` by default. You can pass return_inferencedata=True or return_inferencedata=False to be safe and silence this warning.
  trace = pm.sample(1000, step=step, start=start)
Multiprocess sampling (2 chains in 2 jobs)
Metropolis: [theta]
 |███████████████████████████| 100.00% [5/5 00:00<00:00 logp = -2.7726, ||grad|| = 1]]
/Users/maiyunfei/maiyunfei2000.com/code/just-coding/bayes/martin-2-2.py:47: FutureWarning: In v4.0, pm.sample will return an `arviz.InferenceData` object instead of a `MultiTrace` by default. You can pass return_inferencedata=True or return_inferencedata=False to be safe and silence this warning.
  trace = pm.sample(1000, step=step, start=start)
Multiprocess sampling (2 chains in 2 jobs)
Metropolis: [theta]
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/forkserver.py", line 274, in main
    code = _serve_one(child_r, fds,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/forkserver.py", line 313, in _serve_one
    code = spawn._main(child_r, parent_sentinel)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/maiyunfei/maiyunfei2000.com/code/just-coding/bayes/martin-2-2.py", line 47, in <module>
    trace = pm.sample(1000, step=step, start=start)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/sampling.py", line 559, in sample
    trace = _mp_sample(**sample_args, **parallel_args)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/sampling.py", line 1461, in _mp_sample
    sampler = ps.ParallelSampler(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/parallel_sampling.py", line 431, in __init__
    self._samplers = [
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/parallel_sampling.py", line 432, in <listcomp>
    ProcessAdapter(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/parallel_sampling.py", line 292, in __init__
    self._process.start()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 291, in _Popen
    return Popen(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_forkserver.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "/Users/maiyunfei/maiyunfei2000.com/code/just-coding/bayes/martin-2-2.py", line 47, in <module>
    trace = pm.sample(1000, step=step, start=start)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/sampling.py", line 559, in sample
    trace = _mp_sample(**sample_args, **parallel_args)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/sampling.py", line 1461, in _mp_sample
    sampler = ps.ParallelSampler(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/parallel_sampling.py", line 431, in __init__
    self._samplers = [
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/parallel_sampling.py", line 432, in <listcomp>
    ProcessAdapter(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pymc3/parallel_sampling.py", line 292, in __init__
    self._process.start()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 291, in _Popen
    return Popen(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_forkserver.py", line 58, in _launch
    f.write(buf.getbuffer())
BrokenPipeError: [Errno 32] Broken pipe

I searched with keyword BrokenPipeError and notice this issue: pymc3 tries to sample model twice in simple code · Issue #4089 · pymc-devs/pymc3, which mentioned that chains=3 can be added to sample and see what happens. After doing so, there’s still the same error as above. The only difference in error message was one line changed fromMultiprocess sampling (2 chains in 2 jobs) to Multiprocess sampling (3 chains in 2 jobs).

Versions and main components

karanotsingyu commented 3 years ago

Problem fixed with cores=1 added to sample method.

And I found that I need to read the Second Edition instead of the First :| ...