search

QuantEcon / sphinxcontrib-jupyter

A Sphinx Extension for Generating Jupyter Notebooks
BSD 3-Clause "New" or "Revised" License
74 stars 23 forks source link

BUG: Using large number of workers (parallel execution) results in ZMQErrors #261

Open mmcky opened 4 years ago

mmcky commented 4 years ago

When using > 8 workers it is reasonably common to get ZMQErrors. This should be diagnosed and resolved.

raceback (most recent call last):
  File "/home/qebuild/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/qebuild/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/qebuild/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py", line 16, in <module>
    app.launch_new_instance()
  File "/home/qebuild/anaconda3/lib/python3.7/site-packages/traitlets/config/application.py", line 657, in launch_instance
    app.initialize(argv)
  File "</home/qebuild/anaconda3/lib/python3.7/site-packages/decorator.py:decorator-gen-124>", line 2, in initialize
  File "/home/qebuild/anaconda3/lib/python3.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/qebuild/anaconda3/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 469, in initialize
    self.init_sockets()
  File "/home/qebuild/anaconda3/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 248, in init_sockets
    self.control_port = self._bind_socket(self.control_socket, self.control_port)
  File "/home/qebuild/anaconda3/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 180, in _bind_socket
    s.bind("tcp://%s:%i" % (self.ip, port))
  File "zmq/backend/cython/socket.pyx", line 547, in zmq.backend.cython.socket.Socket.bind
  File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Address already in use
distributed.worker - WARNING -  Compute Failed
Function:  preprocess
args:      ({