Open hustwolf opened 5 years ago
Given you're on an old version of nbconvert I am guessing you're also on old versions of it'd dependencies. I do highly recommend upgrading versions and seeing if the issue goes away.
As a secondary possible issue, are you launching multiple nbconvert calls at the same time or have other kernels being launched simultaneously? There's some known race conditions that I've been helping track down. Upgrading your jupyter_client
to the latest releast will eliminate one dead lock condition that can cause that message. Another is pending a PR merge/release in https://github.com/ipython/ipykernel/pull/412 . And the final fix in nbconvert is merged but waiting on the ipykernel PR before releasing which enables parallel processing over nbconvert (I don't think you're hitting this one atm).
FYI this should now be fixed in ipykernel 5.1.2 -- it was occurring if there were two kernels launched close to each other, with an unresolved race condition for the port.
Nope, not fixed yet in 5.1.2
I am still facing this issue in ipykernel 5.1.3, nbconvert 5.6.1. Executing the notebook in Docker container.
Could you try upgrading jupyter_client instead (5.3.4)? @martinRenou: you fixed this right, or can it still happen?
Hmm... how reproducible is the issue for you? How are you running nbconvert and is there any other processes or notebook related tasks running in parallel? Is the file system mounted or internal to the container?
you fixed this right, or can it still happen?
It can technically still happen. In the case of multiple KernelManagers (multiple servers) on the same machine for example. What I did was only a workaround, I added a cache of the current ports that are used, to prevent the kernel manager to choose the same port for multiple kernels.
Can we reopen this issue?
I continue to see these errors and I'm not so sure how to proceed.
@martinRenou do you think there's a easy workaround for this?
How robust do you think it's to just have a retry loop?
This error still appears even after upgrading to ipykernel-5.5.0 and jupyter-client-6.1.12.
Does anyone have a workaround or suggested resolution for this? We've got jupyter-nbconvert
calls operating in parallel as part of our CI system and we hit this error maybe 25% of the time, creating a bunch of false-positive failures in our testing pipeline.
Is jupyter-nbconvert
intended to be parallel safe?
It is intended to be parallel safe. There was a lot of work in nbclient and jupyter_client to support it at the library level.
The root problem is likely this one: https://github.com/jupyter/jupyter_client/issues/487 where we're having port conflicts with two kernels lanching at the same time. My work on that got interrupted by having a child :baby: so it never got fixed as proposed in that thread. It's back in my queue but I'm not sure when I'll have the time to finish implementation.
Thanks for the update @MSeal !
All who have problem around should check option c.KernelManager.transport = "ipc"
All who have problem around should check option
c.KernelManager.transport = "ipc"
I just tried it. Although no error output, the process hangs. I just use pytorch_lightning:
model = CycleGAN()
ds = DataModule()
trainer = pl.Trainer(max_epochs=1,fast_dev_run=True,gpus=-1,accelerator='ddp',weights_summary=None)
trainer.fit(model, ds)
All who have problem around should check option c.KernelManager.transport = "ipc"
Is there anyone that can confirm that this solution actually fixed their problem?
@basnijholt for me still working, but I not changed anything around env, lobs, etc.
Unfortunately, this didn't work in our CI pipelines.
I managed to circumvent the issue by setting up the kernel in a locked context before calling execute()
:
import asyncio
from pathlib import Path
import nbformat
import portalocker
from nbclient import NotebookClient
def run_nb(filename: Path = Path("some_notebook.ipynb")):
nb = nbformat.read(filename)
executor = NotebookClient(
nb,
kernel_name="python3",
resources={"metadata": {"path": filename.parent}},
)
# Prepare kernel
executor.km = executor.create_kernel_manager()
with portalocker.Lock("jupyter_kernel.lock", timeout=300):
asyncio.run(executor.async_start_new_kernel())
asyncio.run(executor.async_start_new_kernel_client())
return executor.execute()
@basnijholt I think taht Your case is diffrent coz you use jupyter? Right? Try outside jupyter. For jupyter you need management around extensions propably.
I kept getting the following:
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/ipykernel_launcher.py", line 17, in <module>
app.launch_new_instance()
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/traitlets/config/application.py", line 1042, in launch_instance
app.initialize(argv)
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/traitlets/config/application.py", line 113, in inner
return method(app, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/ipykernel/kernelapp.py", line 678, in initialize
self.init_sockets()
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/ipykernel/kernelapp.py", line 332, in init_sockets
self.init_iopub(context)
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/ipykernel/kernelapp.py", line 361, in init_iopub
self.iopub_port = self._bind_socket(self.iopub_socket, self.iopub_port)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/ipykernel/kernelapp.py", line 252, in _bind_socket
return self._try_bind_socket(s, port)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/ipykernel/kernelapp.py", line 228, in _try_bind_socket
s.bind("tcp://%s:%i" % (self.ip, port))
File "/opt/miniconda3/envs/pt/lib/python3.11/site-packages/zmq/sugar/socket.py", line 301, in bind
super().bind(addr)
File "zmq/backend/cython/socket.pyx", line 564, in zmq.backend.cython.socket.Socket.bind
File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Address already in use
It seems to have been fixed by setting c.KernelManager.transport = "ipc"
in Jupyter Lab configuration file.
Environment (Ubuntu 22.04):
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
aiofiles 22.1.0 pyhd8ed1ab_0 conda-forge
aiosqlite 0.18.0 pyhd8ed1ab_0 conda-forge
alsa-lib 1.2.8 h166bdaf_0 conda-forge
anyio 3.6.2 pyhd8ed1ab_0 conda-forge
argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py311hd4cff14_3 conda-forge
asttokens 2.2.1 pyhd8ed1ab_0 conda-forge
attr 2.5.1 h166bdaf_1 conda-forge
attrs 22.2.0 pyh71513ae_0 conda-forge
babel 2.12.1 pyhd8ed1ab_1 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.12.0 pyha770c72_0 conda-forge
blas 1.0 mkl conda-forge
bleach 6.0.0 pyhd8ed1ab_0 conda-forge
brotli 1.0.9 h166bdaf_8 conda-forge
brotli-bin 1.0.9 h166bdaf_8 conda-forge
brotlipy 0.7.0 py311h9bf148f_1002 pytorch-nightly
bzip2 1.0.8 h7f98852_4 conda-forge
ca-certificates 2022.12.7 ha878542_0 conda-forge
cairo 1.16.0 ha61ee94_1014 conda-forge
certifi 2022.12.7 pyhd8ed1ab_0 conda-forge
cffi 1.15.1 py311h9bf148f_3 pytorch-nightly
charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge
comm 0.1.3 pyhd8ed1ab_0 conda-forge
contourpy 1.0.7 py311ha3edf6b_0 conda-forge
cryptography 38.0.4 py311h46ebde7_0 pytorch-nightly
cuda-cudart 11.8.89 0 nvidia
cuda-cupti 11.8.87 0 nvidia
cuda-libraries 11.8.0 0 nvidia
cuda-nvrtc 11.8.89 0 nvidia
cuda-nvtx 11.8.86 0 nvidia
cuda-runtime 11.8.0 0 nvidia
cudatoolkit 11.8.0 h37601d7_11 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
dbus 1.13.6 h5008d03_3 conda-forge
debugpy 1.6.6 py311hcafe171_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
expat 2.5.0 h27087fc_0 conda-forge
ffmpeg 4.3 hf484d3e_0 pytorch
fftw 3.3.10 nompi_hf0379b8_106 conda-forge
filelock 3.9.0 py311_0 pytorch-nightly
flit-core 3.8.0 pyhd8ed1ab_0 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 hab24e00_0 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.39.2 py311h2582759_0 conda-forge
freetype 2.12.1 hca18f0e_1 conda-forge
gettext 0.21.1 h27087fc_0 conda-forge
giflib 5.2.1 h0b41bf4_3 conda-forge
glib 2.74.1 h6239696_1 conda-forge
glib-tools 2.74.1 h6239696_1 conda-forge
gmp 6.2.1 h58526e2_0 conda-forge
gmpy2 2.1.2 py311h6a5fa03_1 conda-forge
gnutls 3.6.13 h85f3911_1 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
gst-plugins-base 1.22.0 h4243ec0_2 conda-forge
gstreamer 1.22.0 h25f0c4b_2 conda-forge
gstreamer-orc 0.4.33 h166bdaf_0 conda-forge
harfbuzz 6.0.0 h8e241bc_0 conda-forge
icu 70.1 h27087fc_0 conda-forge
idna 3.4 pyhd8ed1ab_0 conda-forge
importlib-metadata 6.1.0 pyha770c72_0 conda-forge
importlib_metadata 6.1.0 hd8ed1ab_0 conda-forge
importlib_resources 5.12.0 pyhd8ed1ab_0 conda-forge
intel-openmp 2023.0.0 h9e868ea_25371
ipykernel 6.22.0 pyh210e3f2_0 conda-forge
ipython 8.11.0 pyh41d4057_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
jack 1.9.22 h11f4161_0 conda-forge
jedi 0.18.2 pyhd8ed1ab_0 conda-forge
jinja2 3.1.2 pyhd8ed1ab_1 conda-forge
jpeg 9e h0b41bf4_3 conda-forge
json5 0.9.5 pyh9f0ad1d_0 conda-forge
jsonschema 4.17.3 pyhd8ed1ab_0 conda-forge
jupyter_client 8.1.0 pyhd8ed1ab_0 conda-forge
jupyter_core 5.3.0 py311h38be061_0 conda-forge
jupyter_events 0.6.3 pyhd8ed1ab_0 conda-forge
jupyter_server 2.5.0 pyhd8ed1ab_0 conda-forge
jupyter_server_fileid 0.8.0 pyhd8ed1ab_0 conda-forge
jupyter_server_terminals 0.4.4 pyhd8ed1ab_1 conda-forge
jupyter_server_ydoc 0.8.0 pyhd8ed1ab_0 conda-forge
jupyter_ydoc 0.2.3 pyhd8ed1ab_0 conda-forge
jupyterlab 3.6.2 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge
jupyterlab_server 2.21.0 pyhd8ed1ab_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.4 py311h4dd048b_1 conda-forge
krb5 1.20.1 h81ceb04_0 conda-forge
lame 3.100 h166bdaf_1003 conda-forge
lcms2 2.15 hfd0df8a_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libblas 3.9.0 1_h86c2bf4_netlib conda-forge
libbrotlicommon 1.0.9 h166bdaf_8 conda-forge
libbrotlidec 1.0.9 h166bdaf_8 conda-forge
libbrotlienc 1.0.9 h166bdaf_8 conda-forge
libcap 2.67 he9d0100_0 conda-forge
libcblas 3.9.0 5_h92ddd45_netlib conda-forge
libclang 15.0.7 default_had23c3d_1 conda-forge
libclang13 15.0.7 default_h3e3d535_1 conda-forge
libcublas 11.11.3.6 0 nvidia
libcufft 10.9.0.58 0 nvidia
libcufile 1.6.0.25 0 nvidia
libcups 2.3.3 h36d4200_3 conda-forge
libcurand 10.3.2.56 0 nvidia
libcusolver 11.4.1.48 0 nvidia
libcusparse 11.7.5.86 0 nvidia
libdb 6.2.32 h9c3ff4c_0 conda-forge
libdeflate 1.17 h0b41bf4_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libevent 2.1.10 h28343ad_4 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libflac 1.4.2 h27087fc_0 conda-forge
libgcc-ng 12.2.0 h65d4601_19 conda-forge
libgcrypt 1.10.1 h166bdaf_0 conda-forge
libgfortran-ng 12.2.0 h69a702a_19 conda-forge
libgfortran5 12.2.0 h337968e_19 conda-forge
libglib 2.74.1 h606061b_1 conda-forge
libgomp 12.2.0 h65d4601_19 conda-forge
libgpg-error 1.46 h620e276_0 conda-forge
libhwloc 2.9.0 hd6dc26d_0 conda-forge
libiconv 1.17 h166bdaf_0 conda-forge
liblapack 3.9.0 5_h92ddd45_netlib conda-forge
libllvm15 15.0.7 hadd5161_1 conda-forge
libnpp 11.8.0.86 0 nvidia
libnsl 2.0.0 h7f98852_0 conda-forge
libnvjpeg 11.9.0.86 0 nvidia
libogg 1.3.4 h7f98852_1 conda-forge
libopus 1.3.1 h7f98852_1 conda-forge
libpng 1.6.39 h753d276_0 conda-forge
libpq 15.2 hb675445_0 conda-forge
libsndfile 1.2.0 hb75c966_0 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libsqlite 3.40.0 h753d276_0 conda-forge
libstdcxx-ng 12.2.0 h46fd767_19 conda-forge
libsystemd0 253 h8c4010b_1 conda-forge
libtiff 4.5.0 h6adf6a1_2 conda-forge
libtool 2.4.7 h27087fc_0 conda-forge
libudev1 253 h0b41bf4_1 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libuv 1.44.2 h166bdaf_0 conda-forge
libvorbis 1.3.7 h9c3ff4c_0 conda-forge
libwebp 1.2.4 h1daa5a0_1 conda-forge
libwebp-base 1.2.4 h166bdaf_0 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libxkbcommon 1.5.0 h79f4944_1 conda-forge
libxml2 2.10.3 hca2bb57_4 conda-forge
libzlib 1.2.13 h166bdaf_4 conda-forge
lz4-c 1.9.4 hcb278e6_0 conda-forge
markupsafe 2.1.2 py311h2582759_0 conda-forge
matplotlib 3.7.1 py311h38be061_0 conda-forge
matplotlib-base 3.7.1 py311h8597a09_0 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mistune 2.0.5 pyhd8ed1ab_0 conda-forge
mkl 2023.0.0 h6d00ec8_25399
mpc 1.3.1 hfe3b2da_0 conda-forge
mpfr 4.2.0 hb012696_0 conda-forge
mpg123 1.31.3 hcb278e6_0 conda-forge
mpmath 1.2.1 py311_0 pytorch-nightly
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
mysql-common 8.0.32 ha901b37_1 conda-forge
mysql-libs 8.0.32 hd7da12d_1 conda-forge
nbclassic 0.5.3 pyhb4ecaf3_3 conda-forge
nbclient 0.7.2 pyhd8ed1ab_0 conda-forge
nbconvert 7.2.9 pyhd8ed1ab_0 conda-forge
nbconvert-core 7.2.9 pyhd8ed1ab_0 conda-forge
nbconvert-pandoc 7.2.9 pyhd8ed1ab_0 conda-forge
nbformat 5.8.0 pyhd8ed1ab_0 conda-forge
ncurses 6.3 h27087fc_1 conda-forge
nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge
nettle 3.6 he412f7d_0 conda-forge
networkx 3.0 pyhd8ed1ab_0 conda-forge
nodejs 18.15.0 h8d033a5_0 conda-forge
notebook 6.5.3 pyha770c72_0 conda-forge
notebook-shim 0.2.2 pyhd8ed1ab_0 conda-forge
nspr 4.35 h27087fc_0 conda-forge
nss 3.89 he45b914_0 conda-forge
numpy 1.24.2 py311h8e6699e_0 conda-forge
openh264 2.1.1 h780b84a_0 conda-forge
openssl 3.1.0 h0b41bf4_0 conda-forge
packaging 23.0 pyhd8ed1ab_0 conda-forge
pandas 1.5.3 py311h2872171_0 conda-forge
pandoc 3.1.1 h32600fe_0 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pcre2 10.40 hc3806b6_0 conda-forge
pexpect 4.8.0 pyh1a96a4e_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.3.0 py311h3fd9d12_2 pytorch-nightly
pip 23.0.1 pyhd8ed1ab_0 conda-forge
pixman 0.40.0 h36c2ea0_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_0 conda-forge
platformdirs 3.1.1 pyhd8ed1ab_0 conda-forge
plotly 5.13.1 pyhd8ed1ab_0 conda-forge
ply 3.11 py_1 conda-forge
pooch 1.7.0 pyha770c72_3 conda-forge
prometheus_client 0.16.0 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.38 pyha770c72_0 conda-forge
prompt_toolkit 3.0.38 hd8ed1ab_0 conda-forge
psutil 5.9.4 py311hd4cff14_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pulseaudio 16.1 hcb278e6_3 conda-forge
pulseaudio-client 16.1 h5195f5e_3 conda-forge
pulseaudio-daemon 16.1 ha8d29e2_3 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.14.0 pyhd8ed1ab_0 conda-forge
pyopenssl 23.1.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pyqt 5.15.7 py311ha74522f_3 conda-forge
pyqt5-sip 12.11.0 py311hcafe171_3 conda-forge
pyrsistent 0.19.3 py311h2582759_0 conda-forge
pysocks 1.7.1 py311_0 pytorch-nightly
python 3.11.0 he550d4f_1_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.16.3 pyhd8ed1ab_0 conda-forge
python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge
python_abi 3.11 3_cp311 conda-forge
pytorch 2.1.0.dev20230324 py3.11_cuda11.8_cudnn8.7.0_0 pytorch-nightly
pytorch-cuda 11.8 h7e8668a_3 pytorch-nightly
pytorch-mutex 1.0 cuda pytorch-nightly
pytz 2022.7.1 pyhd8ed1ab_0 conda-forge
pyyaml 6.0 py311hd4cff14_5 conda-forge
pyzmq 25.0.2 py311hd6ccaeb_0 conda-forge
qt-main 5.15.8 h5d23da1_6 conda-forge
readline 8.2 h8228510_1 conda-forge
requests 2.28.1 py311_0 pytorch-nightly
rfc3339-validator 0.1.4 pyhd8ed1ab_0 conda-forge
rfc3986-validator 0.1.1 pyh9f0ad1d_0 conda-forge
scipy 1.10.1 py311h8e6699e_0 conda-forge
send2trash 1.8.0 pyhd8ed1ab_0 conda-forge
setuptools 67.6.0 pyhd8ed1ab_0 conda-forge
sip 6.7.7 py311hcafe171_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sniffio 1.3.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
sympy 1.11.1 pypyh9d50eac_103 conda-forge
tbb 2021.8.0 hf52228f_0 conda-forge
tenacity 8.2.2 pyhd8ed1ab_0 conda-forge
terminado 0.17.1 pyh41d4057_0 conda-forge
tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge
tk 8.6.12 h27826a3_0 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
torchaudio 2.1.0.dev20230324 py311_cu118 pytorch-nightly
torchtriton 2.1.0+e650d3708b py311 pytorch-nightly
torchvision 0.16.0.dev20230324 py311_cu118 pytorch-nightly
tornado 6.2 py311hd4cff14_1 conda-forge
traitlets 5.9.0 pyhd8ed1ab_0 conda-forge
typing-extensions 4.5.0 hd8ed1ab_0 conda-forge
typing_extensions 4.5.0 pyha770c72_0 conda-forge
tzdata 2023b h71feb2d_0 conda-forge
urllib3 1.26.14 py311_0 pytorch-nightly
wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
webencodings 0.5.1 py_1 conda-forge
websocket-client 1.5.1 pyhd8ed1ab_0 conda-forge
wheel 0.40.0 pyhd8ed1ab_0 conda-forge
xcb-util 0.4.0 h516909a_0 conda-forge
xcb-util-image 0.4.0 h166bdaf_0 conda-forge
xcb-util-keysyms 0.4.0 h516909a_0 conda-forge
xcb-util-renderutil 0.3.9 h166bdaf_0 conda-forge
xcb-util-wm 0.4.1 h516909a_0 conda-forge
xkeyboard-config 2.38 h0b41bf4_0 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.0.10 h7f98852_0 conda-forge
xorg-libsm 1.2.3 hd9c2040_1000 conda-forge
xorg-libx11 1.8.4 h0b41bf4_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxrender 0.9.10 h7f98852_1003 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
y-py 0.5.9 py311hfe55011_0 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
ypy-websocket 0.8.2 pyhd8ed1ab_0 conda-forge
zeromq 4.3.4 h9c3ff4c_1 conda-forge
zipp 3.15.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.13 h166bdaf_4 conda-forge
zstd 1.5.2 h3eb15da_6 conda-forge
i use nbconvert to convert notebook to html. occasionally may come to this error of "zmq.error.ZMQError: Address already in use" the full error stacktrace is [NbConvertApp] Executing notebook with kernel: python3 Traceback (most recent call last): File "/data/anaconda3/envs/jupyterhub/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/data/anaconda3/envs/jupyterhub/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "", line 2, in initialize
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, kwargs)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 456, in initialize
self.init_sockets()
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 243, in init_sockets
self.stdin_port = self._bind_socket(self.stdin_socket, self.stdin_port)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 180, in _bind_socket
s.bind("tcp://%s:%i" % (self.ip, port))
File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.socket.Socket.bind
File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Address already in use
Traceback (most recent call last):
File "/data/anaconda3/envs/jupyterhub/bin/jupyter-nbconvert", line 11, in
sys.exit(main())
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, kwargs)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 325, in start
self.convert_notebooks()
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 493, in convert_notebooks
self.convert_single_notebook(notebook_filename)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 464, in convert_single_notebook
output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 393, in export_single_notebook
output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 174, in from_filename
return self.from_file(f, resources=resources, kw)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 192, in from_file
return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, kw)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/exporters/html.py", line 85, in from_notebook_node
return super(HTMLExporter, self).from_notebook_node(nb, resources, kw)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/exporters/templateexporter.py", line 280, in from_notebook_node
nb_copy, resources = super(TemplateExporter, self).from_notebook_node(nb, resources, kw)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 134, in from_notebook_node
nb_copy, resources = self._preprocess(nb_copy, resources)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 311, in _preprocess
nbc, resc = preprocessor(nbc, resc)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/preprocessors/base.py", line 47, in call
return self.preprocess(nb, resources)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py", line 257, in preprocess
cwd=path)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py", line 241, in start_new_kernel
kc.wait_for_ready(timeout=startup_timeout)
File "/data/anaconda3/envs/jupyterhub/lib/python3.6/site-packages/jupyter_client/blocking/client.py", line 120, in wait_for_ready
raise RuntimeError('Kernel died before replying to kernel_info')
RuntimeError: Kernel died before replying to kernel_info
since my job is a cron job by every hour, this error occurs or may not occur
Nbconvert version: 5.3.1