AttributeError: 'KernelManager' object has no attribute 'cleanup'

[ghylander]

Hi

For some reason, executing a notebook results in a AttributeError: 'KernelManager' object has no attribute 'cleanup' error upon terminating execution.

Happens either if the notebook terminates normally or is interrupted by any exception. I have searched for similar issues but found no recent reports of this problem.

I am running in a tensorflow docker container

[ghylander]

As a result of this, the output file (whether it be html, notebook, pdf...) is not saved

[davidbrochart]

Can you show the full traceback and the output of pip freeze? Also, a reproducible example would help.

[ghylander]

Output of pip freeze:

absl-py==1.3.0
ansiwrap==0.8.4
asttokens==2.2.0
astunparse==1.6.3
async-generator==1.10
attrs==22.1.0
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.1
bottle==0.12.21
cachetools==5.2.0
certifi==2019.11.28
chardet==3.0.4
click==8.1.3
contourpy==1.0.6
cycler==0.11.0
dbus-python==1.2.16
debugpy==1.6.4
decorator==5.1.1
defusedxml==0.7.1
distro==1.4.0
entrypoints==0.4
executing==1.2.0
fastjsonschema==2.16.2
flatbuffers==22.10.26
fonttools==4.38.0
gast==0.4.0
google-auth==2.14.1
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.50.0
h5py==3.7.0
idna==2.8
importlib-metadata==5.0.0
importlib-resources==5.10.0
ipykernel==6.17.1
ipython==8.7.0
jedi==0.18.2
Jinja2==3.1.2
joblib==1.1.0
jsonschema==4.17.1
jupyter-client==7.4.7
jupyter-console==6.4.4
jupyter-core==4.11.1
jupyterlab-pygments==0.2.2
keras==2.11.0
Keras-Preprocessing==1.1.2
kiwisolver==1.4.4
libclang==14.0.6
Markdown==3.4.1
MarkupSafe==2.1.1
matplotlib==3.6.2
matplotlib-inline==0.1.6
mistune==2.0.4
nbclient==0.3.0
nbconvert==7.2.5
nbformat==5.6.1
nest-asyncio==1.5.6
numpy==1.23.4
oauthlib==3.2.2
opt-einsum==3.3.0
optuna-dashboard==0.7.1
packaging==21.3
pandas==1.5.2
pandocfilters==1.5.0
papermill==2.4.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.3.0
pkgutil-resolve-name==1.3.10
platformdirs==2.5.4
prompt-toolkit==3.0.33
protobuf==3.19.6
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
Pygments==2.13.0
PyGObject==3.36.0
pyparsing==3.0.9
pyrsistent==0.19.2
python-apt==2.0.0+ubuntu0.20.4.8
python-dateutil==2.8.2
pytz==2022.1
PyYAML==6.0
pyzmq==24.0.1
requests==2.22.0
requests-oauthlib==1.3.1
requests-unixsocket==0.2.0
rsa==4.9
scikit-learn==1.1.2
scipy==1.9.3
six==1.14.0
soupsieve==2.3.2.post1
ssh-import-id==5.10
stack-data==0.6.2
tenacity==8.1.0
tensorboard==2.11.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.11.0
tensorflow-estimator==2.11.0
tensorflow-io-gcs-filesystem==0.27.0
termcolor==2.1.0
textwrap3==0.9.2
threadpoolctl==3.1.0
tinycss2==1.2.1
tornado==6.2
tqdm==4.64.1
traitlets==5.5.0
typing-extensions==4.4.0
urllib3==1.25.8
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.2.2
wrapt==1.14.1
zipp==3.10.0

I'm not sure about how to provide the reproducible example, for me it's any notebook I execute from terminal. This is the command I use: jupyter nbconvert --execute --to html --ExecutePreprocessor.timeout=-1 source.ipynb --output "results/output"

[davidbrochart]

nbconvert 7.2.5 depends on nbclient>=0.5.0, so I don't understand how you can have nbclient 0.3.0. Try updating to nbconvert 7.2.6 and nbclient 0.7.2.

[ghylander]

Hi, currently investigating this issue Unfortunately, I can't simply update the libraries, it may take a bit of time

I tried to find out how I ended up with that conflict in libraries. Seems it was a lengthy process. The machine had been installed the jupyter libraries from ubuntu's focal python packages repository, instead of pypi The jupyter packages in the focal repo are outdated (e.g. jupyter-client is at version 6.1.2-1) However, there is no package for nbclient present in the focal repo, so no version of nbclient was initially installed

At a later point, the library papermill was installed from pypi. This installed newer versions of the jupyter libraries and version 0.7 of nbclient. For some reason, the pypi version of jupyter-client (v7.3.5 at the time) was uninstalled. This resulted on a fallback to the focal repo jupyter-client (v6.1.2-1). This in turn resulted in a documented bug arising from a incompatibility between nbclient v0.7 and jupyter-client<6.1.5. In response to this, nbclient was downgraded to v0.3, but nbconvert was left untouched at the pypi version installed with papermill (v7.2.5 at the time).

Quite the journey! The problem most probably comes from the nbclient and nbconvert versions mismatch, I'll keep this open and close it once I can get to update the libraries or at least bring them both to "compatible" versions