Conflict with unittest : unclosed event loop

[patquem]

Hi,

When executing a nbconvert.ExecutePreprocessor() outside a unittest, all works perfectly. But when launching the same function embeded in a unittest class (to execute as a unittest), the following message appears:

c:\dev\sw\python_3.7.7\lib\asyncio\base_events.py:626: ResourceWarning: unclosed event loop <_WindowsSelectorEventLoop running=False closed=False debug=False> source=self) ResourceWarning: Enable tracemalloc to get the object allocation traceback

... with sometimes, a python crash :(

Assertion failed: pfd.revents & POLLIN (C:\projects\libzmq\src\signaler.cpp:265) (I work on a Windows PC and I have no C:\projects !!!)

python kernel: 3.7.7 virtual env created simply with just >> pip install notebook Imported nbconvert version: 7.0.0

To reproduce the problems:

import json
import unittest

from nbconvert.preprocessors import ExecutePreprocessor
from nbformat import read, NO_CONVERT

def notebook_convert(lines):
    cell_type = 'code'
    cell = {'cell_type': cell_type,
            'metadata': {},
            'source': lines.splitlines(True)}
    cell.update({'outputs': [], 'execution_count': None})

    return {'cells': [cell],
            'metadata': {},
            'nbformat': 4,
            'nbformat_minor': 4}

def test():
    fname = 'test.ipynb'
    with open(fname, 'w', encoding='utf-8') as fid:
        json.dump(notebook_convert("1/2"), fid)

    exproc = ExecutePreprocessor()

    with open(fname) as fid:
        nbn = read(fid, as_version=NO_CONVERT)
        exproc.preprocess(nbn)

class TestNotebook(unittest.TestCase):
    def test_notebook():
        test()

test()

Any idea to fix this and then, to avoid hundreds of ResourceWarning messages in a big test suite ? Thanks.

Patrick

NB : note that creating the virtual env just passing by >> pip install nbconvert engenders problem related to kernel-name='python3' during the execution of the previous script - maybe another bug to report ?

[akx]

I think this could be a bug in nbclient: https://github.com/jupyter/nbclient/issues/249

A workaround for your case would be to ensure there is an asyncio loop so nbclient's bits and pieces don't need to create one for every .preprocess invocation.

[patquem]

Thank you @akx for your comment and the issue you opened in jupyter/nbclient. I don't know how to handle the asyncio loop properly in practice, but my (dirty) idea is presently just to close the BaseEventLoop object (in asyncio/base_events.py) before warnings message management, like so:

    def __del__(self):
        self.close()         # ADDED LINE
        if not self.is_closed():
            warnings.warn(f"unclosed event loop {self!r}", ResourceWarning,
                          source=self)
            if not self.is_running():
                self.close()