search

UCLA-IRL / ndn-python-repo

An NDN Repo implementation in Python
Apache License 2.0
16 stars 13 forks source link

Test cases crash due to `asyncio` API changes #62

Closed phylib closed 3 years ago

phylib commented 3 years ago

Here's the output from pytest:

tests/concurrent_fetcher_test.py F                                       [ 11%]
tests/storage_test.py F..                                                [ 44%]
tests/integration/integration_test.py ....F                              [100%]

=================================== FAILURES ===================================
_____________________ TestConcurrentFetcherBasic.test_main _____________________

self = <concurrent_fetcher_test.TestConcurrentFetcherBasic object at 0x7fdb2b9ee610>

    def test_main(self):
        face = DummyFace(self.face_proc)
        keychain = KeychainDigest()
        self.app = NDNApp(face, keychain)
        face.app = self.app
>       self.app.run_forever(after_start=self.app_main())

tests/concurrent_fetcher_test.py:20: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/app.py:323: in run_forever
    aio.run(self.main_loop(after_start))
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/runners.py:44: in run
    return loop.run_until_complete(main)
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/app.py:284: in main_loop
    await self.face.run()
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/transport/dummy_face.py:45: in run
    await self.test_func(self)
tests/concurrent_fetcher_test.py:33: in face_proc
    await face.consume_output(b'\x05"\x07\x1c\x08\x17test_concurrent_fetcher!\x01\x00\x0c\x02\x03\xe8',
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/transport/dummy_face.py:52: in consume_output
    await aio.wait_for(self.event.wait(), timeout)
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/tasks.py:494: in wait_for
    return fut.result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <asyncio.locks.Event object at 0x7fdb2b9eefd0 [set]>

    async def wait(self):
        """Block until the internal flag is true.

        If the internal flag is true on entry, return True
        immediately.  Otherwise, block until another coroutine calls
        set() to set the flag to true, then return True.
        """
        if self._value:
            return True

        fut = self._loop.create_future()
        self._waiters.append(fut)
        try:
>           await fut
E           RuntimeError: Task <Task pending name='Task-3' coro=<Event.wait() running at /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/locks.py:309> cb=[_release_waiter(<Future pendi...b2b987970>()]>)() at /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/tasks.py:429]> got Future <Future pending> attached to a different loop

/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/locks.py:309: RuntimeError
------------------------------ Captured log call -------------------------------
ERROR    asyncio:base_events.py:1707 unhandled exception during asyncio.run() shutdown
task: <Task finished name='Task-5' coro=<concurrent_fetcher.<locals>._retry() done, defined at /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn_python_repo/utils/concurrent_fetcher.py:38> exception=InterestCanceled()>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/app.py", line 284, in main_loop
    await self.face.run()
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/transport/dummy_face.py", line 45, in run
    await self.test_func(self)
  File "/home/runner/work/ndn-python-repo/ndn-python-repo/tests/concurrent_fetcher_test.py", line 33, in face_proc
    await face.consume_output(b'\x05"\x07\x1c\x08\x17test_concurrent_fetcher!\x01\x00\x0c\x02\x03\xe8',
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/transport/dummy_face.py", line 52, in consume_output
    await aio.wait_for(self.event.wait(), timeout)
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/locks.py", line 309, in wait
    await fut
RuntimeError: Task <Task pending name='Task-3' coro=<Event.wait() running at /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/locks.py:309> cb=[_release_waiter(<Future pendi...b2b987970>()]>)() at /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/tasks.py:429]> got Future <Future pending> attached to a different loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/app.py", line 235, in _wait_for_data
    name, meta_info, content, sig, raw_packet = await aio.wait_for(future, timeout=lifetime/1000.0)
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/tasks.py", line 481, in wait_for
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn_python_repo/utils/concurrent_fetcher.py", line 56, in _retry
    data_name, meta_info, content, data_bytes = await app.express_interest(
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/app.py", line 241, in _wait_for_data
    raise InterestCanceled()
ndn.types.InterestCanceled
_________________________ TestSqliteStorage.test_main __________________________

tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_main0')

    @staticmethod
    def test_main(tmp_path):
>       StorageTestFixture.storage = SqliteStorage(tmp_path / 'test.db')

tests/storage_test.py:112: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn_python_repo/storage/sqlite.py:15: in __init__
    super().__init__()
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn_python_repo/storage/storage_base.py:17: in __init__
    aio.get_event_loop().create_task(self._periodic_write_back())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fdb2b9eeee0>

    def get_event_loop(self):
        """Get the event loop for the current context.

        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                isinstance(threading.current_thread(), threading._MainThread)):
            self.set_event_loop(self.new_event_loop())

        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/events.py:639: RuntimeError
_________________________ TestTcpBulkInsert.test_main __________________________

self = <integration_test.TestTcpBulkInsert object at 0x7fdb2b9ee670>

    def test_main(self):
        # could not get NFD running on travis macos, skipping ...
        if os.getenv('TRAVIS') and platform.system() == 'Darwin':
            return
>       self.startup()

tests/integration/integration_test.py:49: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/integration/integration_test.py:61: in startup
    self.app.run_forever(after_start=self.run())
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/app.py:323: in run_forever
    aio.run(self.main_loop(after_start))
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/runners.py:44: in run
    return loop.run_until_complete(main)
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/app.py:292: in main_loop
    await task
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn/app.py:267: in starting_task
    await after_start
tests/integration/integration_test.py:210: in run
    reader, writer = await aio.open_connection('127.0.0.1', port)
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/streams.py:52: in open_connection
    transport, _ = await loop.create_connection(
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/base_events.py:1025: in create_connection
    raise exceptions[0]
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/base_events.py:1010: in create_connection
    sock = await self._connect_sock(
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/base_events.py:924: in _connect_sock
    await self.sock_connect(sock, address)
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/selector_events.py:496: in sock_connect
    return await fut
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=True debug=False>
fut = <Future finished exception=ConnectionRefusedError(111, "Connect call failed ('127.0.0.1', 7377)")>
sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
address = ('127.0.0.1', 7377)

    def _sock_connect_cb(self, fut, sock, address):
        if fut.done():
            return

        try:
            err = sock.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR)
            if err != 0:
                # Jump to any except clause below.
>               raise OSError(err, f'Connect call failed {address}')
E               ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 7377)

/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/selector_events.py:528: ConnectionRefusedError
----------------------------- Captured stderr call -----------------------------
[2021-07-20 18:03:45]INFO:Read handle: listening to /
[2021-07-20 18:03:45]ERROR:Task was destroyed but it is pending!
task: <Task pending name='Task-1' coro=<Storage._periodic_write_back() running at /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn_python_repo/storage/storage_base.py:33>>
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/base_events.py:641: RuntimeWarning: coroutine 'Storage._periodic_write_back' was never awaited
  self._ready.clear()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
[2021-07-20 18:03:45]ERROR:Task was destroyed but it is pending!
task: <Task pending name='Task-2' coro=<TcpBulkInsertHandle.__init__.<locals>.run() running at /opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/site-packages/ndn_python_repo/handle/tcp_bulk_insert_handle.py:71>>
/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/asyncio/base_events.py:641: RuntimeWarning: coroutine 'TcpBulkInsertHandle.__init__.<locals>.run' was never awaited
  self._ready.clear()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
[2021-07-20 18:03:45]INFO:Subscribing to topic: /testrepo/insert
[2021-07-20 18:03:45]INFO:Subscribing to topic: /testrepo/delete
=========================== short test summary info ============================
FAILED tests/concurrent_fetcher_test.py::TestConcurrentFetcherBasic::test_main
FAILED tests/storage_test.py::TestSqliteStorage::test_main - RuntimeError: Th...
FAILED tests/integration/integration_test.py::TestTcpBulkInsert::test_main - ...
========================= 3 failed, 6 passed in 31.23s =========================
Error: Process completed with exit code 1.