ethereum / trinity

The Trinity client for the Ethereum network
https://trinity.ethereum.org
MIT License
474 stars 145 forks source link

Flaky test in peer packer #1517

Closed pipermerriam closed 4 years ago

pipermerriam commented 4 years ago

What is wrong?

The following test is flaky due to what appears to be a race condition between a channel being closed during shutdown of the packer and a response that tries to send over the channel.

$ pytest tests-trio/ -x
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.7.3, pytest-5.3.1, py-1.8.0, pluggy-0.13.1
Using --randomly-seed=1580410601
rootdir: /home/piper/projects/trinity, inifile: pytest.ini
plugins: hypothesis-4.47.3, cov-2.8.1, xdist-1.29.0, mock-1.12.1, randomly-3.1.0, forked-1.1.3, trio-0.5.2, web3-5.4.0
collected 66 items

tests-trio/eth1-monitor/test_eth1_monitor.py ....                                                                                                                                                   [  6%]
tests-trio/p2p-trio/test_routing_table_manager.py ......                                                                                                                                            [ 15%]
tests-trio/p2p-trio/test_enr_db.py ................                                                                                                                                                 [ 39%]
tests-trio/p2p-trio/test_channel_services.py .....                                                                                                                                                  [ 46%]
tests-trio/p2p-trio/test_packer.py ..F

================================================================================================ FAILURES =================================================================================================
_______________________________________________________________________________ test_packer_processes_handshake_initiation ________________________________________________________________________________

value = <trio.Nursery object at 0x7f18d1fd1d68>

    async def yield_(value=None):
>       return await _yield_(value)

value      = <trio.Nursery object at 0x7f18d1fd1d68>

../../python-environments/trinity/lib/python3.7/site-packages/async_generator/_impl.py:106:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../python-environments/trinity/lib/python3.7/site-packages/async_generator/_impl.py:99: in _yield_
    return (yield _wrap(value))
tests-trio/p2p-trio/test_packer.py:223: in packer
    yield packer
../../python-environments/trinity/lib/python3.7/site-packages/async_generator/_util.py:42: in __aexit__
    await self._agen.asend(None)
../../python-environments/trinity/lib/python3.7/site-packages/async_service/trio.py:409: in background_trio_service
    await manager.stop()
../../python-environments/trinity/lib/python3.7/site-packages/trio/_core/_run.py:730: in __aexit__
    raise combined_error_from_nursery
../../python-environments/trinity/lib/python3.7/site-packages/async_service/trio.py:208: in run
    for _, exc_value, exc_tb in self._errors
../../python-environments/trinity/lib/python3.7/site-packages/async_service/base.py:300: in _run_and_manage_task
    await task.run()
../../python-environments/trinity/lib/python3.7/site-packages/async_service/trio.py:76: in run
    await self._async_fn(*self._async_fn_args)
p2p/discv5/packer.py:535: in handle_incoming_packets
    await managed_peer_packer.incoming_packet_send_channel.send(incoming_packet)
../../python-environments/trinity/lib/python3.7/site-packages/trio/_channel.py:174: in send
    await trio.hazmat.wait_task_rescheduled(abort_fn)
../../python-environments/trinity/lib/python3.7/site-packages/trio/_core/_traps.py:165: in wait_task_rescheduled
    return (await _async_yield(WaitTaskRescheduled(abort_func))).unwrap()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Error(BrokenResourceError())

    def unwrap(self):
        self._set_unwrapped()
        # Tracebacks show the 'raise' line below out of context, so let's give
        # this variable a name that makes sense out of context.
        captured_error = self.error
>       raise captured_error
E       trio.BrokenResourceError

captured_error = BrokenResourceError()
self       = Error(BrokenResourceError())

How can it be fixed

This might be as simple as catching and handling this exception.

jannikluhn commented 4 years ago

Closed by #1522