`kraken`: weird `BsWS` crash on connection reset

[goodboy]

Not sure exactly yet the source of this, and it's super scarce to get, but likely somewhere in the recent history, my guess is this commit..or maybe this one?

Attaching to pdb in crashed actor: ('brokerd.kraken', 'bce89811-23a7-43bc-aeeb-016e7279f91b')

  | trio.MultiError: AssertionError(), Cancelled()
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/home/goodboy/repos/piker/piker/brokers/kraken/broker.py", line 650, in trades_dialogue
    |     await handle_order_updates(
    |   File "/home/goodboy/repos/piker/piker/brokers/kraken/broker.py", line 689, in handle_order_updates
    |     async for msg in ws_stream:
    |   File "/home/goodboy/repos/piker/piker/brokers/kraken/feed.py", line 145, in stream_messages
    |     await ws._connect()
    |   File "/home/goodboy/repos/piker/piker/data/_web_bs.py", line 76, in _connect
    |     await self._stack.aclose()
    |   File "/usr/lib/python3.10/contextlib.py", line 656, in aclose
    |     await self.__aexit__(None, None, None)
    |   File "/usr/lib/python3.10/contextlib.py", line 714, in __aexit__
    |     raise exc_details[1]
    |   File "/usr/lib/python3.10/contextlib.py", line 697, in __aexit__
    |     cb_suppress = await cb(*exc_details)
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/async_generator/_util.py", line 42, in __aexit__
    |     await self._agen.asend(None)
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/async_generator/_impl.py", line 366, in step
    |     return await ANextIter(self._it, start_fn, *args)
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/async_generator/_impl.py", line 202, in send
    |     return self._invoke(self._it.send, value)
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/async_generator/_impl.py", line 209, in _invoke
    |     result = fn(*args)
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio_websocket/_impl.py", line 105, in open_websocket
    |     async with trio.open_nursery() as new_nursery:
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py", line 837, in __aexit__
    |     new_exc = await self._nursery._nested_child_finished(exc)
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py", line 986, in _nested_child_finished
    |     assert popped is self
    | AssertionError
    |
    | During handling of the above exception, another exception occurred:
    |
    | Traceback (most recent call last):
    |   File "/home/goodboy/repos/tractor/tractor/_runtime.py", line 196, in _invoke
    |     res = await coro
    |   File "/home/goodboy/repos/piker/piker/brokers/kraken/broker.py", line 623, in trades_dialogue
    |     async with (
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py", line 837, in __aexit__
    |     new_exc = await self._nursery._nested_child_finished(exc)
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py", line 986, in _nested_child_finished
    |     assert popped is self
    | AssertionError
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py", line 981, in _nested_child_finished
    |     await checkpoint()
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py", line 2421, in checkpoint
    |     await _core.wait_task_rescheduled(lambda _: _core.Abort.SUCCEEDED)
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_traps.py", line 166, in wait_task_rescheduled
    |     return (await _async_yield(WaitTaskRescheduled(abort_func))).unwrap()
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/outcome/_impl.py", line 138, in unwrap
    |     raise captured_error
    |   File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py", line 1222, in raise_cancel
    |     raise Cancelled._create()
    | trio.Cancelled: Cancelled
    +------------------------------------

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/goodboy/repos/tractor/tractor/_runtime.py", line 192, in _invoke
    async with trio.open_nursery() as scope_nursery:
  File "/home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py", line 850, in __aexit__
    raise combined_error_from_nursery
RuntimeError: Cancel scope stack corrupted: attempted to exit <trio.CancelScope at 0x7f77342ea380, active, cancelled> in <Task 'trades_dialogue' at 0x7f77343f7740> that's still within its child <trio.CancelScope at 0x7f7734fa3b20, active, cancelled>

This is probably a bug in your code, that has caused Trio's internal state to
become corrupted. We'll do our best to recover, but from now on there are
no guarantees.

Typically this is caused by one of the following:
  - yielding within a generator or async generator that's opened a cancel
    scope or nursery (unless the generator is a @contextmanager or
    @asynccontextmanager); see https://github.com/python-trio/trio/issues/638
  - manually calling __enter__ or __exit__ on a trio.CancelScope, or
    __aenter__ or __aexit__ on the object returned by trio.open_nursery();
    doing so correctly is difficult and you should use @[async]contextmanager
    instead, or maybe [Async]ExitStack
  - using [Async]ExitStack to interleave the entries/exits of cancel scopes
    and/or nurseries in a way that couldn't be achieved by some nesting of
    'with' and 'async with' blocks
  - using the low-level coroutine object protocol to execute some parts of
    an async function in a different cancel scope/nursery context than
    other parts
If you don't believe you're doing any of these things, please file a bug:
https://github.com/python-trio/trio/issues/new

[1] > /home/goodboy/.virtualenvs/piker310/lib/python3.10/site-packages/trio/_core/_run.py(850)__aexit__()
-> raise combined_error_from_nursery

[goodboy]

Lol figured it out. It's cuz we're trying to call ._connect() again inside the .fixture() when we're already in a disconnected state, which causes a double ._stack.close() call resulting in this error..