search

xorbitsai / xoscar

Python actor framework for heterogeneous computing.
https://xoscar.dev
Apache License 2.0
91 stars 21 forks source link

[BUG] recover_sub_pool may raises RuntimeError: dictionary changed size during iteration #48

Open codingl2k1 opened 1 year ago

codingl2k1 commented 1 year ago

Describe the bug

A clear and concise description of what the bug is.

Traceback (most recent call last):
  File "/Users/codingl2k1/.pyenv/versions/3.11.4/lib/python3.11/site-packages/xoscar/backends/pool.py", line 1402, in monitor_sub_pools
    await self.recover_sub_pool(address)
  File "/Users/codingl2k1/.pyenv/versions/3.11.4/lib/python3.11/site-packages/xoscar/backends/indigen/pool.py", line 329, in recover_sub_pool
    for _, message in self._allocated_actors[address].values():
RuntimeError: dictionary changed size during iteration

To Reproduce

To help us to reproduce this bug, please provide information below:

Not easy to reproduce this error.

  1. Your Python version 3.11.4
  2. The version of Xoscar you use
  3. Versions of crucial packages, such as numpy, scipy and pandas
  4. Full stack of the error.
  5. Minimized code to reproduce the error.

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.