dask / distributed

A distributed task scheduler for Dask
https://distributed.dask.org
BSD 3-Clause "New" or "Revised" License
1.57k stars 719 forks source link

LocalCluster fails in Python3.11 #7211

Open zobac opened 1 year ago

zobac commented 1 year ago

OS: Windows 10 64 bit Python: 3.11.0 Dask: 2022.10.0

>>> from dask.distributed import LocalCluster
>>> cluster = LocalCluster()
Traceback (most recent call last):
  File "E:\Envs\task_management311\Lib\site-packages\distributed\deploy\spec.py", line 277, in __init__
    self.sync(self._correct_state)
  File "E:\Envs\task_management311\Lib\site-packages\distributed\utils.py", line 339, in sync
    return sync(
           ^^^^^
  File "E:\Envs\task_management311\Lib\site-packages\distributed\utils.py", line 406, in sync
    raise exc.with_traceback(tb)
  File "E:\Envs\task_management311\Lib\site-packages\distributed\utils.py", line 379, in f
    result = yield future
             ^^^^^^^^^^^^
  File "E:\Envs\task_management311\Lib\site-packages\tornado\gen.py", line 762, in run
    value = future.result()
            ^^^^^^^^^^^^^^^
  File "E:\Envs\task_management311\Lib\site-packages\distributed\deploy\spec.py", line 366, in _correct_state_internal
    await asyncio.wait(workers)
  File "C:\Python311\Lib\asyncio\tasks.py", line 418, in wait
    return await _wait(fs, timeout, return_when, loop)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\asyncio\tasks.py", line 522, in _wait
    f.add_done_callback(_on_completion)
    ^^^^^^^^^^^^^^^^^^^
AttributeError: 'Nanny' object has no attribute 'add_done_callback'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "E:\Envs\task_management311\Lib\site-packages\distributed\deploy\local.py", line 251, in __init__
    super().__init__(
  File "E:\Envs\task_management311\Lib\site-packages\distributed\deploy\spec.py", line 279, in __init__
    self.sync(self.close)
  File "E:\Envs\task_management311\Lib\site-packages\distributed\utils.py", line 339, in sync
    return sync(
           ^^^^^
  File "E:\Envs\task_management311\Lib\site-packages\distributed\utils.py", line 406, in sync
    raise exc.with_traceback(tb)
  File "E:\Envs\task_management311\Lib\site-packages\distributed\utils.py", line 379, in f
    result = yield future
             ^^^^^^^^^^^^
  File "E:\Envs\task_management311\Lib\site-packages\tornado\gen.py", line 762, in run
    value = future.result()
            ^^^^^^^^^^^^^^^
  File "E:\Envs\task_management311\Lib\site-packages\distributed\deploy\spec.py", line 437, in _close
    assert w.status in {
AssertionError: Status.init
jrbourbeau commented 1 year ago

Thanks for reporting @zobac. This is expected as dask and distributed don't currently have support for Python 3.11 (see https://github.com/dask/distributed/pull/7249). In the meantime, while we work on Python 3.11 compatibility, we recommend keeping Python < 3.11

zobac commented 1 year ago

Thanks. Will do!

Ananaya0303 commented 4 months ago

Facing the same issue in dask 2024.5.0 an python 3.11. Has the above isue been not resolved yet.