Open 5nomads opened 6 months ago
Also running into this. Can confirm distributed="2023.8.0"
works with dask-yarn = "^0.9"
.
This upper bound is wrong because 2024
is a breaking change: Should it be >=2021.1.0,<=2023.8.0
?
Got past that issue and can instantiate a YarnCluster
but can't create a dask.distributed.Client
:
AttributeError: 'YarnCluster' object has no attribute 'status'
Hmm, wasn't the original issue fixed with this commit?
https://github.com/dask/dask-yarn/commit/23a2932a6af9e49e9aacb43555979d878ea5b8f6
Was this never published to pypi?
I setup a new virtual environment and loaded Dask/Dask-Yarn and cannot import due to a deprecated package. I searched for Dask-Yarn and format_bytes and only found the deprecation warning in older versions. I tried both python3.9 and python3.11, I'll include steps for the python3.9 test. I'm not sure what to do about this. Should I pin Dask and Dask-Yarn to specific versions?
Producing the issue:
Possible fix is to import from
dask.utils
in thedask_yarn/core.py
source instead:Setup Steps:
pip freeze:
Python version verification