search

rapidsai / rapids-dask-dependency

Apache License 2.0
1 stars 13 forks source link

rapid-dasks cannot load on python 3.11 #52

Open morotti opened 5 months ago

morotti commented 5 months ago

Hello,

rapid-dasks cannot import on python 3.11+

>>> import dask.dataframe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/__init__.py", line 98, in <module>
    from dask.dataframe import backends, dispatch, rolling
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/backends.py", line 15, in <module>
    from dask.dataframe.core import DataFrame, Index, Scalar, Series, _Frame
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/core.py", line 36, in <module>
    from dask.dataframe import methods
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/methods.py", line 34, in <module>
    from dask.dataframe.utils import is_dataframe_like, is_index_like, is_series_like
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/utils.py", line 20, in <module>
    from dask.dataframe import (  # noqa: F401 register pandas extension types
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/_dtypes.py", line 9, in <module>
    from dask.dataframe.extensions import make_array_nonempty, make_scalar
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/extensions.py", line 8, in <module>
    from dask.dataframe.accessor import (
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 126, in <module>
    class DatetimeAccessor(Accessor):
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 81, in __init_subclass__
    _bind_property(cls, pd_cls, attr, min_version)
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 35, in _bind_property
    setattr(cls, attr, property(derived_from(pd_cls, version=min_version)(func)))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/utils.py", line 981, in wrapper
    method.__doc__ = _derived_from(
                     ^^^^^^^^^^^^^^
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/utils.py", line 934, in _derived_from
    method_args = get_named_args(method)
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/default-pegasus-venv/lib/python3.11/site-packages/dask/utils.py", line 695, in get_named_args
    s = inspect.signature(func)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/default-pegasus-venv/lib/python3.11/inspect.py", line 3263, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/default-pegasus-venv/lib/python3.11/inspect.py", line 3011, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/default-pegasus-venv/lib/python3.11/inspect.py", line 2599, in _signature_from_callable
    call = _descriptor_get(call, obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/default-pegasus-venv/lib/python3.11/inspect.py", line 2432, in _descriptor_get
    return get(descriptor, obj, type(obj))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: descriptor '__call__' for 'type' objects doesn't apply to a 'property' object

I see you are pinning "dask==2024.1.1" in your pyproject.toml https://github.com/rapidsai/rapids-dask-dependency/blob/main/pyproject.toml

dask 2024.1.1 cannot load on python 3.11 The issue is fixed in dask 2024.4.1 https://github.com/dask/dask/pull/11035

Could you remove the pinning?

wence- commented 5 months ago

It should be the case that if rapids-dask-dependency is installed in the environment, then it patches dask.dataframe on import such that this works even in py3.11+

(see https://github.com/rapidsai/rapids-dask-dependency/pull/41)

Presently, there will also be a 24.06 release which pins to a more recent dask. Can you use that instead?

morotti commented 5 months ago

Thanks, we can use 24.06 when you make a new release.

rjzamora commented 5 months ago

Thanks, we can use 24.06 when you make a new release.

Sounds good - That should resolve the issue.

Regarding 24.04: How did you install rapids (cudf/dask-cudf)? Do you have rapids-dask-dependency installed?