search

rapidsai / cudf

cuDF - GPU DataFrame Library
https://docs.rapids.ai/api/cudf/stable/
Apache License 2.0
8.41k stars 899 forks source link

[BUG] AttributeError: type object 'DatetimeProperties' has no attribute 'components' #15522

Closed pentschev closed 6 months ago

pentschev commented 6 months ago

Describe the bug This is likely a manifestation of the same issue from Python 3.11.9 changes that was observed in Dask which was fixed in https://github.com/dask/dask/pull/11035 .

Error stack ```python Traceback (most recent call last): File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cudf/pandas/fast_slow_proxy.py", line 889, in _fast_slow_function_call result = func(*fast_args, **fast_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: type object 'DatetimeProperties' has no attribute 'components' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 3, in File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cuml/__init__.py", line 17, in from cuml.internals.base import Base, UniversalBase File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cuml/internals/__init__.py", line 18, in from cuml.internals.base_helpers import BaseMetaClass, _tags_class_and_instance File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cuml/internals/base_helpers.py", line 20, in from cuml.internals.api_decorators import ( File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cuml/internals/api_decorators.py", line 24, in from cuml.internals import input_utils as iu File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cuml/internals/input_utils.py", line 19, in from cuml.internals.array import CumlArray File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cuml/internals/array.py", line 51, in DaskCudfDataFrame = gpu_only_import_from("dask_cudf.core", "DataFrame") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cuml/internals/safe_imports.py", line 394, in gpu_only_import_from imported_module = importlib.import_module(module) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask_cudf/__init__.py", line 10, in import dask.dataframe as dd File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 676, in _load_unlocked File "", line 573, in module_from_spec File "/opt/conda/envs/cudf/lib/python3.11/site-packages/rapids_dask_dependency/dask_loader.py", line 36, in create_module mod = importlib.import_module(spec.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/__init__.py", line 40, in from dask.dataframe import backends, dispatch, methods, rolling File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 676, in _load_unlocked File "", line 573, in module_from_spec File "/opt/conda/envs/cudf/lib/python3.11/site-packages/rapids_dask_dependency/dask_loader.py", line 36, in create_module mod = importlib.import_module(spec.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/backends.py", line 15, in from dask.dataframe.core import DataFrame, Index, Scalar, Series, _Frame File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 676, in _load_unlocked File "", line 573, in module_from_spec File "/opt/conda/envs/cudf/lib/python3.11/site-packages/rapids_dask_dependency/dask_loader.py", line 36, in create_module mod = importlib.import_module(spec.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/core.py", line 36, in from dask.dataframe import methods File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 676, in _load_unlocked File "", line 573, in module_from_spec File "/opt/conda/envs/cudf/lib/python3.11/site-packages/rapids_dask_dependency/dask_loader.py", line 36, in create_module mod = importlib.import_module(spec.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/methods.py", line 34, in from dask.dataframe.utils import is_dataframe_like, is_index_like, is_series_like File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 676, in _load_unlocked File "", line 573, in module_from_spec File "/opt/conda/envs/cudf/lib/python3.11/site-packages/rapids_dask_dependency/dask_loader.py", line 36, in create_module mod = importlib.import_module(spec.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/utils.py", line 20, in from dask.dataframe import ( # noqa: F401 register pandas extension types File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 676, in _load_unlocked File "", line 573, in module_from_spec File "/opt/conda/envs/cudf/lib/python3.11/site-packages/rapids_dask_dependency/dask_loader.py", line 36, in create_module mod = importlib.import_module(spec.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/_dtypes.py", line 9, in from dask.dataframe.extensions import make_array_nonempty, make_scalar File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 676, in _load_unlocked File "", line 573, in module_from_spec File "/opt/conda/envs/cudf/lib/python3.11/site-packages/rapids_dask_dependency/dask_loader.py", line 36, in create_module mod = importlib.import_module(spec.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/extensions.py", line 8, in from dask.dataframe.accessor import ( File "", line 1176, in _find_and_load File "", line 1147, in _find_and_load_unlocked File "", line 676, in _load_unlocked File "", line 573, in module_from_spec File "/opt/conda/envs/cudf/lib/python3.11/site-packages/rapids_dask_dependency/dask_loader.py", line 36, in create_module mod = importlib.import_module(spec.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 134, in class DatetimeAccessor(Accessor): File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 89, in __init_subclass__ _bind_property(cls, pd_cls, attr, min_version) File "/opt/conda/envs/cudf/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 32, in _bind_property original_prop = getattr(pd_cls, attr) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cudf/pandas/fast_slow_proxy.py", line 474, in __getattr__ return attr.__get__(None, owner=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cudf/pandas/fast_slow_proxy.py", line 417, in __get__ result, _ = _fast_slow_function_call(getattr, obj, self._name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/cudf/lib/python3.11/site-packages/cudf/pandas/fast_slow_proxy.py", line 902, in _fast_slow_function_call result = func(*slow_args, **slow_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: type object 'DatetimeProperties' has no attribute 'components' ```

Steps/Code to reproduce bug

docker run --gpus all --pull always --rm -it --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 rapidsai/ci-conda:cuda12.2.2-ubuntu22.04-py3.11

# Inside the container
mamba create -n cudf python=3.11 cudatoolkit=11.8 cudf=24.06 cuml=24.06

conda activate cudf

cat << EOF | python
import cudf.pandas
cudf.pandas.install()
from cuml import KMeans
import pandas as pd
import cupy as cp
a = cp.asarray([[1.0, 1.0], [1.0, 2.0], [3.0, 2.0], [4.0, 3.0]],
               dtype=cp.float32)
b = pd.DataFrame(a)
# Calling fit
kmeans_float = KMeans(n_clusters=2)
kmeans_float.fit(b)
EOF

Expected behavior The code should complete without errors.

Environment overview (please complete the following information)

Environment details

Click here to see environment details

     **git***
     commit f19d4eb9f2ccbe1833aa8112c053e622bc138301 (HEAD -> branch-24.06, origin/branch-24.06, origin/HEAD)
     Author: Karthikeyan <6488848+karthikeyann@users.noreply.github.com>
     Date:   Thu Apr 11 23:07:23 2024 -0500

     Fix async synchronization issues in json_column.cu (#15497)

     Fixes #15390
     This change fixes async synchronization issues in json_column.cu.
     Related file json_tree.cu does not have async synchronization issues.

     Summary of changes:
     changed debug print async to sync,
     added synchronize after multiple async calls
     changed h_chars to async since subsequent call is sync (it will also help because chars array is usually large).
     changed is_str_column_all_nulls to sync.

     Authors:
     - Karthikeyan (https://github.com/karthikeyann)

     Approvers:
     - David Wendt (https://github.com/davidwendt)
     - Robert (Bobby) Evans (https://github.com/revans2)
     - Nghia Truong (https://github.com/ttnghia)

     URL: https://github.com/rapidsai/cudf/pull/15497
     **git submodules***

     ***OS Information***
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=22.04
     DISTRIB_CODENAME=jammy
     DISTRIB_DESCRIPTION="Ubuntu 22.04.4 LTS"
     PRETTY_NAME="Ubuntu 22.04.4 LTS"
     NAME="Ubuntu"
     VERSION_ID="22.04"
     VERSION="22.04.4 LTS (Jammy Jellyfish)"
     VERSION_CODENAME=jammy
     ID=ubuntu
     ID_LIKE=debian
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     UBUNTU_CODENAME=jammy
     Linux b986deaf397f 5.4.0-159-generic #176-Ubuntu SMP Mon Aug 14 12:04:20 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Fri Apr 12 09:04:15 2024
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.2     |
     |-------------------------------+----------------------+----------------------+
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  Tesla V100-SXM2...  On   | 00000000:06:00.0 Off |                    0 |
     | N/A   33C    P0    57W / 300W |    309MiB / 32768MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   1  Tesla V100-SXM2...  On   | 00000000:07:00.0 Off |                    0 |
     | N/A   30C    P0    42W / 300W |      3MiB / 32768MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   2  Tesla V100-SXM2...  On   | 00000000:0A:00.0 Off |                    0 |
     | N/A   31C    P0    41W / 300W |      3MiB / 32768MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   3  Tesla V100-SXM2...  On   | 00000000:0B:00.0 Off |                    0 |
     | N/A   28C    P0    41W / 300W |      3MiB / 32768MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   4  Tesla V100-SXM2...  On   | 00000000:85:00.0 Off |                    0 |
     | N/A   31C    P0    42W / 300W |      3MiB / 32768MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   5  Tesla V100-SXM2...  On   | 00000000:86:00.0 Off |                    0 |
     | N/A   29C    P0    41W / 300W |      3MiB / 32768MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   6  Tesla V100-SXM2...  On   | 00000000:89:00.0 Off |                    0 |
     | N/A   33C    P0    42W / 300W |      5MiB / 32768MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   7  Tesla V100-SXM2...  On   | 00000000:8A:00.0 Off |                    0 |
     | N/A   29C    P0    40W / 300W |      3MiB / 32768MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+

     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     +-----------------------------------------------------------------------------+

     ***CPU***
     Architecture:                       x86_64
     CPU op-mode(s):                     32-bit, 64-bit
     Address sizes:                      46 bits physical, 48 bits virtual
     Byte Order:                         Little Endian
     CPU(s):                             80
     On-line CPU(s) list:                0-79
     Vendor ID:                          GenuineIntel
     Model name:                         Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz
     CPU family:                         6
     Model:                              79
     Thread(s) per core:                 2
     Core(s) per socket:                 20
     Socket(s):                          2
     Stepping:                           1
     CPU max MHz:                        3600.0000
     CPU min MHz:                        1200.0000
     BogoMIPS:                           4389.94
     Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts md_clear flush_l1d
     Virtualization:                     VT-x
     L1d cache:                          1.3 MiB (40 instances)
     L1i cache:                          1.3 MiB (40 instances)
     L2 cache:                           10 MiB (40 instances)
     L3 cache:                           100 MiB (2 instances)
     NUMA node(s):                       2
     NUMA node0 CPU(s):                  0-19,40-59
     NUMA node1 CPU(s):                  20-39,60-79
     Vulnerability Gather data sampling: Not affected
     Vulnerability Itlb multihit:        KVM: Mitigation: Split huge pages
     Vulnerability L1tf:                 Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
     Vulnerability Mds:                  Mitigation; Clear CPU buffers; SMT vulnerable
     Vulnerability Meltdown:             Mitigation; PTI
     Vulnerability Mmio stale data:      Mitigation; Clear CPU buffers; SMT vulnerable
     Vulnerability Retbleed:             Not affected
     Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
     Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
     Vulnerability Spectre v2:           Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
     Vulnerability Srbds:                Not affected
     Vulnerability Tsx async abort:      Mitigation; Clear CPU buffers; SMT vulnerable

     ***CMake***

     ***g++***

     ***nvcc***

     ***Python***
     /opt/conda/envs/cudf/bin/python
     Python 3.11.8

     ***Environment Variables***
     PATH                            : /opt/conda/envs/cudf/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
     LD_LIBRARY_PATH                 : /usr/local/nvidia/lib:/usr/local/nvidia/lib64
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    : /opt/conda/envs/cudf
     PYTHON_PATH                     :

     ***conda packages***
     /opt/conda/condabin/conda
     # packages in environment at /opt/conda/envs/cudf:
     #
     # Name                    Version                   Build  Channel
     _libgcc_mutex             0.1                 conda_forge    conda-forge
     _openmp_mutex             4.5                       2_gnu    conda-forge
     aws-c-auth                0.7.17               he0b1f16_2    conda-forge
     aws-c-cal                 0.6.11               heb1d5e4_0    conda-forge
     aws-c-common              0.9.15               hd590300_0    conda-forge
     aws-c-compression         0.2.18               hce8ee76_3    conda-forge
     aws-c-event-stream        0.4.2                h01f5eca_8    conda-forge
     aws-c-http                0.8.1               hdb68c23_10    conda-forge
     aws-c-io                  0.14.7               h76a7424_3    conda-forge
     aws-c-mqtt                0.10.3               h50844eb_4    conda-forge
     aws-c-s3                  0.5.7                hb7bd14b_1    conda-forge
     aws-c-sdkutils            0.1.15               hce8ee76_3    conda-forge
     aws-checksums             0.1.18               hce8ee76_3    conda-forge
     aws-crt-cpp               0.26.6               hf567797_4    conda-forge
     aws-sdk-cpp               1.11.267             hbf3e495_6    conda-forge
     bokeh                     3.4.1              pyhd8ed1ab_0    conda-forge
     brotli-python             1.1.0           py311hb755f60_1    conda-forge
     bzip2                     1.0.8                hd590300_5    conda-forge
     c-ares                    1.28.1               hd590300_0    conda-forge
     ca-certificates           2024.2.2             hbcca054_0    conda-forge
     cachetools                5.3.3              pyhd8ed1ab_0    conda-forge
     click                     8.1.7           unix_pyh707e725_0    conda-forge
     cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
     contourpy                 1.2.1           py311h9547e67_0    conda-forge
     cubinlinker               0.3.0           py311h4b55656_1    rapidsai
     cuda-profiler-api         11.8.86                       0    nvidia
     cuda-python               11.8.3          py311h8bf0e4b_1    conda-forge
     cuda-version              11.8                 h70ddcb2_3    conda-forge
     cudatoolkit               11.8.0              h4ba93d1_13    conda-forge
     cudf                      24.06.00a96     cuda11_py311_240412_gf19d4eb9f2_96    rapidsai-nightly
     cuml                      24.06.00a18     cuda11_py311_240411_gd14d7e93e_18    rapidsai-nightly
     cupy                      13.0.0          py311h878bca4_3    conda-forge
     cupy-core                 13.0.0          py311heecd119_3    conda-forge
     cytoolz                   0.12.3          py311h459d7ec_0    conda-forge
     dask                      2024.4.2a240411  py_g66ced13b_1    dask/label/dev
     dask-core                 2024.4.2a240408 py_gb2ec1e1a2_4    dask/label/dev
     dask-cuda                 24.06.00a10     py311_240412_g63198ea_10    rapidsai-nightly
     dask-cudf                 24.06.00a96     cuda11_py311_240412_gf19d4eb9f2_96    rapidsai-nightly
     dask-expr                 1.0.12a240409     py_gad4d195_1    dask/label/dev
     distributed               2024.4.2a240411  py_g66ced13b_1    dask/label/dev
     dlpack                    0.8                  h59595ed_3    conda-forge
     fastrlock                 0.8.2           py311hb755f60_2    conda-forge
     fmt                       10.2.1               h00ab1b0_0    conda-forge
     freetype                  2.12.1               h267a509_2    conda-forge
     fsspec                    2024.3.1           pyhca7485f_0    conda-forge
     gflags                    2.2.2             he1b5a44_1004    conda-forge
     glog                      0.7.0                hed5481d_0    conda-forge
     gmock                     1.14.0               ha770c72_1    conda-forge
     gtest                     1.14.0               h00ab1b0_1    conda-forge
     icu                       73.2                 h59595ed_0    conda-forge
     importlib-metadata        7.1.0              pyha770c72_0    conda-forge
     importlib_metadata        7.1.0                hd8ed1ab_0    conda-forge
     jinja2                    3.1.3              pyhd8ed1ab_0    conda-forge
     joblib                    1.4.0              pyhd8ed1ab_0    conda-forge
     keyutils                  1.6.1                h166bdaf_0    conda-forge
     krb5                      1.21.2               h659d440_0    conda-forge
     lcms2                     2.16                 hb7c19ff_0    conda-forge
     ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
     lerc                      4.0.0                h27087fc_0    conda-forge
     libabseil                 20240116.1      cxx17_h59595ed_2    conda-forge
     libarrow                  14.0.2          he70291f_16_cpu    conda-forge
     libarrow-acero            14.0.2          hac33072_16_cpu    conda-forge
     libarrow-dataset          14.0.2          hac33072_16_cpu    conda-forge
     libarrow-flight           14.0.2          hd42f311_16_cpu    conda-forge
     libarrow-flight-sql       14.0.2          h9241762_16_cpu    conda-forge
     libarrow-gandiva          14.0.2          hd4ab825_16_cpu    conda-forge
     libarrow-substrait        14.0.2          h9241762_16_cpu    conda-forge
     libblas                   3.9.0           22_linux64_openblas    conda-forge
     libbrotlicommon           1.1.0                hd590300_1    conda-forge
     libbrotlidec              1.1.0                hd590300_1    conda-forge
     libbrotlienc              1.1.0                hd590300_1    conda-forge
     libcblas                  3.9.0           22_linux64_openblas    conda-forge
     libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
     libcublas                 11.11.3.6                     0    nvidia
     libcublas-dev             11.11.3.6                     0    nvidia
     libcudf                   24.06.00a96     cuda11_240412_gf19d4eb9f2_96    rapidsai-nightly
     libcufft                  10.9.0.58                     0    nvidia
     libcufile                 1.4.0.31                      0    nvidia
     libcufile-dev             1.4.0.31                      0    nvidia
     libcuml                   24.06.00a18     cuda11_240411_gd14d7e93e_18    rapidsai-nightly
     libcumlprims              24.06.00a       cuda11_240412_g98a3699_7    rapidsai-nightly
     libcurand                 10.3.0.86                     0    nvidia
     libcurand-dev             10.3.0.86                     0    nvidia
     libcurl                   8.7.1                hca28451_0    conda-forge
     libcusolver               11.4.1.48                     0    nvidia
     libcusolver-dev           11.4.1.48                     0    nvidia
     libcusparse               11.7.5.86                     0    nvidia
     libcusparse-dev           11.7.5.86                     0    nvidia
     libdeflate                1.20                 hd590300_0    conda-forge
     libedit                   3.1.20191231         he28a2e2_2    conda-forge
     libev                     4.33                 hd590300_2    conda-forge
     libevent                  2.1.12               hf998b51_1    conda-forge
     libexpat                  2.6.2                h59595ed_0    conda-forge
     libffi                    3.4.2                h7f98852_5    conda-forge
     libgcc-ng                 13.2.0               h807b86a_5    conda-forge
     libgfortran-ng            13.2.0               h69a702a_5    conda-forge
     libgfortran5              13.2.0               ha4646dd_5    conda-forge
     libgomp                   13.2.0               h807b86a_5    conda-forge
     libgoogle-cloud           2.22.0               h9be4e54_1    conda-forge
     libgoogle-cloud-storage   2.22.0               hc7a4891_1    conda-forge
     libgrpc                   1.62.1               h15f2491_0    conda-forge
     libiconv                  1.17                 hd590300_2    conda-forge
     libjpeg-turbo             3.0.0                hd590300_1    conda-forge
     libkvikio                 24.06.00a       cuda11_240412_g7b0231c_11    rapidsai-nightly
     liblapack                 3.9.0           22_linux64_openblas    conda-forge
     libllvm14                 14.0.6               hcd5def8_4    conda-forge
     libllvm15                 15.0.7               hb3ce162_4    conda-forge
     libnghttp2                1.58.0               h47da74e_1    conda-forge
     libnl                     3.9.0                hd590300_0    conda-forge
     libnsl                    2.0.1                hd590300_0    conda-forge
     libopenblas               0.3.27          pthreads_h413a1c8_0    conda-forge
     libparquet                14.0.2          h6a7eafb_16_cpu    conda-forge
     libpng                    1.6.43               h2797004_0    conda-forge
     libprotobuf               4.25.3               h08a7969_0    conda-forge
     libraft                   24.06.00a33     cuda11_240412_g316a0658_33    rapidsai-nightly
     libraft-headers           24.06.00a33     cuda11_240412_g316a0658_33    rapidsai-nightly
     libraft-headers-only      24.06.00a33     cuda11_240412_g316a0658_33    rapidsai-nightly
     libre2-11                 2023.09.01           h5a48ba9_2    conda-forge
     librmm                    24.06.00a11     cuda11_240412_g9e02f34a_11    rapidsai-nightly
     libsqlite                 3.45.2               h2797004_0    conda-forge
     libssh2                   1.11.0               h0841786_0    conda-forge
     libstdcxx-ng              13.2.0               h7e041cc_5    conda-forge
     libthrift                 0.19.0               hb90f79a_1    conda-forge
     libtiff                   4.6.0                h1dd3fc0_3    conda-forge
     libutf8proc               2.8.0                h166bdaf_0    conda-forge
     libuuid                   2.38.1               h0b41bf4_0    conda-forge
     libwebp-base              1.3.2                hd590300_1    conda-forge
     libxcb                    1.15                 h0b41bf4_0    conda-forge
     libxcrypt                 4.4.36               hd590300_1    conda-forge
     libxml2                   2.12.6               h232c23b_1    conda-forge
     libzlib                   1.2.13               hd590300_5    conda-forge
     llvmlite                  0.42.0          py311ha6695c7_1    conda-forge
     locket                    1.0.0              pyhd8ed1ab_0    conda-forge
     lz4                       4.3.3           py311h38e4bf4_0    conda-forge
     lz4-c                     1.9.4                hcb278e6_0    conda-forge
     markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
     markupsafe                2.1.5           py311h459d7ec_0    conda-forge
     mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
     msgpack-python            1.0.7           py311h9547e67_0    conda-forge
     nccl                      2.21.5.1             h6103f9b_0    conda-forge
     ncurses                   6.4.20240210         h59595ed_0    conda-forge
     numba                     0.59.1          py311h96b013e_0    conda-forge
     numpy                     1.26.4          py311h64a7726_0    conda-forge
     nvcomp                    3.0.6                h56904bc_0    conda-forge
     nvtx                      0.2.10          py311h459d7ec_0    conda-forge
     openjpeg                  2.5.2                h488ebb8_0    conda-forge
     openssl                   3.2.1                hd590300_1    conda-forge
     orc                       2.0.0                h17fec99_1    conda-forge
     packaging                 24.0               pyhd8ed1ab_0    conda-forge
     pandas                    2.2.1           py311h320fe9a_0    conda-forge
     partd                     1.4.1              pyhd8ed1ab_0    conda-forge
     pillow                    10.3.0          py311h18e6fac_0    conda-forge
     pip                       24.0               pyhd8ed1ab_0    conda-forge
     protobuf                  4.25.3          py311h7b78aeb_0    conda-forge
     psutil                    5.9.8           py311h459d7ec_0    conda-forge
     pthread-stubs             0.4               h36c2ea0_1001    conda-forge
     ptxcompiler               0.8.1           py311h2c5c81a_2    conda-forge
     pyarrow                   14.0.2          py311h78dcc79_16_cpu    conda-forge
     pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
     pylibraft                 24.06.00a33     cuda11_py311_240411_g316a0658_33    rapidsai-nightly
     pynvml                    11.4.1             pyhd8ed1ab_0    conda-forge
     pysocks                   1.7.1              pyha2e5f31_6    conda-forge
     python                    3.11.8          hab00c5b_0_cpython    conda-forge
     python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
     python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
     python_abi                3.11                    4_cp311    conda-forge
     pytz                      2024.1             pyhd8ed1ab_0    conda-forge
     pyyaml                    6.0.1           py311h459d7ec_1    conda-forge
     raft-dask                 24.06.00a33     cuda11_py311_240411_g316a0658_33    rapidsai-nightly
     rapids-dask-dependency    24.06.00a20                py_0    rapidsai-nightly
     rdma-core                 51.0                 hd3aeb46_0    conda-forge
     re2                       2023.09.01           h7f4b329_2    conda-forge
     readline                  8.2                  h8228510_1    conda-forge
     rich                      13.7.1             pyhd8ed1ab_0    conda-forge
     rmm                       24.06.00a11     cuda11_py311_240412_g9e02f34a_11    rapidsai-nightly
     s2n                       1.4.9                h06160fa_0    conda-forge
     scipy                     1.13.0          py311h64a7726_0    conda-forge
     setuptools                69.2.0             pyhd8ed1ab_0    conda-forge
     six                       1.16.0             pyh6c4a22f_0    conda-forge
     snappy                    1.2.0                hdb0a2a9_1    conda-forge
     sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
     spdlog                    1.12.0               hd2e6256_2    conda-forge
     tblib                     3.0.0              pyhd8ed1ab_0    conda-forge
     tk                        8.6.13          noxft_h4845f30_101    conda-forge
     toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
     tornado                   6.4             py311h459d7ec_0    conda-forge
     treelite                  4.1.2           py311hd96da90_0    conda-forge
     typing_extensions         4.11.0             pyha770c72_0    conda-forge
     tzdata                    2024a                h0c530f3_0    conda-forge
     ucx                       1.15.0               ha691c75_8    conda-forge
     ucx-proc                  1.0.0                       gpu    rapidsai
     ucx-py                    0.38.00a0       py311_240412_g16667b4_0    rapidsai-nightly
     urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
     wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
     xorg-libxau               1.0.11               hd590300_0    conda-forge
     xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
     xyzservices               2024.4.0           pyhd8ed1ab_0    conda-forge
     xz                        5.2.6                h166bdaf_0    conda-forge
     yaml                      0.2.5                h7f98852_2    conda-forge
     zict                      3.0.0              pyhd8ed1ab_0    conda-forge
     zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
     zstd                      1.5.5                hfc55251_0    conda-forge
wence- commented 6 months ago

Let me look, it should be possible to fix this in the cudf.pandas wrapper magic.

wence- commented 6 months ago

Simpler repro:

import cudf.pandas
cudf.pandas.install()
import dask.dataframe
wence- commented 6 months ago

Can you try:

diff --git a/python/cudf/cudf/pandas/_wrappers/pandas.py b/python/cudf/cudf/pandas/_wrappers/pandas.py
index b7c8e92e8d..3c82d57193 100644
--- a/python/cudf/cudf/pandas/_wrappers/pandas.py
+++ b/python/cudf/cudf/pandas/_wrappers/pandas.py
@@ -174,7 +174,7 @@ Series = make_final_proxy_type(
         "__arrow_array__": arrow_array_method,
         "__cuda_array_interface__": cuda_array_interface,
         "__iter__": custom_iter,
-        "dt": _AccessorAttr(DatetimeProperties),
+        "dt": _AccessorAttr(CombinedDatetimelikeProperties),
         "str": _AccessorAttr(StringMethods),
         "cat": _AccessorAttr(_CategoricalAccessor),
         "_constructor": _FastSlowAttribute("_constructor"),
@@ -208,7 +208,7 @@ Index = make_final_proxy_type(
         "__array_function__": array_function_method,
         "__arrow_array__": arrow_array_method,
         "__cuda_array_interface__": cuda_array_interface,
-        "dt": _AccessorAttr(DatetimeProperties),
+        "dt": _AccessorAttr(CombinedDatetimelikeProperties),
         "str": _AccessorAttr(StringMethods),
         "cat": _AccessorAttr(_CategoricalAccessor),
         "__iter__": custom_iter,
wence- commented 6 months ago

This showed up a problem in our pandas wrappers in cudf.pandas (which the above patch fixes), but when we merged dask/dask#11035 we accidentally dropped some exception handling, that I reinstate in https://github.com/dask/dask/pull/11049

pentschev commented 6 months ago

Thanks @wence- , yes, using the patch from https://github.com/rapidsai/cudf/issues/15522#issuecomment-2051507770 the code completes without errors in my docker+conda environment. Could you submit a PR for that?