rapidsai / cudf

cuDF - GPU DataFrame Library
https://docs.rapids.ai/api/cudf/stable/
Apache License 2.0
8.35k stars 890 forks source link

[BUG] Can't pickle cudf pandas `GroupBy` objects #14738

Closed paulsbrookes closed 9 months ago

paulsbrookes commented 9 months ago

Describe the bug We can't pickle GroupBy objects from cudf.pandas

Steps/Code to reproduce bug The following code fails

import cudf.pandas
cudf.pandas.install()
import pandas as pd
import pickle

data = {
    'Category': ['A', 'B', 'A', 'C', 'B', 'A', 'C', 'A', 'B', 'C'],
    'Values': [10, 20, 15, 30, 25, 10, 35, 20, 15, 40]
}

df = pd.DataFrame(data)

grouped = df.groupby('Category')

with open('grouped.pickle', 'wb') as file:
    pickle.dump(grouped, file)

with the error

_pickle.PicklingError: Can't pickle <class 'pandas.core.groupby.generic.DataFrameGroupBy'>: it's not the same object as pandas.core.groupby.generic.DataFrameGroupBy

I think somewhere inside pickle a type check is performed comparing a regular pandas GroupBy object with a cudf pandas GroupBy object. Since these types are different, the code fails.

Expected behavior I would expect to able to pickle theseGroupBy objects.

Environment overview (please complete the following information)

Environment details

Click here to see environment details

     **git***
     commit 5f91a95985923d9ded953b75f5f6c1cfb4243dad (HEAD -> feature/cudf-pandas-integration, origin/feature/cudf-pandas-integration)
     Author: Paul Brookes 
     Date:   Wed Jan 10 19:02:46 2024 +0000

     fix: deepcopy
     **git submodules***
     c71f159aa733ec6d904ffce23039e28259608c13 makefiles (v0.1.5-4-gc71f159)

     ***OS Information***
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=20.04
     DISTRIB_CODENAME=focal
     DISTRIB_DESCRIPTION="Ubuntu 20.04.4 LTS"
     NAME="Ubuntu"
     VERSION="20.04.4 LTS (Focal Fossa)"
     ID=ubuntu
     ID_LIKE=debian
     PRETTY_NAME="Ubuntu 20.04.4 LTS"
     VERSION_ID="20.04"
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     VERSION_CODENAME=focal
     UBUNTU_CODENAME=focal
     Linux TurinTech-0004 5.15.0-91-generic #101~20.04.1-Ubuntu SMP Thu Nov 16 14:22:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Wed Jan 10 20:35:31 2024
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 525.147.05   Driver Version: 525.147.05   CUDA Version: 12.0     |
     |-------------------------------+----------------------+----------------------+
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  NVIDIA GeForce ...  Off  | 00000000:09:00.0  On |                  N/A |
     | 47%   54C    P3    39W / 170W |   3138MiB / 12288MiB |     16%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+

     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     |    0   N/A  N/A      1526      G   /usr/lib/xorg/Xorg                198MiB |
     |    0   N/A  N/A      2714      G   /usr/lib/xorg/Xorg               1974MiB |
     |    0   N/A  N/A      2840      G   /usr/bin/gnome-shell              294MiB |
     |    0   N/A  N/A      3300      G   ...AAAAAAAAA= --shared-files      109MiB |
     |    0   N/A  N/A     33097      G   ...666990459706826775,131072      345MiB |
     |    0   N/A  N/A     49347      G   ...RendererForSitePerProcess       56MiB |
     |    0   N/A  N/A     49997      G   ...RendererForSitePerProcess       81MiB |
     +-----------------------------------------------------------------------------+

     ***CPU***
     Architecture:                       x86_64
     CPU op-mode(s):                     32-bit, 64-bit
     Byte Order:                         Little Endian
     Address sizes:                      48 bits physical, 48 bits virtual
     CPU(s):                             24
     On-line CPU(s) list:                0-23
     Thread(s) per core:                 2
     Core(s) per socket:                 12
     Socket(s):                          1
     NUMA node(s):                       1
     Vendor ID:                          AuthenticAMD
     CPU family:                         25
     Model:                              33
     Model name:                         AMD Ryzen 9 5900X 12-Core Processor
     Stepping:                           0
     Frequency boost:                    enabled
     CPU MHz:                            3700.000
     CPU max MHz:                        3700.0000
     CPU min MHz:                        2200.0000
     BogoMIPS:                           7386.08
     Virtualisation:                     AMD-V
     L1d cache:                          384 KiB
     L1i cache:                          384 KiB
     L2 cache:                           6 MiB
     L3 cache:                           64 MiB
     NUMA node0 CPU(s):                  0-23
     Vulnerability Gather data sampling: Not affected
     Vulnerability Itlb multihit:        Not affected
     Vulnerability L1tf:                 Not affected
     Vulnerability Mds:                  Not affected
     Vulnerability Meltdown:             Not affected
     Vulnerability Mmio stale data:      Not affected
     Vulnerability Retbleed:             Not affected
     Vulnerability Spec rstack overflow: Mitigation; safe RET, no microcode
     Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
     Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
     Vulnerability Spectre v2:           Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
     Vulnerability Srbds:                Not affected
     Vulnerability Tsx async abort:      Not affected
     Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm

     ***CMake***

     ***g++***
     /usr/bin/g++
     g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
     Copyright (C) 2019 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

     ***nvcc***
     /home/paul/anaconda3/envs/rapids-23.12/bin/nvcc
     nvcc: NVIDIA (R) Cuda compiler driver
     Copyright (c) 2005-2022 NVIDIA Corporation
     Built on Mon_Oct_24_19:12:58_PDT_2022
     Cuda compilation tools, release 12.0, V12.0.76
     Build cuda_12.0.r12.0/compiler.31968024_0

     ***Python***
     /home/paul/anaconda3/envs/rapids-23.12/bin/python
     Python 3.9.18

     ***Environment Variables***
     PATH                            : /home/paul/anaconda3/envs/rapids-23.12/bin:/home/paul/anaconda3/condabin:/home/paul/.cargo/bin:/home/paul/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
     LD_LIBRARY_PATH                 :
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    : /home/paul/anaconda3/envs/rapids-23.12
     PYTHON_PATH                     :

     ***conda packages***
     /home/paul/anaconda3/condabin/conda
     # packages in environment at /home/paul/anaconda3/envs/rapids-23.12:
     #
     # Name                    Version                   Build  Channel
     _libgcc_mutex             0.1                 conda_forge    conda-forge
     _openmp_mutex             4.5                       2_gnu    conda-forge
     accelerate                0.19.0                   pypi_0    pypi
     adagio                    0.2.4                    pypi_0    pypi
     aiohttp                   3.9.1            py39hd1e30aa_0    conda-forge
     aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
     alabaster                 0.7.13                   pypi_0    pypi
     alembic                   1.13.1                   pypi_0    pypi
     antlr4-python3-runtime    4.11.1                   pypi_0    pypi
     anyio                     4.2.0              pyhd8ed1ab_0    conda-forge
     aom                       3.7.1                h59595ed_0    conda-forge
     appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
     argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
     argon2-cffi-bindings      21.2.0           py39hd1e30aa_4    conda-forge
     arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
     astroid                   2.15.8                   pypi_0    pypi
     async-timeout             4.0.3              pyhd8ed1ab_0    conda-forge
     attrs                     23.1.0             pyh71513ae_1    conda-forge
     aws-c-auth                0.7.8                hcf8cf63_3    conda-forge
     aws-c-cal                 0.6.9                h5d48c4d_2    conda-forge
     aws-c-common              0.9.10               hd590300_0    conda-forge
     aws-c-compression         0.2.17               h7f92143_7    conda-forge
     aws-c-event-stream        0.3.2                h0bcb0bb_8    conda-forge
     aws-c-http                0.7.15               hd268abd_0    conda-forge
     aws-c-io                  0.13.36              he0cd244_2    conda-forge
     aws-c-mqtt                0.10.0               hbafccad_1    conda-forge
     aws-c-s3                  0.4.5                h47b1690_1    conda-forge
     aws-c-sdkutils            0.1.13               h7f92143_0    conda-forge
     aws-checksums             0.1.17               h7f92143_6    conda-forge
     aws-crt-cpp               0.25.0               h169d4cb_3    conda-forge
     aws-sdk-cpp               1.11.210             h0853bfa_5    conda-forge
     azure-core-cpp            1.10.3               h91d86a7_0    conda-forge
     azure-storage-blobs-cpp   12.10.0              h00ab1b0_0    conda-forge
     azure-storage-common-cpp  12.5.0               hb858b4b_2    conda-forge
     babel                     2.14.0                   pypi_0    pypi
     beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
     black                     23.7.0                   pypi_0    pypi
     bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
     blosc                     1.21.5               h0f2a231_0    conda-forge
     bokeh                     3.3.2              pyhd8ed1ab_0    conda-forge
     branca                    0.7.0              pyhd8ed1ab_1    conda-forge
     brotli                    1.1.0                hd590300_1    conda-forge
     brotli-bin                1.1.0                hd590300_1    conda-forge
     brotli-python             1.1.0            py39h3d6467e_1    conda-forge
     brunsli                   0.1                  h9c3ff4c_0    conda-forge
     bzip2                     1.0.8                hd590300_5    conda-forge
     c-ares                    1.24.0               hd590300_0    conda-forge
     c-blosc2                  2.11.3               hb4ffafa_0    conda-forge
     ca-certificates           2023.11.17           hbcca054_0    conda-forge
     cached-property           1.5.2                hd8ed1ab_1    conda-forge
     cached_property           1.5.2              pyha770c72_1    conda-forge
     cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
     cairo                     1.18.0               h3faef2a_0    conda-forge
     catboost                  1.2.2                    pypi_0    pypi
     category-encoders         2.6.3                    pypi_0    pypi
     certifi                   2023.11.17         pyhd8ed1ab_0    conda-forge
     cffi                      1.16.0           py39h7a31438_0    conda-forge
     cfgv                      3.4.0                    pypi_0    pypi
     cfitsio                   4.3.1                hbdc6101_0    conda-forge
     charls                    2.4.2                h59595ed_0    conda-forge
     charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
     click                     8.0.4                    pypi_0    pypi
     click-plugins             1.1.1                      py_0    conda-forge
     cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
     cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
     cmaes                     0.10.0                   pypi_0    pypi
     colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
     colorcet                  3.0.1              pyhd8ed1ab_0    conda-forge
     colorlog                  6.8.0                    pypi_0    pypi
     contourpy                 1.2.0            py39h7633fee_0    conda-forge
     coverage                  7.3.4                    pypi_0    pypi
     cryptography              41.0.7                   pypi_0    pypi
     cucim                     23.12.01        cuda12_py39_231211_ga3445df_0    rapidsai
     cuda-cccl_linux-64        12.0.90              ha770c72_1    conda-forge
     cuda-cudart               12.0.107             hd3aeb46_8    conda-forge
     cuda-cudart-dev           12.0.107             hd3aeb46_8    conda-forge
     cuda-cudart-dev_linux-64  12.0.107             h59595ed_8    conda-forge
     cuda-cudart-static        12.0.107             hd3aeb46_8    conda-forge
     cuda-cudart-static_linux-64 12.0.107             h59595ed_8    conda-forge
     cuda-cudart_linux-64      12.0.107             h59595ed_8    conda-forge
     cuda-nvcc-dev_linux-64    12.0.76              ha770c72_1    conda-forge
     cuda-nvcc-impl            12.0.76              h59595ed_1    conda-forge
     cuda-nvcc-tools           12.0.76              h59595ed_1    conda-forge
     cuda-nvrtc                12.0.76              hd3aeb46_2    conda-forge
     cuda-nvtx                 12.0.76              h59595ed_1    conda-forge
     cuda-profiler-api         12.0.76              ha770c72_0    conda-forge
     cuda-python               12.0.0           py39h2d39e0c_4    conda-forge
     cuda-version              12.0                 hffde075_2    conda-forge
     cudf                      23.12.01        cuda12_py39_231208_g2ce46216b5_0    rapidsai
     cudf_kafka                23.12.01        cuda12_py39_231208_g2ce46216b5_0    rapidsai
     cugraph                   23.12.00        cuda12_py39_231206_g1309813f_0    rapidsai
     cuml                      23.12.00        cuda12_py39_231206_gad2bd2b65_0    rapidsai
     cuproj                    23.12.00        cuda12_py39_231206_g3a357729_0    rapidsai
     cupy                      12.3.0           py39hc7c1505_0    conda-forge
     cuspatial                 23.12.01        cuda12_py39_231207_g16727064_0    rapidsai
     custreamz                 23.12.01        cuda12_py39_231208_g2ce46216b5_0    rapidsai
     cuxfilter                 23.12.00        cuda12_py39_231206_g63dabeb_0    rapidsai
     cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
     cyrus-sasl                2.1.27               h54b06d7_7    conda-forge
     cython                    3.0.7                    pypi_0    pypi
     cytoolz                   0.12.2           py39hd1e30aa_1    conda-forge
     daal                      2023.2.0                 pypi_0    pypi
     daal4py                   2023.2.0                 pypi_0    pypi
     darts                     0.27.1                   pypi_0    pypi
     dask                      2023.11.0          pyhd8ed1ab_0    conda-forge
     dask-core                 2023.11.0          pyhd8ed1ab_0    conda-forge
     dask-cuda                 23.12.00        py39_231206_ge1638ae_0    rapidsai
     dask-cudf                 23.12.01        cuda12_py39_231208_g2ce46216b5_0    rapidsai
     datasets                  2.15.0                   pypi_0    pypi
     datashader                0.16.0             pyhd8ed1ab_0    conda-forge
     dav1d                     1.2.1                hd590300_0    conda-forge
     defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
     deprecated                1.2.14                   pypi_0    pypi
     dill                      0.3.7                    pypi_0    pypi
     distlib                   0.3.8                    pypi_0    pypi
     distributed               2023.11.0          pyhd8ed1ab_0    conda-forge
     dlpack                    0.5                  h9c3ff4c_0    conda-forge
     docutils                  0.18.1                   pypi_0    pypi
     entrypoints               0.4                pyhd8ed1ab_0    conda-forge
     exceptiongroup            1.2.0              pyhd8ed1ab_0    conda-forge
     execnet                   2.0.2                    pypi_0    pypi
     expat                     2.5.0                hcb278e6_1    conda-forge
     fastrlock                 0.8.2            py39h3d6467e_2    conda-forge
     filelock                  3.13.1                   pypi_0    pypi
     fiona                     1.9.5            py39hcfcd403_2    conda-forge
     flake8                    6.0.0                    pypi_0    pypi
     fmt                       9.1.0                h924138e_0    conda-forge
     folium                    0.15.1             pyhd8ed1ab_0    conda-forge
     font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
     font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
     font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
     font-ttf-ubuntu           0.83                 h77eed37_1    conda-forge
     fontconfig                2.14.2               h14ed4e7_0    conda-forge
     fonts-conda-ecosystem     1                             0    conda-forge
     fonts-conda-forge         1                             0    conda-forge
     fonttools                 4.47.0           py39hd1e30aa_0    conda-forge
     fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
     freetype                  2.12.1               h267a509_2    conda-forge
     freexl                    2.0.0                h743c826_0    conda-forge
     frozendict                2.3.10                   pypi_0    pypi
     frozenlist                1.4.1            py39hd1e30aa_0    conda-forge
     fs                        2.4.16                   pypi_0    pypi
     fsspec                    2023.10.0                pypi_0    pypi
     fugue                     0.8.7                    pypi_0    pypi
     fugue-sql-antlr           0.2.0                    pypi_0    pypi
     gdal                      3.8.2            py39h14df8fe_0    conda-forge
     gdk-pixbuf                2.42.10              h829c605_4    conda-forge
     geopandas                 0.14.1             pyhd8ed1ab_0    conda-forge
     geopandas-base            0.14.1             pyha770c72_0    conda-forge
     geos                      3.12.1               h59595ed_0    conda-forge
     geotiff                   1.7.1               h6b2125f_15    conda-forge
     gettext                   0.21.1               h27087fc_0    conda-forge
     gflags                    2.2.2             he1b5a44_1004    conda-forge
     giflib                    5.2.1                h0b41bf4_3    conda-forge
     glog                      0.6.0                h6f12383_0    conda-forge
     gmock                     1.14.0               ha770c72_1    conda-forge
     greenlet                  3.0.3                    pypi_0    pypi
     gtest                     1.14.0               h00ab1b0_1    conda-forge
     hdf4                      4.2.15               h2a13503_7    conda-forge
     hdf5                      1.14.3          nompi_h4f84152_100    conda-forge
     holidays                  0.27                     pypi_0    pypi
     holoviews                 1.18.1             pyhd8ed1ab_0    conda-forge
     huggingface-hub           0.20.1                   pypi_0    pypi
     icu                       73.2                 h59595ed_0    conda-forge
     identify                  2.5.33                   pypi_0    pypi
     idna                      3.6                pyhd8ed1ab_0    conda-forge
     imagecodecs               2023.9.18        py39hf9b8f0e_2    conda-forge
     imageio                   2.33.1             pyh8c1a49c_0    conda-forge
     imagesize                 1.4.1                    pypi_0    pypi
     imbalanced-learn          0.11.0                   pypi_0    pypi
     importlib-metadata        7.0.0              pyha770c72_0    conda-forge
     importlib-resources       6.1.1              pyhd8ed1ab_0    conda-forge
     importlib_metadata        7.0.0                hd8ed1ab_0    conda-forge
     importlib_resources       6.1.1              pyhd8ed1ab_0    conda-forge
     iniconfig                 2.0.0                    pypi_0    pypi
     isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
     isort                     5.13.2                   pypi_0    pypi
     jaraco-classes            3.3.0                    pypi_0    pypi
     jbig                      2.1               h7f98852_2003    conda-forge
     jeepney                   0.8.0                    pypi_0    pypi
     jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
     joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
     json-c                    0.17                 h7ab15ed_0    conda-forge
     jsonpointer               2.4              py39hf3d152e_3    conda-forge
     jsonschema                4.20.0             pyhd8ed1ab_0    conda-forge
     jsonschema-specifications 2023.11.2          pyhd8ed1ab_0    conda-forge
     jsonschema-with-format-nongpl 4.20.0             pyhd8ed1ab_0    conda-forge
     jupyter-server-proxy      4.1.0              pyhd8ed1ab_0    conda-forge
     jupyter_client            8.6.0              pyhd8ed1ab_0    conda-forge
     jupyter_core              5.5.1            py39hf3d152e_0    conda-forge
     jupyter_events            0.9.0              pyhd8ed1ab_0    conda-forge
     jupyter_server            2.12.1             pyhd8ed1ab_0    conda-forge
     jupyter_server_terminals  0.5.0              pyhd8ed1ab_0    conda-forge
     jupyterlab_pygments       0.3.0              pyhd8ed1ab_0    conda-forge
     jxrlib                    1.1                  h7f98852_2    conda-forge
     kealib                    1.5.3                h2f55d51_0    conda-forge
     keyring                   24.3.0                   pypi_0    pypi
     keyutils                  1.6.1                h166bdaf_0    conda-forge
     kiwisolver                1.4.5            py39h7633fee_1    conda-forge
     krb5                      1.21.2               h659d440_0    conda-forge
     lazy-object-proxy         1.10.0                   pypi_0    pypi
     lazy_loader               0.3                pyhd8ed1ab_0    conda-forge
     lcms2                     2.16                 hb7c19ff_0    conda-forge
     ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
     lerc                      4.0.0                h27087fc_0    conda-forge
     libabseil                 20230802.1      cxx17_h59595ed_0    conda-forge
     libaec                    1.1.2                h59595ed_1    conda-forge
     libarchive                3.7.2                h2aa1ff5_1    conda-forge
     libarrow                  14.0.2           hfb4d3a9_0_cpu    conda-forge
     libarrow-acero            14.0.2           h59595ed_0_cpu    conda-forge
     libarrow-dataset          14.0.2           h59595ed_0_cpu    conda-forge
     libarrow-flight           14.0.2           h120cb0d_0_cpu    conda-forge
     libarrow-flight-sql       14.0.2           h61ff412_0_cpu    conda-forge
     libarrow-gandiva          14.0.2           hacb8726_0_cpu    conda-forge
     libarrow-substrait        14.0.2           h61ff412_0_cpu    conda-forge
     libavif16                 1.0.3                hef5bec9_1    conda-forge
     libblas                   3.9.0           20_linux64_openblas    conda-forge
     libboost-headers          1.84.0               ha770c72_0    conda-forge
     libbrotlicommon           1.1.0                hd590300_1    conda-forge
     libbrotlidec              1.1.0                hd590300_1    conda-forge
     libbrotlienc              1.1.0                hd590300_1    conda-forge
     libcblas                  3.9.0           20_linux64_openblas    conda-forge
     libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
     libcublas                 12.0.1.189           hd3aeb46_3    conda-forge
     libcublas-dev             12.0.1.189           hd3aeb46_3    conda-forge
     libcucim                  23.12.01        cuda12_231211_ga3445df_0    rapidsai
     libcudf                   23.12.01        cuda12_231208_g2ce46216b5_0    rapidsai
     libcudf_kafka             23.12.01        cuda12_231208_g2ce46216b5_0    rapidsai
     libcufft                  11.0.0.21            hd3aeb46_2    conda-forge
     libcufile                 1.5.0.59             hd3aeb46_1    conda-forge
     libcufile-dev             1.5.0.59             hd3aeb46_1    conda-forge
     libcugraph                23.12.00        cuda12_231206_g1309813f_0    rapidsai
     libcugraph_etl            23.12.00        cuda12_231206_g1309813f_0    rapidsai
     libcugraphops             23.12.00        cuda12_231206_g42d08202_0    nvidia
     libcuml                   23.12.00        cuda12_231206_gad2bd2b65_0    rapidsai
     libcumlprims              23.12.00        cuda12_231206_gc120fe0_0    nvidia
     libcurand                 10.3.1.50            hd3aeb46_1    conda-forge
     libcurand-dev             10.3.1.50            hd3aeb46_1    conda-forge
     libcurl                   8.5.0                hca28451_0    conda-forge
     libcusolver               11.4.2.57            hd3aeb46_2    conda-forge
     libcusolver-dev           11.4.2.57            hd3aeb46_2    conda-forge
     libcusparse               12.0.0.76            hd3aeb46_2    conda-forge
     libcusparse-dev           12.0.0.76            hd3aeb46_2    conda-forge
     libcuspatial              23.12.01        cuda12_231207_g16727064_0    rapidsai
     libdeflate                1.19                 hd590300_0    conda-forge
     libedit                   3.1.20191231         he28a2e2_2    conda-forge
     libev                     4.33                 hd590300_2    conda-forge
     libevent                  2.1.12               hf998b51_1    conda-forge
     libexpat                  2.5.0                hcb278e6_1    conda-forge
     libffi                    3.4.2                h7f98852_5    conda-forge
     libgcc-ng                 13.2.0               h807b86a_3    conda-forge
     libgdal                   3.8.2                hed8bd54_0    conda-forge
     libgfortran-ng            13.2.0               h69a702a_3    conda-forge
     libgfortran5              13.2.0               ha4646dd_3    conda-forge
     libglib                   2.78.3               h783c2da_0    conda-forge
     libgomp                   13.2.0               h807b86a_3    conda-forge
     libgoogle-cloud           2.12.0               h5206363_4    conda-forge
     libgrpc                   1.59.3               hd6c4280_0    conda-forge
     libiconv                  1.17                 hd590300_2    conda-forge
     libjpeg-turbo             3.0.0                hd590300_1    conda-forge
     libkml                    1.3.0             h01aab08_1018    conda-forge
     libkvikio                 23.12.00        cuda12_231206_gf90bfbe_0    rapidsai
     liblapack                 3.9.0           20_linux64_openblas    conda-forge
     libllvm14                 14.0.6               hcd5def8_4    conda-forge
     libllvm15                 15.0.7               hb3ce162_4    conda-forge
     libnetcdf                 4.9.2           nompi_h9612171_113    conda-forge
     libnghttp2                1.58.0               h47da74e_1    conda-forge
     libnl                     3.9.0                hd590300_0    conda-forge
     libnsl                    2.0.1                hd590300_0    conda-forge
     libntlm                   1.4               h7f98852_1002    conda-forge
     libnuma                   2.0.16               h0b41bf4_1    conda-forge
     libnvjitlink              12.0.76              hd3aeb46_2    conda-forge
     libnvjpeg                 12.0.0.28            h59595ed_1    conda-forge
     libopenblas               0.3.25          pthreads_h413a1c8_0    conda-forge
     libparquet                14.0.2           h352af49_0_cpu    conda-forge
     libpng                    1.6.39               h753d276_0    conda-forge
     libpq                     16.1                 h33b98f1_7    conda-forge
     libprotobuf               4.24.4               hf27288f_0    conda-forge
     libraft                   23.12.00        cuda12_231206_g9e2d6277_0    rapidsai
     libraft-headers           23.12.00        cuda12_231206_g9e2d6277_0    rapidsai
     libraft-headers-only      23.12.00        cuda12_231206_g9e2d6277_0    rapidsai
     librdkafka                1.9.2                ha5a0de0_2    conda-forge
     libre2-11                 2023.06.02           h7a70373_0    conda-forge
     librmm                    23.12.00        cuda12_231206_g2db5cbb3_0    rapidsai
     librttopo                 1.1.0               h8917695_15    conda-forge
     libsodium                 1.0.18               h36c2ea0_1    conda-forge
     libspatialindex           1.9.3                h9c3ff4c_4    conda-forge
     libspatialite             5.1.0                h7bd4643_4    conda-forge
     libsqlite                 3.44.2               h2797004_0    conda-forge
     libssh2                   1.11.0               h0841786_0    conda-forge
     libstdcxx-ng              13.2.0               h7e041cc_3    conda-forge
     libthrift                 0.19.0               hb90f79a_1    conda-forge
     libtiff                   4.6.0                ha9c0a0a_2    conda-forge
     libutf8proc               2.8.0                h166bdaf_0    conda-forge
     libuuid                   2.38.1               h0b41bf4_0    conda-forge
     libuv                     1.46.0               hd590300_0    conda-forge
     libwebp                   1.3.2                h658648e_1    conda-forge
     libwebp-base              1.3.2                hd590300_0    conda-forge
     libxcb                    1.15                 h0b41bf4_0    conda-forge
     libxgboost                1.7.6           rapidsai_h52ede06_7    rapidsai
     libxml2                   2.12.3               h232c23b_0    conda-forge
     libzip                    1.10.1               h2629f0a_3    conda-forge
     libzlib                   1.2.13               hd590300_5    conda-forge
     libzopfli                 1.0.3                h9c3ff4c_0    conda-forge
     lightgbm                  3.3.5                    pypi_0    pypi
     lightning-utilities       0.10.0                   pypi_0    pypi
     linkify-it-py             2.0.0              pyhd8ed1ab_0    conda-forge
     llvmlite                  0.41.1                   pypi_0    pypi
     locket                    1.0.0              pyhd8ed1ab_0    conda-forge
     lz4                       4.3.2            py39h79d96da_1    conda-forge
     lz4-c                     1.9.4                hcb278e6_0    conda-forge
     lzo                       2.10              h516909a_1000    conda-forge
     mako                      1.3.0                    pypi_0    pypi
     mapclassify               2.6.1              pyhd8ed1ab_0    conda-forge
     markdown                  3.5.1              pyhd8ed1ab_0    conda-forge
     markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
     markupsafe                2.1.3            py39hd1e30aa_1    conda-forge
     matplotlib-base           3.8.2            py39he9076e7_0    conda-forge
     mccabe                    0.7.0                    pypi_0    pypi
     mdit-py-plugins           0.4.0              pyhd8ed1ab_0    conda-forge
     mdurl                     0.1.0              pyhd8ed1ab_0    conda-forge
     metaml                    1.0.19rc0                pypi_0    pypi
     minizip                   4.0.3                h0ab5242_0    conda-forge
     mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
     more-itertools            10.1.0                   pypi_0    pypi
     mpmath                    1.3.0                    pypi_0    pypi
     mrmr-selection            0.2.8                    pypi_0    pypi
     msgpack-python            1.0.7            py39h7633fee_0    conda-forge
     multidict                 6.0.4            py39hd1e30aa_1    conda-forge
     multipledispatch          0.6.0                      py_0    conda-forge
     multiprocess              0.70.15                  pypi_0    pypi
     munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
     mypy                      1.4.1                    pypi_0    pypi
     mypy-extensions           1.0.0                    pypi_0    pypi
     nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
     nbconvert-core            7.13.1             pyhd8ed1ab_0    conda-forge
     nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
     nccl                      2.19.4.1             h3a97aeb_0    conda-forge
     ncurses                   6.4                  h59595ed_2    conda-forge
     networkx                  3.2                      pypi_0    pypi
     nfoursid                  1.0.1                    pypi_0    pypi
     nh3                       0.2.15                   pypi_0    pypi
     nodeenv                   1.8.0                    pypi_0    pypi
     nodejs                    20.9.0               hb753e55_0    conda-forge
     nspr                      4.35                 h27087fc_0    conda-forge
     nss                       3.96                 h1d7d5a4_0    conda-forge
     nuitka                    1.7.5                    pypi_0    pypi
     numba                     0.58.1                   pypi_0    pypi
     numpy                     1.24.4           py39h6183b62_0    conda-forge
     nvcomp                    3.0.4                h10b603f_1    conda-forge
     nvidia-cublas-cu12        12.1.3.1                 pypi_0    pypi
     nvidia-cuda-cupti-cu12    12.1.105                 pypi_0    pypi
     nvidia-cuda-nvrtc-cu12    12.1.105                 pypi_0    pypi
     nvidia-cuda-runtime-cu12  12.1.105                 pypi_0    pypi
     nvidia-cudnn-cu12         8.9.2.26                 pypi_0    pypi
     nvidia-cufft-cu12         11.0.2.54                pypi_0    pypi
     nvidia-curand-cu12        10.3.2.106               pypi_0    pypi
     nvidia-cusolver-cu12      11.4.5.107               pypi_0    pypi
     nvidia-cusparse-cu12      12.1.0.106               pypi_0    pypi
     nvidia-nccl-cu12          2.18.1                   pypi_0    pypi
     nvidia-nvjitlink-cu12     12.3.101                 pypi_0    pypi
     nvidia-nvtx-cu12          12.1.105                 pypi_0    pypi
     nvtx                      0.2.8            py39hd1e30aa_1    conda-forge
     openjpeg                  2.5.0                h488ebb8_3    conda-forge
     openslide                 3.4.1               h58ba908_12    conda-forge
     openssl                   3.2.0                hd590300_1    conda-forge
     optuna                    3.3.0                    pypi_0    pypi
     orc                       1.9.2                h4b38347_0    conda-forge
     ordered-set               4.1.0                    pypi_0    pypi
     overrides                 7.4.0              pyhd8ed1ab_0    conda-forge
     packaging                 23.2               pyhd8ed1ab_0    conda-forge
     pandas                    1.5.3            py39h2ad29b5_1    conda-forge
     pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
     panel                     1.3.4              pyhd8ed1ab_0    conda-forge
     param                     2.0.1              pyhca7485f_0    conda-forge
     partd                     1.4.1              pyhd8ed1ab_0    conda-forge
     pathspec                  0.12.1                   pypi_0    pypi
     patsy                     0.5.4                    pypi_0    pypi
     pcre2                     10.42                hcad00b1_0    conda-forge
     pillow                    10.1.0           py39had0adad_0    conda-forge
     pip                       23.3.2             pyhd8ed1ab_0    conda-forge
     pipdeptree                2.10.2                   pypi_0    pypi
     pixman                    0.42.2               h59595ed_0    conda-forge
     pkginfo                   1.9.6                    pypi_0    pypi
     pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
     platformdirs              4.1.0              pyhd8ed1ab_0    conda-forge
     plotly                    5.18.0                   pypi_0    pypi
     pluggy                    1.3.0                    pypi_0    pypi
     pmdarima                  2.0.4                    pypi_0    pypi
     polars                    0.20.2                   pypi_0    pypi
     poppler                   23.12.0              h590f24d_0    conda-forge
     poppler-data              0.4.12               hd8ed1ab_0    conda-forge
     portion                   2.4.2                    pypi_0    pypi
     postgresql                16.1                 h7387d8b_7    conda-forge
     pre-commit                3.3.3                    pypi_0    pypi
     proj                      9.3.1                h1d62c97_0    conda-forge
     prometheus_client         0.19.0             pyhd8ed1ab_0    conda-forge
     protobuf                  3.20.3                   pypi_0    pypi
     psutil                    5.9.7            py39hd1e30aa_0    conda-forge
     pthread-stubs             0.4               h36c2ea0_1001    conda-forge
     ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
     py-xgboost                1.7.6           rapidsai_py39h84d37f7_7    rapidsai
     pyarrow                   14.0.2          py39h6925388_0_cpu    conda-forge
     pyarrow-hotfix            0.6                pyhd8ed1ab_0    conda-forge
     pycodestyle               2.10.0                   pypi_0    pypi
     pycparser                 2.21               pyhd8ed1ab_0    conda-forge
     pyct                      0.5.0              pyhd8ed1ab_0    conda-forge
     pydantic                  1.10.13                  pypi_0    pypi
     pyee                      8.1.0              pyhd8ed1ab_0    conda-forge
     pyflakes                  3.0.1                    pypi_0    pypi
     pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
     pylibcugraph              23.12.00        cuda12_py39_231206_g1309813f_0    rapidsai
     pylibraft                 23.12.00        cuda12_py39_231206_g9e2d6277_0    rapidsai
     pylint                    2.17.4                   pypi_0    pypi
     pynndescent               0.5.11                   pypi_0    pypi
     pynvml                    11.4.1             pyhd8ed1ab_0    conda-forge
     pyod                      1.1.2                    pypi_0    pypi
     pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
     pyppeteer                 1.0.2              pyhd8ed1ab_0    conda-forge
     pyproj                    3.6.1            py39h15b0fa6_5    conda-forge
     pysocks                   1.7.1              pyha2e5f31_6    conda-forge
     pytest                    7.4.0                    pypi_0    pypi
     pytest-cov                4.1.0                    pypi_0    pypi
     pytest-split              0.8.1                    pypi_0    pypi
     pytest-xdist              3.3.1                    pypi_0    pypi
     python                    3.9.18          h0755675_0_cpython    conda-forge
     python-confluent-kafka    1.9.2            py39hb9d737c_2    conda-forge
     python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
     python-fastjsonschema     2.19.0             pyhd8ed1ab_0    conda-forge
     python-graphviz           0.20.1                   pypi_0    pypi
     python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
     python_abi                3.9                      4_cp39    conda-forge
     pytorch-lightning         2.1.3                    pypi_0    pypi
     pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
     pyviz_comms               3.0.0              pyhd8ed1ab_0    conda-forge
     pywavelets                1.4.1            py39h44dd56e_1    conda-forge
     pyyaml                    6.0.1            py39hd1e30aa_1    conda-forge
     pyzmq                     25.1.2           py39h8c080ef_0    conda-forge
     qpd                       0.4.4                    pypi_0    pypi
     raft-dask                 23.12.00        cuda12_py39_231206_g9e2d6277_0    rapidsai
     rapids                    23.12.00        cuda12_py39_231206_g1d8bed4_0    rapidsai
     rapids-dask-dependency    23.12.01                      0    rapidsai
     rapids-xgboost            23.12.00        cuda12_py39_231206_g1d8bed4_0    rapidsai
     rav1e                     0.6.6                he8a937b_2    conda-forge
     rdma-core                 49.0                 hd3aeb46_2    conda-forge
     re2                       2023.06.02           h2873b5e_0    conda-forge
     readline                  8.2                  h8228510_1    conda-forge
     readme-renderer           42.0                     pypi_0    pypi
     referencing               0.32.0             pyhd8ed1ab_0    conda-forge
     regex                     2023.10.3                pypi_0    pypi
     requests                  2.31.0             pyhd8ed1ab_0    conda-forge
     requests-toolbelt         1.0.0                    pypi_0    pypi
     rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
     rfc3986                   2.0.0                    pypi_0    pypi
     rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
     rich                      13.7.0             pyhd8ed1ab_0    conda-forge
     rmm                       23.12.00        cuda12_py39_231206_g2db5cbb3_0    rapidsai
     rpds-py                   0.15.2           py39h9fdd4d6_0    conda-forge
     rtree                     1.1.0            py39hb102c33_0    conda-forge
     ruamel-yaml               0.18.5                   pypi_0    pypi
     ruamel-yaml-clib          0.2.8                    pypi_0    pypi
     s2n                       1.4.0                h06160fa_0    conda-forge
     scikit-image              0.21.0           py39h3d6467e_0    conda-forge
     scikit-learn              1.1.3                    pypi_0    pypi
     scikit-learn-intelex      2023.2.0                 pypi_0    pypi
     scipy                     1.11.4           py39h474f0d3_0    conda-forge
     secretstorage             3.3.3                    pypi_0    pypi
     send2trash                1.8.2              pyh41d4057_0    conda-forge
     setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
     shap                      0.44.0                   pypi_0    pypi
     shapely                   2.0.2            py39h6404dd3_1    conda-forge
     simpervisor               1.0.0              pyhd8ed1ab_0    conda-forge
     six                       1.16.0             pyh6c4a22f_0    conda-forge
     sklearn-contrib-lightning 0.6.2.post0              pypi_0    pypi
     sktime                    0.17.1                   pypi_0    pypi
     slicer                    0.0.7                    pypi_0    pypi
     snappy                    1.1.10               h9fff704_0    conda-forge
     sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
     snowballstemmer           2.2.0                    pypi_0    pypi
     sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
     soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
     spdlog                    1.11.0               h9b3ece8_1    conda-forge
     sphinx                    6.2.1                    pypi_0    pypi
     sphinx-rtd-theme          1.2.2                    pypi_0    pypi
     sphinxcontrib-applehelp   1.0.7                    pypi_0    pypi
     sphinxcontrib-devhelp     1.0.5                    pypi_0    pypi
     sphinxcontrib-htmlhelp    2.0.4                    pypi_0    pypi
     sphinxcontrib-jquery      4.1                      pypi_0    pypi
     sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
     sphinxcontrib-qthelp      1.0.6                    pypi_0    pypi
     sphinxcontrib-serializinghtml 1.1.9                    pypi_0    pypi
     sqlalchemy                2.0.23                   pypi_0    pypi
     sqlglot                   20.4.0                   pypi_0    pypi
     sqlite                    3.44.2               h2c6b66d_0    conda-forge
     statsforecast             1.7.0                    pypi_0    pypi
     statsmodels               0.14.1                   pypi_0    pypi
     streamz                   0.6.4              pyh6c4a22f_0    conda-forge
     svt-av1                   1.8.0                h59595ed_0    conda-forge
     sympy                     1.12                     pypi_0    pypi
     tbats                     1.1.3                    pypi_0    pypi
     tbb                       2021.11.0                pypi_0    pypi
     tblib                     3.0.0              pyhd8ed1ab_0    conda-forge
     tenacity                  8.2.3                    pypi_0    pypi
     tensorboardx              2.6                      pypi_0    pypi
     terminado                 0.18.0             pyh0d859eb_0    conda-forge
     threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
     tifffile                  2023.12.9          pyhd8ed1ab_0    conda-forge
     tiledb                    2.18.3               hc1131af_1    conda-forge
     tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
     tk                        8.6.13          noxft_h4845f30_101    conda-forge
     tokenizers                0.13.3                   pypi_0    pypi
     tomli                     2.0.1                    pypi_0    pypi
     tomlkit                   0.12.3                   pypi_0    pypi
     toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
     torch                     2.1.2                    pypi_0    pypi
     torchmetrics              1.2.1                    pypi_0    pypi
     tornado                   6.3.3            py39hd1e30aa_1    conda-forge
     tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
     traitlets                 5.14.0             pyhd8ed1ab_0    conda-forge
     transformers              4.28.1                   pypi_0    pypi
     treelite                  3.9.1            py39h9b5fa3e_0    conda-forge
     treelite-runtime          3.9.1                    pypi_0    pypi
     triad                     0.9.3                    pypi_0    pypi
     triton                    2.1.0                    pypi_0    pypi
     twine                     4.0.2                    pypi_0    pypi
     types-python-dateutil     2.8.19.14          pyhd8ed1ab_0    conda-forge
     typing-extensions         4.9.0                hd8ed1ab_0    conda-forge
     typing_extensions         4.9.0              pyha770c72_0    conda-forge
     typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
     tzcode                    2023c                h0b41bf4_0    conda-forge
     tzdata                    2023c                h71feb2d_0    conda-forge
     uc-micro-py               1.0.1              pyhd8ed1ab_0    conda-forge
     ucx                       1.15.0               h6d2d1ec_2    conda-forge
     ucx-proc                  1.0.0                       gpu    rapidsai
     ucx-py                    0.35.00         py39_231206_gb5f60ca_0    rapidsai
     umap-learn                0.5.5                    pypi_0    pypi
     unicodedata2              15.1.0           py39hd1e30aa_0    conda-forge
     uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
     uriparser                 0.9.7                hcb278e6_1    conda-forge
     urllib3                   1.26.18            pyhd8ed1ab_0    conda-forge
     utilsforecast             0.0.23                   pypi_0    pypi
     vecstack                  0.4.0                    pypi_0    pypi
     virtualenv                20.25.0                  pypi_0    pypi
     webcolors                 1.13               pyhd8ed1ab_0    conda-forge
     webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
     websocket-client          1.7.0              pyhd8ed1ab_0    conda-forge
     websockets                10.4             py39hb9d737c_1    conda-forge
     wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
     wrapt                     1.16.0                   pypi_0    pypi
     xarray                    2023.12.0          pyhd8ed1ab_0    conda-forge
     xerces-c                  3.2.5                hac6953d_0    conda-forge
     xgboost                   1.7.6           rapidsai_py39h0b6c2bb_7    rapidsai
     xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
     xorg-libice               1.1.1                hd590300_0    conda-forge
     xorg-libsm                1.2.4                h7391055_0    conda-forge
     xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
     xorg-libxau               1.0.11               hd590300_0    conda-forge
     xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
     xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
     xorg-libxrender           0.9.11               hd590300_0    conda-forge
     xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
     xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
     xorg-xproto               7.0.31            h7f98852_1007    conda-forge
     xxhash                    3.4.1                    pypi_0    pypi
     xyzservices               2023.10.1          pyhd8ed1ab_0    conda-forge
     xz                        5.2.6                h166bdaf_0    conda-forge
     yaml                      0.2.5                h7f98852_2    conda-forge
     yarl                      1.9.3            py39hd1e30aa_0    conda-forge
     zeromq                    4.3.5                h59595ed_0    conda-forge
     zfp                       1.0.1                h59595ed_0    conda-forge
     zict                      3.0.0              pyhd8ed1ab_0    conda-forge
     zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
     zlib                      1.2.13               hd590300_5    conda-forge
     zlib-ng                   2.0.7                h0b41bf4_0    conda-forge
     zstandard                 0.22.0                   pypi_0    pypi
     zstd                      1.5.5                hfc55251_0    conda-forge

davidwendt commented 9 months ago

Reference https://github.com/rapidsai/cudf/pull/14693

paulsbrookes commented 9 months ago

I think reference https://github.com/rapidsai/cudf/pull/14693 is more about the difficulty in unpickling objects and the fact that you need to use a consistent version of pandas whereas this bug is about the difficulty of pickling a cudf.pandas object to begin with.

I suspect this bug has a similar cause and solution to this one: https://github.com/rapidsai/cudf/issues/14674, but if there isn't a solution then some explanation could be added to https://github.com/rapidsai/cudf/blob/1078326535c9989a2e904d78ceb708a097be989b/docs/cudf/source/cudf_pandas/faq.md explaining that currently not all cudf.pandas objects can be pickled.

shwina commented 9 months ago

Thanks! This one is a legitimate bug and one that we can fix. I'll open a PR soon that addresses this.

paulsbrookes commented 9 months ago

Thanks @shwina!