rapidsai / cudf

cuDF - GPU DataFrame Library
https://docs.rapids.ai/api/cudf/stable/
Apache License 2.0
8.23k stars 884 forks source link

[BUG] cudf.DataFrame.equals reports False for inputs with NaNs #12266

Open csadorf opened 1 year ago

csadorf commented 1 year ago

Describe the bug

According to the documentation, the result of DataFrame.equals() should be True when the two dataframes are equal, even when they contain NaNs:

NaNs in the same location are considered equal.

However, this does not seem to be the case under certain circumstances.

Steps/Code to reproduce bug

The following snippet demonstrates the unexpected behavior:

import numpy as np
import cudf

input_array = np.array([np.nan])

a = cudf.DataFrame(input_array)
assert a.equals(a.copy())

will fail with an AssertionError.

Curiously the following snippet does not fail:

a = cudf.DataFrame([np.nan])
assert a.equals(a.copy())

Expected behavior

The result of DataFrame.equals() should be True in case that values are the same and NaNs are in the same place as described by the method's documentation.

Environment overview (please complete the following information)

Environment details

Click here to see environment details

     **git***
     Not inside a git repository

     ***OS Information***
     DGX_NAME="DGX Server"
     DGX_PRETTY_NAME="NVIDIA DGX Server"
     DGX_SWBUILD_DATE="2020-03-04"
     DGX_SWBUILD_VERSION="4.4.0"
     DGX_COMMIT_ID="ee09ebc"
     DGX_PLATFORM="DGX Server for DGX-1"
     DGX_SERIAL_NUMBER="QTFCOU6300082"
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=18.04
     DISTRIB_CODENAME=bionic
     DISTRIB_DESCRIPTION="Ubuntu 18.04.4 LTS"
     NAME="Ubuntu"
     VERSION="18.04.4 LTS (Bionic Beaver)"
     ID=ubuntu
     ID_LIKE=debian
     PRETTY_NAME="Ubuntu 18.04.4 LTS"
     VERSION_ID="18.04"
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     VERSION_CODENAME=bionic
     UBUNTU_CODENAME=bionic
     Linux dgx06 4.15.0-76-generic #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Wed Nov 30 05:42:50 2022
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 495.29.05    Driver Version: 495.29.05    CUDA Version: 11.5     |
     |-------------------------------+----------------------+----------------------+
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  Tesla V100-SXM2...  On   | 00000000:06:00.0 Off |                    0 |
     | N/A   32C    P0    55W / 300W |    668MiB / 32510MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   1  Tesla V100-SXM2...  On   | 00000000:07:00.0 Off |                    0 |
     | N/A   32C    P0    42W / 300W |      3MiB / 32510MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   2  Tesla V100-SXM2...  On   | 00000000:0A:00.0 Off |                    0 |
     | N/A   30C    P0    42W / 300W |      3MiB / 32510MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   3  Tesla V100-SXM2...  On   | 00000000:0B:00.0 Off |                    0 |
     | N/A   29C    P0    42W / 300W |      3MiB / 32510MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   4  Tesla V100-SXM2...  On   | 00000000:85:00.0 Off |                    0 |
     | N/A   30C    P0    42W / 300W |      3MiB / 32510MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   5  Tesla V100-SXM2...  On   | 00000000:86:00.0 Off |                    0 |
     | N/A   30C    P0    42W / 300W |      3MiB / 32510MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   6  Tesla V100-SXM2...  On   | 00000000:89:00.0 Off |                    0 |
     | N/A   31C    P0    42W / 300W |      3MiB / 32510MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+
     |   7  Tesla V100-SXM2...  On   | 00000000:8A:00.0 Off |                    0 |
     | N/A   29C    P0    41W / 300W |      3MiB / 32510MiB |      0%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+

     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     |    0   N/A  N/A    630188      C   python                            665MiB |
     +-----------------------------------------------------------------------------+

     ***CPU***
     Architecture:        x86_64
     CPU op-mode(s):      32-bit, 64-bit
     Byte Order:          Little Endian
     CPU(s):              80
     On-line CPU(s) list: 0-79
     Thread(s) per core:  2
     Core(s) per socket:  20
     Socket(s):           2
     NUMA node(s):        2
     Vendor ID:           GenuineIntel
     CPU family:          6
     Model:               79
     Model name:          Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz
     Stepping:            1
     CPU MHz:             3226.018
     CPU max MHz:         3600.0000
     CPU min MHz:         1200.0000
     BogoMIPS:            4390.02
     Virtualization:      VT-x
     L1d cache:           32K
     L1i cache:           32K
     L2 cache:            256K
     L3 cache:            51200K
     NUMA node0 CPU(s):   0-19,40-59
     NUMA node1 CPU(s):   20-39,60-79
     Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts md_clear flush_l1d

     ***CMake***
     /raid/sadorf/mambaforge/envs/cuml_dev/bin/cmake
     cmake version 3.24.3

     CMake suite maintained and supported by Kitware (kitware.com/cmake).

     ***g++***
     /raid/sadorf/mambaforge/envs/cuml_dev/bin/g++
     g++ (conda-forge gcc 9.5.0-19) 9.5.0
     Copyright (C) 2019 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

     ***nvcc***
     /usr/local/cuda/bin/nvcc
     nvcc: NVIDIA (R) Cuda compiler driver
     Copyright (c) 2005-2021 NVIDIA Corporation
     Built on Thu_Nov_18_09:45:30_PST_2021
     Cuda compilation tools, release 11.5, V11.5.119
     Build cuda_11.5.r11.5/compiler.30672275_0

     ***Python***
     /raid/sadorf/mambaforge/envs/cuml_dev/bin/python
     Python 3.8.13

     ***Environment Variables***
     PATH                            : /home/nfs/sadorf/.vscode-server/bin/6261075646f055b99068d3688932416f2346dd3b/bin/remote-cli:/raid/sadorf/mambaforge/envs/cuml_dev/bin:/raid/sadorf/mambaforge/condabin:/usr/local/cuda/bin:/opt/bin:/usr/local/cuda/bin:/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
     LD_LIBRARY_PATH                 :
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    : /raid/sadorf/mambaforge/envs/cuml_dev
     PYTHON_PATH                     :

     ***conda packages***
     /raid/sadorf/mambaforge/envs/cuml_dev/bin/conda
     # packages in environment at /raid/sadorf/mambaforge/envs/cuml_dev:
     #
     # Name                    Version                   Build  Channel
     _libgcc_mutex             0.1                 conda_forge    conda-forge
     _openmp_mutex             4.5                       2_gnu    conda-forge
     _sysroot_linux-64_curr_repodata_hack 3                   h5bd9786_13    conda-forge
     abseil-cpp                20210324.2           h9c3ff4c_0    conda-forge
     aiobotocore               2.4.0              pyhd8ed1ab_0    conda-forge
     aiohttp                   3.8.3            py38h0a891b7_1    conda-forge
     aioitertools              0.11.0             pyhd8ed1ab_0    conda-forge
     aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
     alabaster                 0.7.12                     py_0    conda-forge
     anyio                     3.6.2              pyhd8ed1ab_0    conda-forge
     aom                       3.5.0                h27087fc_0    conda-forge
     appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
     argon2-cffi               21.3.0             pyhd8ed1ab_0    conda-forge
     argon2-cffi-bindings      21.2.0           py38h0a891b7_3    conda-forge
     arrow-cpp                 9.0.0           py38ha7276ea_1_cpu    conda-forge
     astroid                   2.12.13          py38h578d9bd_0    conda-forge
     asvdb                     0.4.2               g90e8f2c_40    rapidsai
     async-timeout             4.0.2              pyhd8ed1ab_0    conda-forge
     atk-1.0                   2.38.0               hd4edc92_1    conda-forge
     attrs                     22.1.0             pyh71513ae_1    conda-forge
     autoconf                  2.69            pl5321hd708f79_11    conda-forge
     automake                  1.16.5          pl5321ha770c72_0    conda-forge
     aws-c-cal                 0.5.11               h95a6274_0    conda-forge
     aws-c-common              0.6.2                h7f98852_0    conda-forge
     aws-c-event-stream        0.2.7               h3541f99_13    conda-forge
     aws-c-io                  0.10.5               hfb6a706_0    conda-forge
     aws-checksums             0.1.11               ha31a3da_7    conda-forge
     aws-sam-translator        1.53.0             pyhd8ed1ab_0    conda-forge
     aws-sdk-cpp               1.8.186              hecaee15_4    conda-forge
     aws-xray-sdk              2.10.0             pyhd8ed1ab_0    conda-forge
     babel                     2.11.0             pyhd8ed1ab_0    conda-forge
     backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
     backports                 1.0                        py_2    conda-forge
     backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
     backports.zoneinfo        0.2.1            py38h0a891b7_7    conda-forge
     bcrypt                    3.2.2            py38h0a891b7_1    conda-forge
     beautifulsoup4            4.11.1             pyha770c72_0    conda-forge
     benchmark                 1.5.1                he1b5a44_2    conda-forge
     binutils                  2.39                 hdd6e379_0    conda-forge
     binutils_impl_linux-64    2.39                 h6ceecb4_0    conda-forge
     binutils_linux-64         2.39                h5fc0e48_11    conda-forge
     black                     22.3.0             pyhd8ed1ab_0    conda-forge
     blas                      1.1                    openblas    conda-forge
     bleach                    5.0.1              pyhd8ed1ab_0    conda-forge
     blinker                   1.5                pyhd8ed1ab_0    conda-forge
     blosc                     1.21.1               h83bc5f7_3    conda-forge
     bokeh                     2.4.3              pyhd8ed1ab_3    conda-forge
     boost                     1.74.0           py38h2b96118_5    conda-forge
     boost-cpp                 1.74.0               h75c5d50_8    conda-forge
     boto3                     1.24.59            pyhd8ed1ab_0    conda-forge
     botocore                  1.27.59            pyhd8ed1ab_0    conda-forge
     branca                    0.6.0              pyhd8ed1ab_0    conda-forge
     breathe                   4.34.0             pyhd8ed1ab_0    conda-forge
     brotli                    1.0.9                h166bdaf_8    conda-forge
     brotli-bin                1.0.9                h166bdaf_8    conda-forge
     brotlipy                  0.7.0           py38h0a891b7_1005    conda-forge
     brunsli                   0.1                  h9c3ff4c_0    conda-forge
     bzip2                     1.0.8                h7f98852_4    conda-forge
     c-ares                    1.18.1               h7f98852_0    conda-forge
     c-blosc2                  2.4.3                h7a311fb_0    conda-forge
     c-compiler                1.3.0                h7f98852_0    conda-forge
     ca-certificates           2022.9.24            ha878542_0    conda-forge
     cached-property           1.5.2                hd8ed1ab_1    conda-forge
     cached_property           1.5.2              pyha770c72_1    conda-forge
     cachetools                5.2.0              pyhd8ed1ab_0    conda-forge
     cairo                     1.16.0            ha61ee94_1014    conda-forge
     ccache                    4.7.3                h2599c5e_0    conda-forge
     certifi                   2022.9.24          pyhd8ed1ab_0    conda-forge
     cffi                      1.15.1           py38h4a40e3a_2    conda-forge
     cfgv                      3.3.1              pyhd8ed1ab_0    conda-forge
     cfitsio                   4.1.0                hd9d235c_0    conda-forge
     cfn-lint                  0.71.0             pyhd8ed1ab_0    conda-forge
     chardet                   5.0.0            py38h578d9bd_1    conda-forge
     charls                    2.3.4                h9c3ff4c_0    conda-forge
     charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
     clang                     11.1.0               ha770c72_1    conda-forge
     clang-11                  11.1.0          default_ha53f305_1    conda-forge
     clang-tools               11.1.0          default_ha53f305_1    conda-forge
     clangxx                   11.1.0          default_ha53f305_1    conda-forge
     click                     8.1.3           unix_pyhd8ed1ab_2    conda-forge
     click-plugins             1.1.1                      py_0    conda-forge
     cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
     cloudpickle               2.2.0              pyhd8ed1ab_0    conda-forge
     cmake                     3.24.3               h816a3e0_0    conda-forge
     cmake_setuptools          0.1.3                      py_0    rapidsai
     cmakelang                 0.6.13             pyhd8ed1ab_0    conda-forge
     cmarkgfm                  0.8.0            py38h0a891b7_2    conda-forge
     colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
     colorcet                  3.0.1              pyhd8ed1ab_0    conda-forge
     commonmark                0.9.1                      py_0    conda-forge
     conda                     4.12.0           py38h578d9bd_0    conda-forge
     conda-build               3.22.0           py38h578d9bd_3    conda-forge
     conda-package-handling    1.9.0            py38h0a891b7_1    conda-forge
     conda-verify              3.1.1           py38h578d9bd_1006    conda-forge
     contourpy                 1.0.6            py38h43d8883_0    conda-forge
     coverage                  6.5.0            py38h0a891b7_1    conda-forge
     cryptography              38.0.3           py38h2b5fc30_0    conda-forge
     cubinlinker               0.2.0            py38h7144610_1    rapidsai
     cuda-python               11.7.1           py38h7525318_1    conda-forge
     cudatoolkit               11.5.1              h59c8dcf_11    conda-forge
     cudf                      23.02.00a221130 cuda_11_py38_g5f83a84916_111    rapidsai-nightly
     cuml                      22.12.0a0+56.gdd6c9d0e6          pypi_0    pypi
     cupy                      11.2.0           py38h405e1b6_0    conda-forge
     curl                      7.86.0               h7bff187_1    conda-forge
     cxx-compiler              1.3.0                h4bd325d_0    conda-forge
     cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
     cyrus-sasl                2.1.27               h230043b_5    conda-forge
     cython                    0.29.32          py38hfa26641_1    conda-forge
     cytoolz                   0.12.0           py38h0a891b7_1    conda-forge
     dask                      2022.11.2a221129 py_gc23ee621_16    dask/label/dev
     dask-core                 2022.11.2a221125 py_g3ac3b8d6e_9    dask/label/dev
     dask-cuda                 23.02.00a221130 py38_gb087770_16    rapidsai-nightly
     dask-cudf                 23.02.00a221130 cuda_11_py38_g5f83a84916_111    rapidsai-nightly
     dask-glm                  0.2.1.dev52+g1daf4c5          pypi_0    pypi
     dask-labextension         6.0.0              pyhd8ed1ab_0    conda-forge
     dask-ml                   1.9.0              pyhd8ed1ab_0    conda-forge
     dataclasses               0.8                pyhc8e2a94_3    conda-forge
     datashader                0.13.1a                    py_0    rapidsai
     datashape                 0.5.4                      py_1    conda-forge
     dav1d                     1.0.0                h166bdaf_1    conda-forge
     dbus                      1.13.6               h5008d03_3    conda-forge
     debugpy                   1.6.3            py38hfa26641_1    conda-forge
     decopatch                 1.4.10             pyhd8ed1ab_0    conda-forge
     decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
     defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
     dill                      0.3.6              pyhd8ed1ab_1    conda-forge
     distlib                   0.3.5              pyhd8ed1ab_0    conda-forge
     distributed               2022.11.2a221129 py_gc23ee621_16    dask/label/dev
     distro                    1.6.0              pyhd8ed1ab_0    conda-forge
     dlpack                    0.5                  h9c3ff4c_0    conda-forge
     docker-py                 6.0.0              pyhd8ed1ab_0    conda-forge
     docutils                  0.19             py38h578d9bd_1    conda-forge
     double-conversion         3.1.5                h9c3ff4c_2    conda-forge
     doxygen                   1.8.20               had0d8f1_0    conda-forge
     ecdsa                     0.18.0             pyhd8ed1ab_1    conda-forge
     entrypoints               0.4                pyhd8ed1ab_0    conda-forge
     exceptiongroup            1.0.1              pyhd8ed1ab_0    conda-forge
     execnet                   1.9.0              pyhd8ed1ab_0    conda-forge
     expat                     2.5.0                h27087fc_0    conda-forge
     faiss-proc                1.0.0                      cuda    rapidsai
     fastavro                  1.7.0            py38h0a891b7_0    conda-forge
     fastrlock                 0.8              py38hfa26641_3    conda-forge
     feather-format            0.4.1              pyh9f0ad1d_0    conda-forge
     filelock                  3.8.0              pyhd8ed1ab_0    conda-forge
     filterpy                  1.4.5                      py_1    conda-forge
     fiona                     1.8.22           py38hc72d8cd_0    conda-forge
     flake8                    3.8.4                      py_0    conda-forge
     flask                     2.1.3              pyhd8ed1ab_0    conda-forge
     flask_cors                3.0.10             pyhd3deb0d_0    conda-forge
     flit-core                 3.8.0              pyhd8ed1ab_0    conda-forge
     folium                    0.13.0             pyhd8ed1ab_0    conda-forge
     font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
     font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
     font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
     font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
     fontconfig                2.14.1               hc2a2eb6_0    conda-forge
     fonts-conda-ecosystem     1                             0    conda-forge
     fonts-conda-forge         1                             0    conda-forge
     fonttools                 4.38.0           py38h0a891b7_1    conda-forge
     freetype                  2.12.1               hca18f0e_0    conda-forge
     freexl                    1.0.6                h166bdaf_1    conda-forge
     fribidi                   1.0.10               h36c2ea0_0    conda-forge
     frozenlist                1.3.3            py38h0a891b7_0    conda-forge
     fsspec                    2022.10.0          pyhd8ed1ab_0    conda-forge
     future                    0.18.2             pyhd8ed1ab_6    conda-forge
     gcc                       9.5.0               h1fea6ba_11    conda-forge
     gcc_impl_linux-64         9.5.0               h99780fb_19    conda-forge
     gcc_linux-64              9.5.0               h4258300_11    conda-forge
     gcsfs                     2022.10.0          pyhd8ed1ab_0    conda-forge
     gdal                      3.5.2            py38h1f15b03_3    conda-forge
     gdk-pixbuf                2.42.8               hff1cb4f_1    conda-forge
     geopandas                 0.12.1             pyhd8ed1ab_1    conda-forge
     geopandas-base            0.12.1             pyha770c72_1    conda-forge
     geos                      3.11.0               h27087fc_0    conda-forge
     geotiff                   1.7.1                h4fc65e6_3    conda-forge
     gettext                   0.21.1               h27087fc_0    conda-forge
     gflags                    2.2.2             he1b5a44_1004    conda-forge
     giflib                    5.2.1                h36c2ea0_2    conda-forge
     git                       2.38.1          pl5321h5e804b7_1    conda-forge
     glib                      2.74.1               h6239696_1    conda-forge
     glib-tools                2.74.1               h6239696_1    conda-forge
     glob2                     0.7                        py_0    conda-forge
     glog                      0.6.0                h6f12383_0    conda-forge
     gmock                     1.10.0               h4bd325d_7    conda-forge
     gmp                       6.2.1                h58526e2_0    conda-forge
     gmpy2                     2.1.2            py38h793c122_1    conda-forge
     google-api-core           2.10.1             pyhd8ed1ab_0    conda-forge
     google-auth               2.14.0             pyh1a96a4e_0    conda-forge
     google-auth-oauthlib      0.7.1              pyhd8ed1ab_0    conda-forge
     google-cloud-core         2.3.2              pyhd8ed1ab_0    conda-forge
     google-cloud-storage      2.6.0              pyh1a96a4e_0    conda-forge
     google-crc32c             1.1.2            py38h57c428a_4    conda-forge
     google-resumable-media    2.4.0              pyhd8ed1ab_0    conda-forge
     googleapis-common-protos  1.56.4           py38h578d9bd_1    conda-forge
     graphite2                 1.3.13            h58526e2_1001    conda-forge
     graphql-core              3.2.3              pyhd8ed1ab_0    conda-forge
     graphviz                  6.0.1                h5abf519_0    conda-forge
     greenlet                  2.0.1            py38hfa26641_0    conda-forge
     grpc-cpp                  1.45.2               h9d3bbbb_5    conda-forge
     grpcio                    1.46.3           py38ha0cdfde_0    conda-forge
     gtest                     1.10.0               h4bd325d_7    conda-forge
     gtk2                      2.24.33              h90689f9_2    conda-forge
     gts                       0.7.6                h64030ff_2    conda-forge
     gxx                       9.5.0               h1fea6ba_11    conda-forge
     gxx_impl_linux-64         9.5.0               h99780fb_19    conda-forge
     gxx_linux-64              9.5.0               h43f449f_11    conda-forge
     h5py                      3.7.0           nompi_py38h7927eab_102    conda-forge
     harfbuzz                  5.3.0                h418a68e_0    conda-forge
     hdbscan                   0.8.29           py38h26c90d9_1    conda-forge
     hdf4                      4.2.15               h9772cbc_5    conda-forge
     hdf5                      1.12.2          nompi_h2386368_100    conda-forge
     heapdict                  1.0.1                      py_0    conda-forge
     holoviews                 1.14.6             pyhd8ed1ab_0    conda-forge
     html5lib                  1.1                pyh9f0ad1d_0    conda-forge
     httpretty                 1.1.4              pyhd8ed1ab_0    conda-forge
     huggingface_hub           0.10.1             pyhd8ed1ab_0    conda-forge
     hypothesis                6.56.4             pyha770c72_0    conda-forge
     icu                       70.1                 h27087fc_0    conda-forge
     identify                  2.5.8              pyhd8ed1ab_0    conda-forge
     idna                      3.4                pyhd8ed1ab_0    conda-forge
     imagecodecs               2022.9.26        py38h839e5d1_3    conda-forge
     imageio                   2.22.0             pyhfa7a67d_0    conda-forge
     imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
     importlib-metadata        5.0.0              pyha770c72_1    conda-forge
     importlib_metadata        5.0.0                hd8ed1ab_1    conda-forge
     importlib_resources       3.3.1              pyhd8ed1ab_1    conda-forge
     iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
     ipykernel                 6.17.0             pyh210e3f2_0    conda-forge
     ipython                   7.31.1           py38h578d9bd_0    conda-forge
     ipython_genutils          0.2.0                      py_1    conda-forge
     ipywidgets                8.0.2              pyhd8ed1ab_1    conda-forge
     isort                     5.10.1             pyhd8ed1ab_0    conda-forge
     itsdangerous              2.1.2              pyhd8ed1ab_0    conda-forge
     jaraco.classes            3.2.3              pyhd8ed1ab_0    conda-forge
     jedi                      0.18.1             pyhd8ed1ab_2    conda-forge
     jeepney                   0.8.0              pyhd8ed1ab_0    conda-forge
     jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
     jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
     joblib                    1.2.0              pyhd8ed1ab_0    conda-forge
     jpeg                      9e                   h166bdaf_2    conda-forge
     jschema-to-python         1.2.3              pyhd8ed1ab_0    conda-forge
     json-c                    0.16                 hc379101_0    conda-forge
     json5                     0.9.5              pyh9f0ad1d_0    conda-forge
     jsondiff                  2.0.0              pyhd8ed1ab_0    conda-forge
     jsonpatch                 1.32               pyhd8ed1ab_0    conda-forge
     jsonpickle                2.2.0              pyhd8ed1ab_0    conda-forge
     jsonpointer               2.0                        py_0    conda-forge
     jsonschema                3.2.0              pyhd8ed1ab_3    conda-forge
     junit-xml                 1.9                pyh9f0ad1d_0    conda-forge
     jupyter-cache             0.5.0              pyhd8ed1ab_0    conda-forge
     jupyter-packaging         0.7.12             pyhd8ed1ab_0    conda-forge
     jupyter-server-proxy      3.2.2              pyhd8ed1ab_0    conda-forge
     jupyter_client            7.3.4              pyhd8ed1ab_0    conda-forge
     jupyter_core              4.11.2           py38h578d9bd_0    conda-forge
     jupyter_server            1.23.0             pyhd8ed1ab_0    conda-forge
     jupyter_sphinx            0.4.0            py38h578d9bd_1    conda-forge
     jupyterlab                3.5.0              pyhd8ed1ab_0    conda-forge
     jupyterlab-favorites      3.0.0              pyhd8ed1ab_0    conda-forge
     jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
     jupyterlab_server         2.16.2             pyhd8ed1ab_0    conda-forge
     jupyterlab_widgets        3.0.3              pyhd8ed1ab_0    conda-forge
     jxrlib                    1.1                  h7f98852_2    conda-forge
     kealib                    1.4.15               ha7026e8_1    conda-forge
     kernel-headers_linux-64   3.10.0              h4a8ded7_13    conda-forge
     keyring                   23.11.0          py38h578d9bd_0    conda-forge
     keyutils                  1.6.1                h166bdaf_0    conda-forge
     kiwisolver                1.4.4            py38h43d8883_1    conda-forge
     krb5                      1.19.3               h3790be6_0    conda-forge
     lapack                    3.9.0                    netlib    conda-forge
     lazy-object-proxy         1.8.0            py38h0a891b7_0    conda-forge
     lcms2                     2.14                 h6ed2654_0    conda-forge
     ld_impl_linux-64          2.39                 hc81fddc_0    conda-forge
     lerc                      4.0.0                h27087fc_0    conda-forge
     libaec                    1.0.6                h9c3ff4c_0    conda-forge
     libarchive                3.6.1                hc6fc967_0    conda-forge
     libavif                   0.11.1               h5cdd6b5_0    conda-forge
     libblas                   3.9.0           16_linux64_openblas    conda-forge
     libbrotlicommon           1.0.9                h166bdaf_8    conda-forge
     libbrotlidec              1.0.9                h166bdaf_8    conda-forge
     libbrotlienc              1.0.9                h166bdaf_8    conda-forge
     libcblas                  3.9.0           16_linux64_openblas    conda-forge
     libclang-cpp11.1          11.1.0          default_ha53f305_1    conda-forge
     libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
     libcudf                   23.02.00a221130 cuda11_g5f83a84916_111    rapidsai-nightly
     libcumlprims              23.02.00a221129 cuda11_gad3dcef_4    rapidsai-nightly
     libcurl                   7.86.0               h7bff187_1    conda-forge
     libcusolver               11.4.1.48                     0    nvidia
     libcusparse               11.7.5.86                     0    nvidia
     libcypher-parser          0.6.2                         1    rapidsai
     libdap4                   3.20.6               hd7c4107_2    conda-forge
     libdeflate                1.14                 h166bdaf_0    conda-forge
     libedit                   3.1.20191231         he28a2e2_2    conda-forge
     libev                     4.33                 h516909a_1    conda-forge
     libevent                  2.1.10               h9b69904_4    conda-forge
     libfaiss                  1.7.0           cuda112h5bea7ad_8_cuda    conda-forge
     libffi                    3.4.2                h7f98852_5    conda-forge
     libgcc-devel_linux-64     9.5.0               h0a57e50_19    conda-forge
     libgcc-ng                 12.2.0              h65d4601_19    conda-forge
     libgcrypt                 1.10.1               h166bdaf_0    conda-forge
     libgd                     2.3.3                h18fbbfe_3    conda-forge
     libgdal                   3.5.2                hc23bfc3_3    conda-forge
     libgfortran-ng            12.2.0              h69a702a_19    conda-forge
     libgfortran5              12.2.0              h337968e_19    conda-forge
     libglib                   2.74.1               h606061b_1    conda-forge
     libgomp                   12.2.0              h65d4601_19    conda-forge
     libgoogle-cloud           1.40.2               habd0e3a_0    conda-forge
     libgpg-error              1.45                 hc0c96e0_0    conda-forge
     libgsasl                  1.10.0               h5b4c23d_0    conda-forge
     libhiredis                1.0.2                h2cc385e_0    conda-forge
     libiconv                  1.17                 h166bdaf_0    conda-forge
     libkml                    1.3.0             h238a007_1014    conda-forge
     liblapack                 3.9.0           16_linux64_openblas    conda-forge
     liblief                   0.12.3               h27087fc_0    conda-forge
     libllvm11                 11.1.0               he0ac6c6_5    conda-forge
     libnetcdf                 4.8.1           nompi_h261ec11_106    conda-forge
     libnghttp2                1.47.0               hdcd2b5c_1    conda-forge
     libnsl                    2.0.0                h7f98852_0    conda-forge
     libntlm                   1.4               h7f98852_1002    conda-forge
     libopenblas               0.3.21          pthreads_h78a6416_3    conda-forge
     libpng                    1.6.38               h753d276_0    conda-forge
     libpq                     14.5                 hd77ab85_1    conda-forge
     libprotobuf               3.20.1               h6239696_4    conda-forge
     libraft-distance          23.02.00a221130 cuda11_g0d76264_35    rapidsai-nightly
     libraft-headers           23.02.00a221130 cuda11_g0d76264_35    rapidsai-nightly
     libraft-nn                23.02.00a221130 cuda11_g0d76264_35    rapidsai-nightly
     librdkafka                1.7.0                hc49e61c_1    conda-forge
     librmm                    23.02.00a221117 cuda11_gd132e523_8    rapidsai-nightly
     librsvg                   2.54.4               h7abd40a_0    conda-forge
     librttopo                 1.1.0               hf730bdb_11    conda-forge
     libsanitizer              9.5.0               h2f262e1_19    conda-forge
     libsodium                 1.0.18               h36c2ea0_1    conda-forge
     libspatialindex           1.9.3                h9c3ff4c_4    conda-forge
     libspatialite             5.0.1               hd36657c_19    conda-forge
     libsqlite                 3.39.4               h753d276_0    conda-forge
     libssh2                   1.10.0               haa6b8db_3    conda-forge
     libstdcxx-devel_linux-64  9.5.0               h0a57e50_19    conda-forge
     libstdcxx-ng              12.2.0              h46fd767_19    conda-forge
     libthrift                 0.16.0               h491838f_2    conda-forge
     libtiff                   4.4.0                h55922b4_4    conda-forge
     libtool                   2.4.6             h9c3ff4c_1008    conda-forge
     libutf8proc               2.8.0                h166bdaf_0    conda-forge
     libuuid                   2.32.1            h7f98852_1000    conda-forge
     libuv                     1.44.2               h166bdaf_0    conda-forge
     libwebp                   1.2.4                h522a892_0    conda-forge
     libwebp-base              1.2.4                h166bdaf_0    conda-forge
     libxcb                    1.13              h7f98852_1004    conda-forge
     libxml2                   2.10.3               h7463322_0    conda-forge
     libzip                    1.9.2                hc869a4a_1    conda-forge
     libzlib                   1.2.13               h166bdaf_4    conda-forge
     libzopfli                 1.0.3                h9c3ff4c_0    conda-forge
     lightgbm                  3.3.3            py38hfa26641_1    conda-forge
     livereload                2.6.3              pyh9f0ad1d_0    conda-forge
     llvmlite                  0.39.1           py38h38d86a4_1    conda-forge
     locket                    1.0.0              pyhd8ed1ab_0    conda-forge
     lz4                       4.0.2            py38h1bf946c_0    conda-forge
     lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
     lzo                       2.10              h516909a_1000    conda-forge
     m4                        1.4.18            h516909a_1001    conda-forge
     make                      4.3                  hd18ef5c_1    conda-forge
     makefun                   1.15.0             pyhd8ed1ab_0    conda-forge
     mapclassify               2.4.3              pyhd8ed1ab_0    conda-forge
     markdown                  3.4.1              pyhd8ed1ab_0    conda-forge
     markdown-it-py            2.1.0              pyhd8ed1ab_0    conda-forge
     markupsafe                2.1.1            py38h0a891b7_2    conda-forge
     matplotlib-base           3.6.2            py38hb021067_0    conda-forge
     matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
     mccabe                    0.6.1                      py_1    conda-forge
     mdit-py-plugins           0.3.1              pyhd8ed1ab_0    conda-forge
     mdurl                     0.1.0              pyhd8ed1ab_0    conda-forge
     mimesis                   6.1.1              pyhd8ed1ab_0    conda-forge
     mistune                   2.0.4              pyhd8ed1ab_0    conda-forge
     mock                      4.0.3              pyhd8ed1ab_4    conda-forge
     more-itertools            9.0.0              pyhd8ed1ab_0    conda-forge
     moto                      4.0.9              pyhd8ed1ab_0    conda-forge
     mpc                       1.2.1                h9f54685_0    conda-forge
     mpfr                      4.1.0                h9202a9a_1    conda-forge
     msgpack-python            1.0.4            py38h43d8883_1    conda-forge
     multidict                 6.0.2            py38h0a891b7_2    conda-forge
     multipledispatch          0.6.0                      py_0    conda-forge
     munch                     2.5.0                      py_0    conda-forge
     munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
     mypy                      0.971            py38h0a891b7_0    conda-forge
     mypy_extensions           0.4.3            py38h578d9bd_6    conda-forge
     myst-nb                   0.17.1             pyhd8ed1ab_0    conda-forge
     myst-parser               0.18.1             pyhd8ed1ab_0    conda-forge
     nbclassic                 0.4.8              pyhd8ed1ab_0    conda-forge
     nbclient                  0.5.13             pyhd8ed1ab_0    conda-forge
     nbconvert                 7.2.3              pyhd8ed1ab_0    conda-forge
     nbconvert-core            7.2.3              pyhd8ed1ab_0    conda-forge
     nbconvert-pandoc          7.2.3              pyhd8ed1ab_0    conda-forge
     nbformat                  5.7.0              pyhd8ed1ab_0    conda-forge
     nbsphinx                  0.8.9              pyhd8ed1ab_0    conda-forge
     nccl                      2.14.3.1             h0800d71_0    conda-forge
     ncurses                   6.3                  h27087fc_1    conda-forge
     nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
     networkx                  2.6.3              pyhd8ed1ab_1    conda-forge
     ninja                     1.11.0               h924138e_0    conda-forge
     nltk                      3.7                pyhd8ed1ab_0    conda-forge
     nodeenv                   1.7.0              pyhd8ed1ab_0    conda-forge
     nodejs                    18.11.0              h96d913c_0    conda-forge
     notebook                  6.5.2              pyha770c72_1    conda-forge
     notebook-shim             0.2.2              pyhd8ed1ab_0    conda-forge
     nspr                      4.32                 h9c3ff4c_1    conda-forge
     nss                       3.78                 h2350873_0    conda-forge
     numba                     0.56.3           py38h9a4aae9_0    conda-forge
     numpy                     1.23.4           py38h7042d01_1    conda-forge
     numpydoc                  1.5.0              pyhd8ed1ab_0    conda-forge
     nvtx                      0.2.3            py38h0a891b7_2    conda-forge
     oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
     openapi-schema-validator  0.2.3              pyhd8ed1ab_0    conda-forge
     openapi-spec-validator    0.4.0              pyhd8ed1ab_1    conda-forge
     openblas                  0.3.21          pthreads_h320a7e8_3    conda-forge
     openjpeg                  2.5.0                h7d73246_1    conda-forge
     openslide                 3.4.1                h71beb9a_5    conda-forge
     openssl                   1.1.1s               h166bdaf_0    conda-forge
     orc                       1.7.5                h6c59b99_0    conda-forge
     packaging                 21.3               pyhd8ed1ab_0    conda-forge
     pandas                    1.5.1            py38h8f669ce_1    conda-forge
     pandoc                    1.19.2                        0    conda-forge
     pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
     panel                     0.12.7             pyhd8ed1ab_0    conda-forge
     pango                     1.50.11              h382ae3d_0    conda-forge
     param                     1.12.2             pyh6c4a22f_0    conda-forge
     paramiko                  2.12.0             pyhd8ed1ab_0    conda-forge
     parquet-cpp               1.5.1                         2    conda-forge
     parso                     0.8.3              pyhd8ed1ab_0    conda-forge
     partd                     1.3.0              pyhd8ed1ab_0    conda-forge
     patch                     2.7.6             h7f98852_1002    conda-forge
     patchelf                  0.17.0               h58526e2_0    conda-forge
     pathspec                  0.10.1             pyhd8ed1ab_0    conda-forge
     patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
     pbr                       5.11.0             pyhd8ed1ab_0    conda-forge
     pcre                      8.45                 h9c3ff4c_0    conda-forge
     pcre2                     10.40                hc3806b6_0    conda-forge
     perl                      5.32.1          2_h7f98852_perl5    conda-forge
     pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
     pickleshare               0.7.5                   py_1003    conda-forge
     pillow                    9.2.0            py38h9eb91d8_3    conda-forge
     pip                       22.3.1             pyhd8ed1ab_0    conda-forge
     pixman                    0.40.0               h36c2ea0_0    conda-forge
     pkg-config                0.29.2            h36c2ea0_1008    conda-forge
     pkginfo                   1.8.3              pyhd8ed1ab_0    conda-forge
     platformdirs              2.5.2              pyhd8ed1ab_1    conda-forge
     pluggy                    1.0.0              pyhd8ed1ab_5    conda-forge
     poppler                   22.04.0              h8b295ee_2    conda-forge
     poppler-data              0.4.11               hd8ed1ab_0    conda-forge
     postgresql                14.5                 hdeef612_1    conda-forge
     pre-commit                2.20.0           py38h578d9bd_1    conda-forge
     proj                      9.0.1                h93bde94_1    conda-forge
     prometheus_client         0.15.0             pyhd8ed1ab_0    conda-forge
     prompt-toolkit            3.0.32             pyha770c72_0    conda-forge
     protobuf                  3.20.1           py38hfa26641_0    conda-forge
     psutil                    5.9.4            py38h0a891b7_0    conda-forge
     pthread-stubs             0.4               h36c2ea0_1001    conda-forge
     ptxcompiler               0.7.0            py38h7525318_2    conda-forge
     ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
     py-cpuinfo                9.0.0              pyhd8ed1ab_0    conda-forge
     py-lief                   0.12.3           py38hfa26641_0    conda-forge
     pyarrow                   9.0.0           py38he3cd574_1_cpu    conda-forge
     pyasn1                    0.4.8                      py_0    conda-forge
     pyasn1-modules            0.2.7                      py_0    conda-forge
     pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
     pycosat                   0.6.4            py38h0a891b7_1    conda-forge
     pycparser                 2.21               pyhd8ed1ab_0    conda-forge
     pyct                      0.4.6                      py_0    conda-forge
     pyct-core                 0.4.6                      py_0    conda-forge
     pydata-sphinx-theme       0.11.0             pyhd8ed1ab_1    conda-forge
     pydeck                    0.5.0              pyh9f0ad1d_0    conda-forge
     pydocstyle                6.1.1              pyhd8ed1ab_0    conda-forge
     pyee                      8.1.0              pyhd8ed1ab_0    conda-forge
     pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
     pygments                  2.13.0             pyhd8ed1ab_0    conda-forge
     pyjwt                     2.6.0              pyhd8ed1ab_0    conda-forge
     pylibraft                 23.02.00a221130 cuda11_py38_g0d76264_35    rapidsai-nightly
     pylint                    2.15.6             pyhd8ed1ab_0    conda-forge
     pynacl                    1.5.0            py38h0a891b7_2    conda-forge
     pynndescent               0.5.8              pyh1a96a4e_0    conda-forge
     pynvml                    11.4.1             pyhd8ed1ab_0    conda-forge
     pyopenssl                 22.1.0             pyhd8ed1ab_0    conda-forge
     pyorc                     0.7.0            py38hd2b0b8b_0    conda-forge
     pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
     pyppeteer                 1.0.2              pyhd8ed1ab_0    conda-forge
     pyproj                    3.4.0            py38he1635e7_0    conda-forge
     pyrsistent                0.19.2           py38h0a891b7_0    conda-forge
     pysocks                   1.7.1              pyha2e5f31_6    conda-forge
     pytest                    7.2.0              pyhd8ed1ab_2    conda-forge
     pytest-asyncio            0.12.0           py38h32f6830_2    conda-forge
     pytest-benchmark          4.0.0              pyhd8ed1ab_0    conda-forge
     pytest-cases              3.6.13             pyhd8ed1ab_0    conda-forge
     pytest-cov                4.0.0              pyhd8ed1ab_0    conda-forge
     pytest-timeout            2.1.0              pyhd8ed1ab_0    conda-forge
     pytest-xdist              3.0.2              pyhd8ed1ab_0    conda-forge
     python                    3.8.13          h582c2e5_0_cpython    conda-forge
     python-confluent-kafka    1.7.0            py38h497a2fe_2    conda-forge
     python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
     python-fastjsonschema     2.16.2             pyhd8ed1ab_0    conda-forge
     python-jose               3.3.0              pyh6c4a22f_1    conda-forge
     python-libarchive-c       4.0              py38h578d9bd_2    conda-forge
     python-louvain            0.15               pyhd8ed1ab_1    conda-forge
     python-snappy             0.6.0            py38h1ddbb56_2    conda-forge
     python_abi                3.8                      2_cp38    conda-forge
     pytz                      2022.6             pyhd8ed1ab_0    conda-forge
     pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
     pyviz_comms               2.2.1              pyhd8ed1ab_1    conda-forge
     pywavelets                1.3.0            py38h26c90d9_2    conda-forge
     pywin32-on-windows        0.1.0              pyh1179c8e_3    conda-forge
     pyyaml                    6.0              py38h0a891b7_5    conda-forge
     pyzmq                     24.0.1           py38hfc09fa9_1    conda-forge
     raft-dask                 23.02.00a221130 cuda11_py38_g0d76264_35    rapidsai-nightly
     rapidjson                 1.1.0             he1b5a44_1002    conda-forge
     rapids-build-env          23.02.00a221129 cuda11_py38_g284f34a_22    rapidsai-nightly
     rapids-doc-env            23.02.00a221130 py38_g9feda8d_25    rapidsai-nightly
     rapids-notebook-env       23.02.00a221130 cuda11_py38_g9feda8d_25    rapidsai-nightly
     re2                       2022.06.01           h27087fc_0    conda-forge
     readline                  8.1.2                h0f457ee_0    conda-forge
     readme_renderer           37.3               pyhd8ed1ab_0    conda-forge
     recommonmark              0.7.1              pyhd8ed1ab_0    conda-forge
     regex                     2022.10.31       py38h0a891b7_0    conda-forge
     requests                  2.28.1             pyhd8ed1ab_1    conda-forge
     requests-oauthlib         1.3.1              pyhd8ed1ab_0    conda-forge
     requests-toolbelt         0.10.1             pyhd8ed1ab_0    conda-forge
     responses                 0.21.0             pyhd8ed1ab_0    conda-forge
     rfc3986                   2.0.0              pyhd8ed1ab_0    conda-forge
     rhash                     1.4.3                h166bdaf_0    conda-forge
     rich                      12.6.0             pyhd8ed1ab_0    conda-forge
     ripgrep                   13.0.0               h2f28480_2    conda-forge
     rmm                       23.02.00a221117 cuda11_py38_gd132e523_8    rapidsai-nightly
     rsa                       4.9                pyhd8ed1ab_0    conda-forge
     rtree                     1.0.1            py38h02d302b_1    conda-forge
     ruamel_yaml               0.15.80         py38h0a891b7_1008    conda-forge
     s2n                       1.0.10               h9b69904_0    conda-forge
     s3fs                      2022.10.0          pyhd8ed1ab_0    conda-forge
     s3transfer                0.6.0              pyhd8ed1ab_0    conda-forge
     sacremoses                0.0.53             pyhd8ed1ab_0    conda-forge
     sarif-om                  1.0.4              pyhd8ed1ab_0    conda-forge
     scikit-build              0.13.1             pyhca92ed8_0    conda-forge
     scikit-image              0.19.3           py38h8f669ce_2    conda-forge
     scikit-learn              0.24.2           py38hacb3eff_1    conda-forge
     scipy                     1.6.0            py38hb2138dd_0    conda-forge
     seaborn                   0.12.1               hd8ed1ab_0    conda-forge
     seaborn-base              0.12.1             pyhd8ed1ab_0    conda-forge
     secretstorage             3.3.3            py38h578d9bd_1    conda-forge
     send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
     setuptools                60.10.0          py38h578d9bd_0    conda-forge
     shapely                   1.8.5            py38hc9bb657_1    conda-forge
     shellcheck                0.8.0                ha770c72_0    conda-forge
     simpervisor               0.4                pyhd8ed1ab_0    conda-forge
     six                       1.16.0             pyh6c4a22f_0    conda-forge
     snappy                    1.1.9                hbd366e4_2    conda-forge
     sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
     snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
     sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
     soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
     sparse                    0.13.0                   pypi_0    pypi
     spdlog                    1.8.5                h4bd325d_1    conda-forge
     sphinx                    5.3.0              pyhd8ed1ab_0    conda-forge
     sphinx-autobuild          2021.3.14          pyhd8ed1ab_0    conda-forge
     sphinx-click              4.3.0              pyhd8ed1ab_0    conda-forge
     sphinx-copybutton         0.5.0              pyhd8ed1ab_0    conda-forge
     sphinx-markdown-tables    0.0.17             pyh6c4a22f_0    conda-forge
     sphinx_rtd_theme          0.5.2              pyhd8ed1ab_0    conda-forge
     sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
     sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
     sphinxcontrib-htmlhelp    2.0.0              pyhd8ed1ab_0    conda-forge
     sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
     sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
     sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_2    conda-forge
     sphinxcontrib-websupport  1.2.4              pyhd8ed1ab_1    conda-forge
     sqlalchemy                1.4.43           py38h0a891b7_0    conda-forge
     sqlite                    3.39.4               h4ff8645_0    conda-forge
     sshpubkeys                3.3.1              pyhd8ed1ab_0    conda-forge
     statsmodels               0.13.5           py38h26c90d9_2    conda-forge
     streamz                   0.6.4              pyh6c4a22f_0    conda-forge
     sysroot_linux-64          2.17                h4a8ded7_13    conda-forge
     tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
     tbb                       2021.6.0             h924138e_1    conda-forge
     tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
     terminado                 0.17.0             pyh41d4057_0    conda-forge
     threadpoolctl             3.1.0              pyh8a188c0_0    conda-forge
     tifffile                  2022.10.10         pyhd8ed1ab_0    conda-forge
     tiledb                    2.11.3               h1e4a385_1    conda-forge
     tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
     tk                        8.6.12               h27826a3_0    conda-forge
     tokenizers                0.10.3           py38hb63a372_1    conda-forge
     toml                      0.10.2             pyhd8ed1ab_0    conda-forge
     tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
     tomlkit                   0.11.6             pyha770c72_0    conda-forge
     toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
     tornado                   6.1              py38h0a891b7_3    conda-forge
     tqdm                      4.64.1             pyhd8ed1ab_0    conda-forge
     traitlets                 5.5.0              pyhd8ed1ab_0    conda-forge
     transformers              4.6.1              pyhd8ed1ab_0    conda-forge
     treelite                  3.0.0            py38h8e2129e_1    conda-forge
     treelite-runtime          3.0.0                    pypi_0    pypi
     twine                     4.0.1              pyhd8ed1ab_1    conda-forge
     typed-ast                 1.5.4            py38h0a891b7_1    conda-forge
     types-cachetools          5.2.1              pyhd8ed1ab_0    conda-forge
     typing                    3.10.0.0           pyhd8ed1ab_0    conda-forge
     typing-extensions         4.4.0                hd8ed1ab_0    conda-forge
     typing_extensions         4.4.0              pyha770c72_0    conda-forge
     tzcode                    2022f                h166bdaf_0    conda-forge
     tzdata                    2022f                h191b570_0    conda-forge
     ucx                       1.13.1               h538f049_0    conda-forge
     ucx-proc                  1.0.0                       gpu    rapidsai
     ucx-py                    0.30.00a221115  py38_gb080671_0    rapidsai-nightly
     ukkonen                   1.0.1            py38h43d8883_3    conda-forge
     umap-learn                0.5.3            py38h578d9bd_0    conda-forge
     unicodedata2              15.0.0           py38h0a891b7_0    conda-forge
     urllib3                   1.26.11            pyhd8ed1ab_0    conda-forge
     versioneer                0.28               pyhd8ed1ab_0    conda-forge
     virtualenv                20.16.5          py38h578d9bd_1    conda-forge
     wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
     webencodings              0.5.1                      py_1    conda-forge
     websocket-client          1.4.2              pyhd8ed1ab_0    conda-forge
     websockets                10.4             py38h0a891b7_1    conda-forge
     werkzeug                  2.1.2              pyhd8ed1ab_1    conda-forge
     wheel                     0.38.3             pyhd8ed1ab_0    conda-forge
     widgetsnbextension        4.0.3              pyhd8ed1ab_0    conda-forge
     wrapt                     1.14.1           py38h0a891b7_1    conda-forge
     xarray                    2022.11.0          pyhd8ed1ab_0    conda-forge
     xerces-c                  3.2.4                h55805fa_1    conda-forge
     xmltodict                 0.13.0             pyhd8ed1ab_0    conda-forge
     xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
     xorg-libice               1.0.10               h7f98852_0    conda-forge
     xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
     xorg-libx11               1.7.2                h7f98852_0    conda-forge
     xorg-libxau               1.0.9                h7f98852_0    conda-forge
     xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
     xorg-libxext              1.3.4                h7f98852_1    conda-forge
     xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
     xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
     xorg-xextproto            7.3.0             h7f98852_1002    conda-forge
     xorg-xproto               7.0.31            h7f98852_1007    conda-forge
     xyzservices               2022.9.0           pyhd8ed1ab_0    conda-forge
     xz                        5.2.6                h166bdaf_0    conda-forge
     yaml                      0.2.5                h7f98852_2    conda-forge
     yarl                      1.8.1            py38h0a891b7_0    conda-forge
     zeromq                    4.3.4                h9c3ff4c_1    conda-forge
     zfp                       1.0.0                h27087fc_3    conda-forge
     zict                      2.2.0              pyhd8ed1ab_0    conda-forge
     zipp                      3.10.0             pyhd8ed1ab_0    conda-forge
     zlib                      1.2.13               h166bdaf_4    conda-forge
     zlib-ng                   2.0.6                h166bdaf_0    conda-forge
     zstd                      1.5.2                h6239696_4    conda-forge

galipremsagar commented 1 year ago

@karthikeyann Looks like libcudf NULL_EQUALS binary op isn't accounting for np.nan values

shwina commented 1 year ago

I don't think this is a bug in libcudf (typically, NaNs do not compare equal). Possibly something we need to find a solution for in Python land.

davidwendt commented 1 year ago

The libcudf row-comparator has specialized supported to report equality when comparing NaNs: https://github.com/rapidsai/cudf/blob/5f83a8491603eadc1de5cb174016801c1cca5824/cpp/include/cudf/table/row_operators.cuh#L138-L143 I think we can make this consistent at least for NULL_EQUALS https://github.com/rapidsai/cudf/blob/5f83a8491603eadc1de5cb174016801c1cca5824/cpp/src/binaryop/compiled/operation.cuh#L387-L400

davidwendt commented 1 year ago

I made a quick change to NullEquals in https://github.com/rapidsai/cudf/pull/12275 for discussion purposes and in case you want to try it out.

vyasr commented 3 months ago

@wence- is #15731 relevant here?

wence- commented 3 months ago

In this sense that it was adding another specialised binop equality comparator, yes. But using it wouldn't fix this problem.

If we want to carve out new NANS_LIKE_NULL_EQUALS/NANS_LIKE_NULL_NOT_EQUALS binops, we could do so. Otherwise we could support the requirements of DataFrame.equals by calling nans_to_nulls on the columns first and then comparing with NULL_EQUALS.

If we don't want to add to libcudf, we could have a PTX-jitted binop for this that we pass in via GENERIC_PTX or whatever it's called, I think.