Closed mpenn closed 4 months ago
jarmak-nv
commented
8 months ago Hi @mpenn!
Thanks for submitting this issue - our team has been notified and we'll get back to you as soon as we can! In the mean time, feel free to add any relevant information to this issue.
efajardo-nv
commented
7 months ago UPDATE: This issue happens in the call to MessageMetaInterfaceProxy::get_data_frame which first creates a TableInfo object from the underlying cuDF DataFrame and then creates a new cuDF DataFrame using CudfHelper::table_from_table_info. In CudfHelper::table_from_table_info, the column metadata which holds the struct key names are not retained after calling cudf.DataFrame._from_data with information from our TableInfo. We can manually create the column metadata and use cuDF's cudf._lib.io.utils.update_struct_field_names function to add the metadata back to DataFrame. However, we run into an issue if the dataframe is sliced, specifically when a column's _offset is not 0. We get the following exception:
File cudf_helpers.pyx:226, in morpheus._lib.cudf_helpers.update_struct_field_names()
File cudf_helpers.pyx:250, in morpheus._lib.cudf_helpers.update_column_struct_field_names()
File ~/miniconda3/envs/morpheus/lib/python3.10/site-packages/cudf/core/column/struct.py:111, in StructColumn._rename_fields(self, names)
105 def _rename_fields(self, names):
106 """
107 Return a StructColumn with the same field values as this StructColumn,
108 but with the field names equal to `names`.
109 """
110 dtype = cudf.core.dtypes.StructDtype(
--> 111 {name: col.dtype for name, col in zip(names, self.children)}
112 )
113 return StructColumn(
114 data=None,
115 size=self.size,
(...)
120 children=self.base_children,
121 )
File column.pyx:290, in cudf._lib.column.Column.children.__get__()
RuntimeError: CUDF failure at: /opt/conda/conda-bld/work/cpp/src/copying/slice.cu:44: Slice range out of bounds.The update_struct_field_names function in cuDF is only used in their JSON and Parquet readers immediately after reading a file into a DataFrame so apparently it was not intended to be used on sliced DataFrames.
This issue is also seen with C++ MultiMessage.get_meta and MultiMessage.copy_ranges because they also use TableInfo for slicing and cudf_helpers to create the new DataFrame.
One workaround for this is to bypass the TableInfoand cudf_helpers layer and slice the underlying cuDF DataFrame directly. This has been implemented for C++ MultiMessage.get_meta in this issue's corresponding PR (#1538) and seems to be working.
The C++ MessageMeta.get_data_frame in the PR currently uses cuDF's update_struct_field_names so will throw the above exception if SlicedMessageMeta is used with struct or list columns. We should also be able to slice the cuDF DataFrame directly as we did with MultiMessage.get_meta if that approach is acceptable.
Version
24.03
Which installation method(s) does this occur on?
Docker, Conda, Source
Describe the bug.
When storing
cudf.core.dtypes.StructDtypecolumns in amorpheus._lib.messages.MessageMetafrom Python, the key names are not preserved. Instead, they appear to be converted into incrementing integers that have been cast to strings. This behavior can also be observed when trying to store the same kind ofcudf.core.dtypes.StructDtypecolumns in aControlMessagepayload.Here is an example of the desired output that works when using a Python
MessageMeta(i.e.CppConfig.set_should_use_cpp(False)). In this example, column "c0" is acudf.core.dtypes.StructDtypecolumn containing properly named keys.Here is the offending output when using the C++
morpheus._lib.messages.MessageMetaalternative (i.e.CppConfig.set_should_use_cpp(True)). In this example column "c0" is acudf.core.dtypes.StructDtypecolumn containing improperly named keys.Minimum reproducible example
Relevant log output
Click here to see error details
(morpheus) foo@bar:/workspace# python reproducer.py Result (MM) use_cpp=False: c0 c1 0 {'k0': 'v0', 'k1': 'v3'} 0 1 {'k0': 'v1', 'k1': 'v4'} 1 2 {'k0': 'v2', 'k1': 'v5'} 2 Result (CM) use_cpp=False: c0 c1 0 {'0': 'v0', '1': 'v3'} 0 1 {'0': 'v1', '1': 'v4'} 1 2 {'0': 'v2', '1': 'v5'} 2 (morpheus) foo@bar:/workspace# python reproducer.py Result (MM) use_cpp=True: c0 c1 0 {'0': 'v0', '1': 'v3'} 0 1 {'0': 'v1', '1': 'v4'} 1 2 {'0': 'v2', '1': 'v5'} 2 Result (CM) use_cpp=True: c0 c1 0 {'0': 'v0', '1': 'v3'} 0 1 {'0': 'v1', '1': 'v4'} 1 2 {'0': 'v2', '1': 'v5'} 2Full env printout
Click here to see environment details
**git*** Not inside a git repository ***OS Information*** DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS" PRETTY_NAME="Ubuntu 22.04.3 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.3 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy Linux 4ae420838a70 5.19.0-45-generic #46~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jun 7 15:06:04 UTC 20 x86_64 x86_64 x86_64 GNU/Linux ***GPU Information*** Sun Feb 18 01:25:03 2024 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 530.30.02 Driver Version: 530.30.02 CUDA Version: 12.1 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 Quadro RTX 8000 On | 00000000:15:00.0 Off | Off | | 33% 35C P8 39W / 260W| 2852MiB / 49152MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| +---------------------------------------------------------------------------------------+ ***CPU*** Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 46 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 24 On-line CPU(s) list: 0-23 Vendor ID: GenuineIntel Model name: Intel(R) Xeon(R) Gold 6128 CPU @ 3.40GHz CPU family: 6 Model: 85 Thread(s) per core: 2 Core(s) per socket: 6 Socket(s): 2 Stepping: 4 CPU max MHz: 3700.0000 CPU min MHz: 1200.0000 BogoMIPS: 6800.00 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke md_clear flush_l1d arch_capabilities L1d cache: 384 KiB (12 instances) L1i cache: 384 KiB (12 instances) L2 cache: 12 MiB (12 instances) L3 cache: 38.5 MiB (2 instances) NUMA node(s): 2 NUMA node0 CPU(s): 0-5,12-17 NUMA node1 CPU(s): 6-11,18-23 Vulnerability Itlb multihit: KVM: Mitigation: VMX unsupported Vulnerability L1tf: Mitigation; PTE Inversion Vulnerability Mds: Mitigation; Clear CPU buffers; SMT vulnerable Vulnerability Meltdown: Mitigation; PTI Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable Vulnerability Retbleed: Mitigation; IBRS Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT vulnerable ***CMake*** ***g++*** /usr/bin/g++ g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ***nvcc*** /opt/conda/envs/morpheus/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Mon_Apr__3_17:16:06_PDT_2023 Cuda compilation tools, release 12.1, V12.1.105 Build cuda_12.1.r12.1/compiler.32688072_0 ***Python*** /opt/conda/envs/morpheus/bin/python Python 3.10.13 ***Environment Variables*** PATH : /root/.vscode-server/bin/05047486b6df5eb8d44b2ecd70ea3bdf775fd937/bin/remote-cli:/opt/conda/envs/morpheus/bin:/opt/conda/condabin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/conda/bin LD_LIBRARY_PATH : /usr/local/nvidia/lib:/usr/local/nvidia/lib64 NUMBAPRO_NVVM : NUMBAPRO_LIBDEVICE : CONDA_PREFIX : /opt/conda/envs/morpheus PYTHON_PATH : ***conda packages*** /opt/conda/condabin/conda # packages in environment at /opt/conda/envs/morpheus: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge absl-py 1.4.0 pyhd8ed1ab_0 conda-forge adagio 0.2.4 pyhd8ed1ab_0 conda-forge aiofiles 22.1.0 pypi_0 pypi aiohttp 3.9.3 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi aiosqlite 0.19.0 pypi_0 pypi alabaster 0.7.16 pyhd8ed1ab_0 conda-forge alembic 1.13.1 pyhd8ed1ab_1 conda-forge annotated-types 0.6.0 pyhd8ed1ab_0 conda-forge antlr-python-runtime 4.11.1 pyhd8ed1ab_0 conda-forge antlr4-python3-runtime 4.11.1 pyh1a96a4e_0 conda-forge anyio 3.7.1 pypi_0 pypi appdirs 1.4.4 pyh9f0ad1d_0 conda-forge argon2-cffi 23.1.0 pyhd8ed1ab_0 conda-forge argon2-cffi-bindings 21.2.0 py310h2372a71_4 conda-forge arrow 1.3.0 pyhd8ed1ab_0 conda-forge arxiv 1.4.8 pyhd8ed1ab_0 conda-forge asn1crypto 1.5.1 pyhd8ed1ab_0 conda-forge asttokens 2.4.1 pyhd8ed1ab_0 conda-forge async-timeout 4.0.3 pypi_0 pypi atk-1.0 2.38.0 hd4edc92_1 conda-forge attrs 23.2.0 pyh71513ae_0 conda-forge aws-c-auth 0.7.11 h0b4cabd_1 conda-forge aws-c-cal 0.6.9 h14ec70c_3 conda-forge aws-c-common 0.9.12 hd590300_0 conda-forge aws-c-compression 0.2.17 h572eabf_8 conda-forge aws-c-event-stream 0.4.1 h97bb272_2 conda-forge aws-c-http 0.8.0 h9129f04_2 conda-forge aws-c-io 0.14.0 hf8f278a_1 conda-forge aws-c-mqtt 0.10.1 h2b97f5f_0 conda-forge aws-c-s3 0.4.9 hca09fc5_0 conda-forge aws-c-sdkutils 0.1.13 h572eabf_1 conda-forge aws-checksums 0.1.17 h572eabf_7 conda-forge aws-crt-cpp 0.26.0 h04327c0_8 conda-forge aws-sdk-cpp 1.11.210 hba3e011_10 conda-forge azure-ai-formrecognizer 3.3.2 pypi_0 pypi azure-common 1.1.28 pypi_0 pypi azure-core 1.30.0 pypi_0 pypi babel 2.14.0 pyhd8ed1ab_0 conda-forge backoff 2.2.1 pypi_0 pypi beautifulsoup4 4.12.3 pyha770c72_0 conda-forge betterproto 1.2.5 pyhd3deb0d_1 conda-forge blas 1.0 mkl conda-forge bleach 6.1.0 pyhd8ed1ab_0 conda-forge blinker 1.7.0 pyhd8ed1ab_0 conda-forge boilerpy3 1.0.7 pypi_0 pypi bokeh 3.3.4 pyhd8ed1ab_0 conda-forge boost-cpp 1.84.0 h44aadfe_1 conda-forge brotli 1.1.0 hd590300_1 conda-forge brotli-bin 1.1.0 hd590300_1 conda-forge brotli-python 1.1.0 py310hc6cd4ac_1 conda-forge bzip2 1.0.8 hd590300_5 conda-forge c-ares 1.26.0 hd590300_0 conda-forge ca-certificates 2024.2.2 hbcca054_0 conda-forge cached-property 1.5.2 hd8ed1ab_1 conda-forge cached_property 1.5.2 pyha770c72_1 conda-forge cachetools 5.3.2 pyhd8ed1ab_0 conda-forge cairo 1.18.0 h3faef2a_0 conda-forge cattrs 23.2.3 pyhd8ed1ab_0 conda-forge certifi 2024.2.2 pyhd8ed1ab_0 conda-forge cffi 1.16.0 py310h2fee648_0 conda-forge charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge click 8.1.7 unix_pyh707e725_0 conda-forge cloudpickle 3.0.0 pyhd8ed1ab_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge comm 0.2.1 pyhd8ed1ab_0 conda-forge configargparse 1.5.5 pyhd8ed1ab_0 conda-forge configparser 5.3.0 pyhd8ed1ab_0 conda-forge contourpy 1.2.0 py310hd41b1e2_0 conda-forge cryptography 42.0.3 py310h75e40e8_0 conda-forge cssselect 1.2.0 pyhd8ed1ab_0 conda-forge cuda-cccl_linux-64 12.1.109 ha770c72_0 conda-forge cuda-cudart 12.1.105 hd3aeb46_0 conda-forge cuda-cudart-dev 12.1.105 hd3aeb46_0 conda-forge cuda-cudart-dev_linux-64 12.1.105 h59595ed_0 conda-forge cuda-cudart-static 12.1.105 hd3aeb46_0 conda-forge cuda-cudart-static_linux-64 12.1.105 h59595ed_0 conda-forge cuda-cudart_linux-64 12.1.105 h59595ed_0 conda-forge cuda-cupti 12.1.105 h59595ed_0 conda-forge cuda-libraries 12.1.0 0 nvidia cuda-nvcc-dev_linux-64 12.1.105 ha770c72_0 conda-forge cuda-nvcc-impl 12.1.105 hd3aeb46_0 conda-forge cuda-nvcc-tools 12.1.105 hd3aeb46_0 conda-forge cuda-nvrtc 12.1.105 hd3aeb46_0 conda-forge cuda-nvtx 12.1.105 h59595ed_0 conda-forge cuda-opencl 12.1.105 h59595ed_0 conda-forge cuda-python 12.3.0 py310h9f9f131_2 conda-forge cuda-runtime 12.1.0 0 nvidia cuda-version 12.1 h1d6eff3_2 conda-forge cudf 24.02.01 cuda12_py310_240213_g33ffdf5b71_0 rapidsai cupy 13.0.0 py310h7aad9d2_3 conda-forge cupy-core 13.0.0 py310had4011e_3 conda-forge cycler 0.12.1 pyhd8ed1ab_0 conda-forge cyrus-sasl 2.1.27 h54b06d7_7 conda-forge cytoolz 0.12.3 py310h2372a71_0 conda-forge dask 2024.2.0 pyhd8ed1ab_0 conda-forge dask-core 2024.2.0 pyhd8ed1ab_0 conda-forge dask-cuda 24.02.00 py310_240212_g96bedbc_0 rapidsai dask-cudf 24.02.01 cuda12_py310_240213_g33ffdf5b71_0 rapidsai databricks-cli 0.18.0 pyhd8ed1ab_0 conda-forge databricks-connect 14.3.0 pypi_0 pypi databricks-sdk 0.19.1 pypi_0 pypi dataclasses 0.8 pyhc8e2a94_3 conda-forge dataclasses-json 0.6.4 pypi_0 pypi datacompy 0.10.5 pyhd8ed1ab_0 conda-forge debugpy 1.8.1 py310hc6cd4ac_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge dill 0.3.7 pyhd8ed1ab_0 conda-forge distributed 2024.2.0 pyhd8ed1ab_0 conda-forge dlpack 0.5 h9c3ff4c_0 conda-forge docker-py 5.0.3 pyhd8ed1ab_4 conda-forge docker-pycreds 0.4.0 py_0 conda-forge docopt 0.6.2 pypi_0 pypi docutils 0.20.1 py310hff52083_3 conda-forge elastic-transport 8.12.0 pyhd8ed1ab_0 conda-forge elasticsearch 8.9.0 pyhd8ed1ab_0 conda-forge entrypoints 0.4 pyhd8ed1ab_0 conda-forge environs 9.5.0 pyhd8ed1ab_0 conda-forge events 0.5 pypi_0 pypi exceptiongroup 1.2.0 pyhd8ed1ab_2 conda-forge executing 2.0.1 pyhd8ed1ab_0 conda-forge expat 2.5.0 hcb278e6_1 conda-forge farm-haystack 1.24.1 pypi_0 pypi fastrlock 0.8.2 py310hc6cd4ac_2 conda-forge feedfinder2 0.0.4 py_0 conda-forge feedparser 6.0.10 pyhd8ed1ab_0 conda-forge filelock 3.13.1 pyhd8ed1ab_0 conda-forge flask 3.0.2 pyhd8ed1ab_0 conda-forge fmt 10.2.1 h00ab1b0_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 h77eed37_1 conda-forge fontconfig 2.14.2 h14ed4e7_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.49.0 py310h2372a71_0 conda-forge fqdn 1.5.1 pyhd8ed1ab_0 conda-forge freetype 2.12.1 h267a509_2 conda-forge fribidi 1.0.10 h36c2ea0_0 conda-forge frozenlist 1.4.1 pypi_0 pypi fs 2.4.16 pyhd8ed1ab_0 conda-forge fsspec 2024.2.0 pyhca7485f_0 conda-forge fugue 0.8.7 pyhd8ed1ab_0 conda-forge fugue-sql-antlr 0.2.0 pyhd8ed1ab_0 conda-forge future 0.18.3 pyhd8ed1ab_0 conda-forge gdk-pixbuf 2.42.10 h829c605_4 conda-forge gettext 0.21.1 h27087fc_0 conda-forge gflags 2.2.2 he1b5a44_1004 conda-forge giflib 5.2.1 h0b41bf4_3 conda-forge gitdb 4.0.11 pyhd8ed1ab_0 conda-forge gitpython 3.1.42 pyhd8ed1ab_0 conda-forge glog 0.6.0 h6f12383_0 conda-forge gmock 1.14.0 ha770c72_1 conda-forge gmp 6.3.0 h59595ed_0 conda-forge gmpy2 2.1.2 py310h3ec546c_1 conda-forge google-auth 2.28.0 pypi_0 pypi googleapis-common-protos 1.62.0 pyhd8ed1ab_0 conda-forge graphite2 1.3.13 h58526e2_1001 conda-forge graphviz 9.0.0 h78e8752_1 conda-forge greenlet 3.0.3 py310hc6cd4ac_0 conda-forge grpcio 1.58.0 pypi_0 pypi grpcio-status 1.58.0 pypi_0 pypi grpclib 0.4.7 pyhd8ed1ab_0 conda-forge gtest 1.14.0 h00ab1b0_1 conda-forge gtk2 2.24.33 h7f000aa_3 conda-forge gts 0.7.6 h977cf35_4 conda-forge gunicorn 21.2.0 py310hff52083_1 conda-forge h11 0.14.0 pypi_0 pypi h2 4.1.0 pyhd8ed1ab_0 conda-forge harfbuzz 8.3.0 h3d44ed6_0 conda-forge hpack 4.0.0 pyh9f0ad1d_0 conda-forge httpcore 1.0.3 pypi_0 pypi httpx 0.26.0 pypi_0 pypi huggingface-hub 0.20.3 pypi_0 pypi hyperframe 6.0.1 pyhd8ed1ab_0 conda-forge icu 73.2 h59595ed_0 conda-forge idna 3.6 pyhd8ed1ab_0 conda-forge imagesize 1.4.1 pyhd8ed1ab_0 conda-forge importlib-metadata 7.0.1 pyha770c72_0 conda-forge importlib_metadata 7.0.1 hd8ed1ab_0 conda-forge importlib_resources 6.1.1 pyhd8ed1ab_0 conda-forge inflect 7.0.0 pypi_0 pypi intel-openmp 2022.1.0 h9e868ea_3769 ipykernel 6.29.2 pyhd33586a_0 conda-forge ipython 8.21.0 pyh707e725_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 8.1.2 pyhd8ed1ab_0 conda-forge isodate 0.6.1 pypi_0 pypi isoduration 20.11.0 pyhd8ed1ab_0 conda-forge itsdangerous 2.1.2 pyhd8ed1ab_0 conda-forge jedi 0.19.1 pyhd8ed1ab_0 conda-forge jieba3k 0.35.1 py310hff52083_1008 conda-forge jinja2 3.1.3 pyhd8ed1ab_0 conda-forge joblib 1.3.2 pyhd8ed1ab_0 conda-forge json5 0.9.14 pypi_0 pypi jsonpatch 1.33 pypi_0 pypi jsonpointer 2.4 py310hff52083_3 conda-forge jsonschema 4.21.1 pyhd8ed1ab_0 conda-forge jsonschema-specifications 2023.12.1 pyhd8ed1ab_0 conda-forge jsonschema-with-format-nongpl 4.21.1 pyhd8ed1ab_0 conda-forge jupyter-server-fileid 0.9.1 pypi_0 pypi jupyter-server-proxy 4.1.0 pypi_0 pypi jupyter-server-ydoc 0.8.0 pypi_0 pypi jupyter-ydoc 0.2.5 pypi_0 pypi jupyter_client 7.4.9 pyhd8ed1ab_0 conda-forge jupyter_contrib_core 0.4.0 pyhd8ed1ab_0 conda-forge jupyter_contrib_nbextensions 0.7.0 pyhd8ed1ab_0 conda-forge jupyter_core 5.7.1 py310hff52083_0 conda-forge jupyter_events 0.9.0 pyhd8ed1ab_0 conda-forge jupyter_highlight_selected_word 0.2.0 pyhd8ed1ab_1006 conda-forge jupyter_latex_envs 1.4.6 pyhd8ed1ab_1002 conda-forge jupyter_nbextensions_configurator 0.6.1 pyhd8ed1ab_0 conda-forge jupyter_server 2.12.5 pyhd8ed1ab_0 conda-forge jupyter_server_terminals 0.5.2 pyhd8ed1ab_0 conda-forge jupyterlab 3.6.7 pypi_0 pypi jupyterlab-nvdashboard 0.9.0 pypi_0 pypi jupyterlab-server 2.25.3 pypi_0 pypi jupyterlab_pygments 0.3.0 pyhd8ed1ab_1 conda-forge jupyterlab_widgets 3.0.10 pyhd8ed1ab_0 conda-forge keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.4.5 py310hd41b1e2_1 conda-forge krb5 1.21.2 h659d440_0 conda-forge langchain 0.0.310 pypi_0 pypi langsmith 0.0.92 pypi_0 pypi lazy-imports 0.3.1 pypi_0 pypi lcms2 2.16 hb7c19ff_0 conda-forge ld_impl_linux-64 2.40 h41732ed_0 conda-forge lerc 4.0.0 h27087fc_0 conda-forge libabseil 20230802.1 cxx17_h59595ed_0 conda-forge libarrow 14.0.2 h7303f25_3_cuda conda-forge libarrow-acero 14.0.2 h27087fc_3_cuda conda-forge libarrow-dataset 14.0.2 h27087fc_3_cuda conda-forge libarrow-flight 14.0.2 hc63cbfb_3_cuda conda-forge libarrow-flight-sql 14.0.2 hd924b76_3_cuda conda-forge libarrow-gandiva 14.0.2 hfa8be3f_3_cuda conda-forge libarrow-substrait 14.0.2 hd924b76_3_cuda conda-forge libblas 3.9.0 16_linux64_mkl conda-forge libboost 1.84.0 h8013b2b_1 conda-forge libboost-devel 1.84.0 h00ab1b0_1 conda-forge libboost-headers 1.84.0 ha770c72_1 conda-forge libbrotlicommon 1.1.0 hd590300_1 conda-forge libbrotlidec 1.1.0 hd590300_1 conda-forge libbrotlienc 1.1.0 hd590300_1 conda-forge libcblas 3.9.0 16_linux64_mkl conda-forge libcrc32c 1.1.2 h9c3ff4c_0 conda-forge libcublas 12.1.0.26 0 nvidia libcudf 24.02.01 cuda12_240213_g33ffdf5b71_0 rapidsai libcufft 11.0.2.4 0 nvidia libcufile 1.6.1.9 hd3aeb46_0 conda-forge libcufile-dev 1.6.1.9 hd3aeb46_0 conda-forge libcurand 10.3.2.106 hd3aeb46_0 conda-forge libcurl 8.5.0 hca28451_0 conda-forge libcusolver 11.4.4.55 0 nvidia libcusparse 12.0.2.55 0 nvidia libdeflate 1.19 hd590300_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 hd590300_2 conda-forge libevent 2.1.12 hf998b51_1 conda-forge libexpat 2.5.0 hcb278e6_1 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 13.2.0 h807b86a_5 conda-forge libgd 2.3.3 h119a65a_9 conda-forge libgfortran-ng 13.2.0 h69a702a_5 conda-forge libgfortran5 13.2.0 ha4646dd_5 conda-forge libglib 2.78.3 h783c2da_0 conda-forge libgomp 13.2.0 h807b86a_5 conda-forge libgoogle-cloud 2.12.0 h5206363_4 conda-forge libgrpc 1.59.3 hd6c4280_0 conda-forge libhwloc 2.9.2 default_h554bfaf_1009 conda-forge libiconv 1.17 hd590300_2 conda-forge libjpeg-turbo 3.0.0 hd590300_1 conda-forge libkvikio 24.02.00 cuda12_240212_g1a2becd_0 rapidsai liblapack 3.9.0 16_linux64_mkl conda-forge libllvm14 14.0.6 hcd5def8_4 conda-forge libllvm15 15.0.7 hb3ce162_4 conda-forge libmrc 24.03.00a cuda_12.1_h0dae25b_13 nvidia/label/dev libnghttp2 1.58.0 h47da74e_1 conda-forge libnl 3.9.0 hd590300_0 conda-forge libnpp 12.0.2.50 0 nvidia libnsl 2.0.1 hd590300_0 conda-forge libntlm 1.4 h7f98852_1002 conda-forge libnuma 2.0.16 h0b41bf4_1 conda-forge libnvjitlink 12.1.105 hd3aeb46_0 conda-forge libnvjpeg 12.1.1.14 0 nvidia libopenblas 0.3.26 pthreads_h413a1c8_0 conda-forge libparquet 14.0.2 habd00f8_3_cuda conda-forge libpng 1.6.42 h2797004_0 conda-forge libprotobuf 4.24.4 hf27288f_0 conda-forge librdkafka 1.9.2 ha5a0de0_2 conda-forge libre2-11 2023.06.02 h7a70373_0 conda-forge librmm 24.02.00 h82930bc_1 conda-forge librsvg 2.56.3 he3f83f7_1 conda-forge libsodium 1.0.18 h36c2ea0_1 conda-forge libsqlite 3.45.1 h2797004_0 conda-forge libssh2 1.11.0 h0841786_0 conda-forge libstdcxx-ng 13.2.0 h7e041cc_5 conda-forge libthrift 0.19.0 hb90f79a_1 conda-forge libtiff 4.6.0 ha9c0a0a_2 conda-forge libutf8proc 2.8.0 h166bdaf_0 conda-forge libuuid 2.38.1 h0b41bf4_0 conda-forge libuv 1.47.0 hd590300_0 conda-forge libwebp 1.3.2 h658648e_1 conda-forge libwebp-base 1.3.2 hd590300_0 conda-forge libxcb 1.15 h0b41bf4_0 conda-forge libxcrypt 4.4.36 hd590300_1 conda-forge libxml2 2.12.5 h232c23b_0 conda-forge libxslt 1.1.39 h76b75d6_0 conda-forge libzlib 1.2.13 hd590300_5 conda-forge llvm-openmp 15.0.7 h0cdce71_0 conda-forge llvmlite 0.42.0 py310h1b8f574_1 conda-forge locket 1.0.0 pyhd8ed1ab_0 conda-forge lxml 5.1.0 py310hcfd0673_0 conda-forge lz4 4.3.3 py310h350c4a5_0 conda-forge lz4-c 1.9.4 hcb278e6_0 conda-forge mako 1.3.2 pyhd8ed1ab_0 conda-forge markdown 3.5.2 pyhd8ed1ab_0 conda-forge markdown-it-py 3.0.0 pyhd8ed1ab_0 conda-forge markupsafe 2.1.5 py310h2372a71_0 conda-forge marshmallow 3.20.2 pyhd8ed1ab_0 conda-forge matplotlib-base 3.8.3 py310h62c0568_0 conda-forge matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mdurl 0.1.2 pyhd8ed1ab_0 conda-forge merlin-core 23.08.00 py_0 nvidia merlin-dataloader 23.08.00 py_0 nvidia milvus 2.3.5 pypi_0 pypi minio 7.2.4 pypi_0 pypi mistune 3.0.2 pyhd8ed1ab_0 conda-forge mkl 2022.1.0 hc2b9512_224 mlflow 2.9.2 py310ha13cd29_0 conda-forge monotonic 1.6 pypi_0 pypi more-itertools 10.2.0 pypi_0 pypi morpheus 24.03 cuda_12.1_py3.10_gaa8d42e7_0 file://localhost/opt/conda/conda-bld mpc 1.3.1 hfe3b2da_0 conda-forge mpfr 4.2.1 h9458935_0 conda-forge mpmath 1.3.0 pyhd8ed1ab_0 conda-forge mrc 24.03.00a cuda_12.1_py310_h572eed8_13 nvidia/label/dev msgpack-python 1.0.7 py310hd41b1e2_0 conda-forge msrest 0.7.1 pypi_0 pypi multidict 6.0.5 py310h2372a71_0 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge mypy-extensions 1.0.0 pypi_0 pypi nb_conda_kernels 2.3.1 pyhd8ed1ab_3 conda-forge nbclassic 1.0.0 pyhb4ecaf3_1 conda-forge nbclient 0.8.0 pyhd8ed1ab_0 conda-forge nbconvert 7.16.0 pyhd8ed1ab_0 conda-forge nbconvert-core 7.16.0 pyhd8ed1ab_0 conda-forge nbconvert-pandoc 7.16.0 pyhd8ed1ab_0 conda-forge nbformat 5.9.2 pyhd8ed1ab_0 conda-forge ncurses 6.4 h59595ed_2 conda-forge nest-asyncio 1.6.0 pyhd8ed1ab_0 conda-forge networkx 2.8.8 pyhd8ed1ab_0 conda-forge newspaper3k 0.2.8 pyhd8ed1ab_3 conda-forge nlohmann_json 3.9.1 h9c3ff4c_1 conda-forge nltk 3.8.1 pyhd8ed1ab_0 conda-forge notebook 6.5.6 pyha770c72_0 conda-forge notebook-shim 0.2.4 pyhd8ed1ab_0 conda-forge npy-append-array 0.9.16 pyhd8ed1ab_0 conda-forge num2words 0.5.13 pypi_0 pypi numba 0.59.0 py310h7dc5dd1_1 conda-forge numpy 1.24.4 py310ha4c1d20_0 conda-forge numpydoc 1.5.0 pyhd8ed1ab_0 conda-forge nvcomp 3.0.5 h10b603f_0 conda-forge nvtabular 23.08.00 py310_0 nvidia nvtx 0.2.8 py310h2372a71_1 conda-forge oauthlib 3.2.2 pyhd8ed1ab_0 conda-forge ocl-icd 2.3.2 hd590300_0 conda-forge onnx 1.15.0 py310h6cf2406_0 conda-forge openjpeg 2.5.0 h488ebb8_3 conda-forge openssl 3.2.1 hd590300_0 conda-forge orc 1.9.2 h4b38347_0 conda-forge ordered-set 4.1.0 pyhd8ed1ab_0 conda-forge overrides 7.7.0 pyhd8ed1ab_0 conda-forge packaging 23.2 pyhd8ed1ab_0 conda-forge pandas 1.5.3 py310h9b08913_1 conda-forge pandoc 3.1.12 ha770c72_0 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge pango 1.50.14 ha41ecd1_2 conda-forge parso 0.8.3 pyhd8ed1ab_0 conda-forge partd 1.4.1 pyhd8ed1ab_0 conda-forge pcre2 10.42 hcad00b1_0 conda-forge pexpect 4.9.0 pyhd8ed1ab_0 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 10.2.0 py310h01dd4db_0 conda-forge pip 24.0 pyhd8ed1ab_0 conda-forge pixman 0.43.2 h59595ed_0 conda-forge pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1 conda-forge platformdirs 4.2.0 pyhd8ed1ab_0 conda-forge pluggy 1.3.0 pyhd8ed1ab_0 conda-forge posthog 3.4.1 pypi_0 pypi prometheus_client 0.20.0 pyhd8ed1ab_0 conda-forge prometheus_flask_exporter 0.23.0 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.42 pyha770c72_0 conda-forge prompthub-py 4.0.0 pypi_0 pypi protobuf 4.24.4 py310h620c231_0 conda-forge psutil 5.9.8 py310h2372a71_0 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge py4j 0.10.9.7 pypi_0 pypi pyarrow 14.0.2 py310h9a2f4d7_3_cuda conda-forge pyarrow-hotfix 0.6 pyhd8ed1ab_0 conda-forge pyasn1 0.5.1 pypi_0 pypi pyasn1-modules 0.3.0 pypi_0 pypi pycparser 2.21 pyhd8ed1ab_0 conda-forge pycryptodome 3.20.0 pypi_0 pypi pydantic 1.10.14 pypi_0 pypi pydantic-core 2.10.1 py310hcb5633a_0 conda-forge pygments 2.17.2 pyhd8ed1ab_0 conda-forge pyjwt 2.8.0 pyhd8ed1ab_1 conda-forge pymilvus 2.3.2 pypi_0 pypi pymupdf 1.23.21 pypi_0 pypi pymupdfb 1.23.9 pypi_0 pypi pynvjitlink 0.1.12 py310hbc2f0e0_0 rapidsai pynvml 11.4.1 pyhd8ed1ab_0 conda-forge pyopenssl 24.0.0 pyhd8ed1ab_0 conda-forge pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge pypdf 3.16.4 pyhd8ed1ab_0 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge python 3.10.13 hd12c33a_1_cpython conda-forge python-confluent-kafka 1.9.2 py310h5764c6d_2 conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-docx 1.1.0 pypi_0 pypi python-dotenv 1.0.1 pyhd8ed1ab_0 conda-forge python-fastjsonschema 2.19.1 pyhd8ed1ab_0 conda-forge python-frontmatter 1.1.0 pypi_0 pypi python-graphviz 0.20.1 pyh22cad53_0 conda-forge python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge python-magic 0.4.27 pypi_0 pypi python-pptx 0.6.23 pypi_0 pypi python-rapidjson 1.14 py310hc6cd4ac_0 conda-forge python_abi 3.10 4_cp310 conda-forge pytorch 2.2.0 py3.10_cuda12.1_cudnn8.9.2_0 pytorch pytorch-cuda 12.1 ha16c6d3_5 pytorch pytorch-mutex 1.0 cuda pytorch pytz 2023.4 pyhd8ed1ab_0 conda-forge pywin32-on-windows 0.1.0 pyh1179c8e_3 conda-forge pyyaml 6.0.1 py310h2372a71_1 conda-forge pyzmq 24.0.1 py310h330234f_1 conda-forge qpd 0.4.4 pyhd8ed1ab_1 conda-forge quantulum3 0.9.0 pypi_0 pypi querystring_parser 1.2.4 py_0 conda-forge rank-bm25 0.2.2 pypi_0 pypi rapids-dask-dependency 24.02.00a11 0 rapidsai-nightly rdma-core 50.0 hd3aeb46_0 conda-forge re2 2023.06.02 h2873b5e_0 conda-forge readline 8.2 h8228510_1 conda-forge referencing 0.33.0 pyhd8ed1ab_0 conda-forge regex 2023.12.25 py310h2372a71_0 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge requests-cache 0.9.8 pypi_0 pypi requests-file 2.0.0 pyhd8ed1ab_0 conda-forge requests-oauthlib 1.3.1 pypi_0 pypi rfc3339-validator 0.1.4 pyhd8ed1ab_0 conda-forge rfc3986-validator 0.1.1 pyh9f0ad1d_0 conda-forge rich 13.7.0 pyhd8ed1ab_0 conda-forge rmm 24.02.00 cuda12_py310_240212_g09b406c1_0 rapidsai rpds-py 0.18.0 py310hcb5633a_0 conda-forge rsa 4.9 pypi_0 pypi s2n 1.4.1 h06160fa_0 conda-forge safetensors 0.4.2 pypi_0 pypi scikit-learn 1.3.2 py310h1fdf081_2 conda-forge scipy 1.12.0 py310hb13e2d6_2 conda-forge send2trash 1.8.2 pyh41d4057_0 conda-forge sentence-transformers 2.3.0 pypi_0 pypi sentencepiece 0.1.99 pypi_0 pypi setuptools 69.0.3 pyhd8ed1ab_0 conda-forge sgmllib3k 1.0.0 pyh9f0ad1d_0 conda-forge simpervisor 1.0.0 pypi_0 pypi six 1.16.0 pyh6c4a22f_0 conda-forge sleef 3.5.1 h9b69904_2 conda-forge smmap 5.0.0 pyhd8ed1ab_0 conda-forge snappy 1.1.10 h9fff704_0 conda-forge sniffio 1.3.0 pyhd8ed1ab_0 conda-forge snowballstemmer 2.2.0 pyhd8ed1ab_0 conda-forge sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge soupsieve 2.5 pyhd8ed1ab_1 conda-forge spdlog 1.12.0 hd2e6256_2 conda-forge sphinx 7.2.6 pyhd8ed1ab_0 conda-forge sphinxcontrib-applehelp 1.0.8 pyhd8ed1ab_0 conda-forge sphinxcontrib-devhelp 1.0.6 pyhd8ed1ab_0 conda-forge sphinxcontrib-htmlhelp 2.0.5 pyhd8ed1ab_0 conda-forge sphinxcontrib-jsmath 1.0.1 pyhd8ed1ab_0 conda-forge sphinxcontrib-qthelp 1.0.7 pyhd8ed1ab_0 conda-forge sphinxcontrib-serializinghtml 1.1.10 pyhd8ed1ab_0 conda-forge sqlalchemy 1.4.49 py310h2372a71_1 conda-forge sqlglot 21.1.0 pyhd8ed1ab_0 conda-forge sqlparse 0.4.4 pyhd8ed1ab_0 conda-forge sseclient-py 1.8.0 pypi_0 pypi stack_data 0.6.2 pyhd8ed1ab_0 conda-forge stringcase 1.2.0 py_0 conda-forge sympy 1.12 pypyh9d50eac_103 conda-forge tabulate 0.9.0 pyhd8ed1ab_1 conda-forge tblib 3.0.0 pyhd8ed1ab_0 conda-forge tenacity 8.2.3 pypi_0 pypi tensorflow-metadata 1.13.1 pyhd8ed1ab_0 conda-forge terminado 0.18.0 pyh0d859eb_0 conda-forge threadpoolctl 3.3.0 pyhc1e730c_0 conda-forge tika 2.6.0 pypi_0 pypi tiktoken 0.6.0 pypi_0 pypi tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge tinysegmenter 0.3 pyhd8ed1ab_0 conda-forge tk 8.6.13 noxft_h4845f30_101 conda-forge tldextract 5.1.1 pyhd8ed1ab_0 conda-forge tokenizers 0.15.2 pypi_0 pypi tomli 2.0.1 pypi_0 pypi toolz 0.12.1 pyhd8ed1ab_0 conda-forge torchtriton 2.2.0 py310 pytorch tornado 6.3.3 py310h2372a71_1 conda-forge tqdm 4.66.2 pyhd8ed1ab_0 conda-forge traitlets 5.14.1 pyhd8ed1ab_0 conda-forge transformers 4.36.2 pypi_0 pypi triad 0.9.5 pyhd8ed1ab_0 conda-forge tritonclient 2.34.0 pyhff2d567_0 conda-forge types-python-dateutil 2.8.19.20240106 pyhd8ed1ab_0 conda-forge typing-extensions 4.9.0 hd8ed1ab_0 conda-forge typing-inspect 0.9.0 pypi_0 pypi typing_extensions 4.9.0 pyha770c72_0 conda-forge typing_utils 0.1.0 pyhd8ed1ab_0 conda-forge tzdata 2024a h0c530f3_0 conda-forge ucx 1.15.0 h6d2d1ec_3 conda-forge ujson 5.8.0 py310hc6cd4ac_0 conda-forge unicodedata2 15.1.0 py310h2372a71_0 conda-forge uri-template 1.3.0 pyhd8ed1ab_0 conda-forge url-normalize 1.4.3 pyhd8ed1ab_0 conda-forge urllib3 2.2.0 pyhd8ed1ab_0 conda-forge watchdog 3.0.0 py310hff52083_1 conda-forge wcwidth 0.2.13 pyhd8ed1ab_0 conda-forge webcolors 1.13 pyhd8ed1ab_0 conda-forge webencodings 0.5.1 pyhd8ed1ab_2 conda-forge websocket-client 1.7.0 pyhd8ed1ab_0 conda-forge websockets 12.0 py310h2372a71_0 conda-forge werkzeug 3.0.1 pyhd8ed1ab_0 conda-forge wheel 0.42.0 pyhd8ed1ab_0 conda-forge widgetsnbextension 4.0.10 pyhd8ed1ab_0 conda-forge xlsxwriter 3.1.9 pypi_0 pypi xorg-kbproto 1.0.7 h7f98852_1002 conda-forge xorg-libice 1.1.1 hd590300_0 conda-forge xorg-libsm 1.2.4 h7391055_0 conda-forge xorg-libx11 1.8.7 h8ee46fc_0 conda-forge xorg-libxau 1.0.11 hd590300_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xorg-libxext 1.3.4 h0b41bf4_2 conda-forge xorg-libxrender 0.9.11 hd590300_0 conda-forge xorg-renderproto 0.11.1 h7f98852_1002 conda-forge xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge xorg-xproto 7.0.31 h7f98852_1007 conda-forge xyzservices 2023.10.1 pyhd8ed1ab_0 conda-forge xz 5.2.6 h166bdaf_0 conda-forge y-py 0.6.2 pypi_0 pypi yaml 0.2.5 h7f98852_2 conda-forge yarl 1.9.4 pypi_0 pypi ypy-websocket 0.8.4 pypi_0 pypi zeromq 4.3.5 h59595ed_0 conda-forge zict 3.0.0 pyhd8ed1ab_0 conda-forge zipp 3.17.0 pyhd8ed1ab_0 conda-forge zlib 1.2.13 hd590300_5 conda-forge zstd 1.5.5 hfc55251_0 conda-forgeOther/Misc.
In the minimum reproducible example, turn
use_cpptoTrueandFalseto explore the behavior.Code of Conduct