[BUG] Analysis of pytest execution difference between running tests locally vs CI

[dantegd]

Describe the bug This issue is not really a bug, but more of a description of the results of a quick analysis of why sometimes some tests that locally work super fast, end up in the list of the slowest pytests by orders of magnitude. It probably does not explain entirely why does this happen since it was a naive fast test. But is a good place to start the analysis, and can serve as a good tool for developers to analyze tests locally and reproduce CI conditions.

The spoiler (TL;DR) of the issue is: pytests that have CPU processing (particularly if it is multi-threaded) get slowed down immensely , up to 2 orders of magnitude, more than pytests that are pure GPU (or close to) bound when a system is loaded by other tasks. CI probably is exhibiting and magnifying this behavior.

This started because locally I was seeing this:

1.00s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-50-float32]

while CI showed this:

58.32s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-50-float32]

which ended up delaying the merge of #3126

Steps/Code to reproduce bug and full results I built cuML from source, but using nightly conda packages should work just as well. First I ran the test suite on my workstation (details at the end of the issue, but quick specs: 3950x 16 core CPU, 3080 10GB GPU, 64 GB RAM), and got the usual fast results. Then I created 31 processes that load the CPU (naive simple yes > /dev/null &) and reniced them (with different priorities, but for the worst result it was a negative value) and then ran the test suite. This was to emulate the fact that CI can be very loaded (beyond CPU, the hardware probably is heavily loaded in RAM, bandwidth and disk usage beyond my naive test).

1. test_kernel_shap_standalone: in this tests kernel shap does l1_regularization using Scikit-learn's lars_path, that even if its only one part of the algorithm, it is not only is CPU based, but is heavily multi threaded.

Results for system without additional load:

1.05s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-100-50-float32]
1.00s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-50-float32]
0.97s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-100-50-float32]
0.94s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-50-float64]
0.94s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-100-50-float64]
0.92s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-10-50-float32]
0.91s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-100-50-float64]
0.88s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-10-50-float64]
0.47s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-100-20-float64]
0.47s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-100-20-float64]
0.46s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-20-float64]
0.45s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-100-20-float32]
0.44s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-100-20-float32]
0.44s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-20-float32]
0.43s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-10-20-float32]
0.43s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-10-20-float64]

Results for system with the yes load:

20.17s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-10-50-float64]
19.04s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-100-50-float64]
18.99s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-50-float64]
18.31s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-10-50-float32]
18.27s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-100-50-float32]
18.22s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-50-float32]
17.99s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-100-50-float32]
17.17s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-100-50-float64]
12.57s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-10-20-float32]
12.00s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-100-20-float32]
11.73s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-100-20-float32]
11.38s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-10-20-float64]
11.04s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[PCA-100-20-float64]
10.54s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-100-20-float64]
9.47s call     cuml/test/experimental/test_explainer_kernel_shap.py::test_kernel_shap_standalone[TruncatedSVD-10-20-float64]

2. test_explainer_permutation_shap: permutation SHAP is pretty much pure GPU code when explaining cuML models, so this contrasts siginificantly with the above.

Results with no load:

0.57s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-50-50-float64]
0.35s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-10-50-float64]
0.25s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-50-11-float64]
0.24s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-50-50-float32]
0.23s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-10-11-float64]
0.21s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-50-50-float64]
0.20s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-10-50-float32]
0.19s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-10-11-float32]
0.18s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-50-11-float32]
0.17s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-10-50-float64]
0.17s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-50-50-float32]
0.17s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-50-11-float64]
0.17s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-10-11-float32]
0.17s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-10-50-float32]
0.16s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-50-11-float32]
0.16s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-10-11-float64]

Results with yes load:

0.54s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-10-50-float64]
0.50s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-50-50-float64]
0.43s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-10-50-float64]
0.39s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-50-11-float64]
0.38s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-10-11-float32]
0.36s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-10-11-float64]
0.35s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-50-50-float32]
0.35s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-50-11-float32]
0.32s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-50-50-float32]
0.31s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-SVR-10-50-float32]
0.27s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-10-11-float32]
0.27s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-10-11-float64]
0.27s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-10-50-float32]
0.26s call     cuml/test/experimental/test_explainer_permutation_shap.py::test_different_parameters[50-LinearRegression-50-11-float64]

As can be seen, there is some impact on the permutation SHAP (pure GPU based) pytests, but the impact on Kernel SHAP was way higher.

Environment details (please complete the following information):

• Environment location: Bare metal
• Linux Distro/Architecture: Ubuntu 20.04
• GPU Model/Driver: 3080 (laptop with 2070S showed similar behavior, but numbers are from 3080)
• CUDA: 11.0 (and 10.2 on laptop)
• Method of cuDF & cuML install: cuDF 0.17 nightly, cuML from source

Additional context Full environment details:

Click here to see environment details

     **git***
     commit 2e0374f3d4db6d1cf51e0e203239a946a1e6a1c3 (HEAD -> 017-fea-kshap)
     Author: Dante Gama Dessavre 
     Date:   Tue Dec 1 13:55:16 2020 -0600

     FIX clang format fixes
     **git submodules***

     ***OS Information***
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=20.04
     DISTRIB_CODENAME=focal
     DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"
     NAME="Ubuntu"
     VERSION="20.04.1 LTS (Focal Fossa)"
     ID=ubuntu
     ID_LIKE=debian
     PRETTY_NAME="Ubuntu 20.04.1 LTS"
     VERSION_ID="20.04"
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     VERSION_CODENAME=focal
     UBUNTU_CODENAME=focal
     Linux schwifty1 5.4.0-53-generic #59-Ubuntu SMP Wed Oct 21 09:38:44 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Tue Dec  1 18:50:50 2020
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 455.23.04    Driver Version: 455.23.04    CUDA Version: 11.1     |
     |-------------------------------+----------------------+----------------------+
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |                               |                      |               MIG M. |
     |===============================+======================+======================|
     |   0  GeForce RTX 3080    Off  | 00000000:0B:00.0  On |                  N/A |
     |  0%   56C    P5    31W / 320W |    984MiB /  9984MiB |     16%      Default |
     |                               |                      |                  N/A |
     +-------------------------------+----------------------+----------------------+

     +-----------------------------------------------------------------------------+
     | Processes:                                                                  |
     |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
     |        ID   ID                                                   Usage      |
     |=============================================================================|
     |    0   N/A  N/A      1427      G   /usr/lib/xorg/Xorg                192MiB |
     |    0   N/A  N/A      2181      G   /usr/lib/xorg/Xorg                569MiB |
     |    0   N/A  N/A      2309      G   /usr/bin/gnome-shell              128MiB |
     |    0   N/A  N/A     32808      G   .../debug.log --shared-files       15MiB |
     |    0   N/A  N/A     41931      G   ...AAAAAAAA== --shared-files       28MiB |
     +-----------------------------------------------------------------------------+

     ***CPU***
     Architecture:                    x86_64
     CPU op-mode(s):                  32-bit, 64-bit
     Byte Order:                      Little Endian
     Address sizes:                   43 bits physical, 48 bits virtual
     CPU(s):                          32
     On-line CPU(s) list:             0-31
     Thread(s) per core:              2
     Core(s) per socket:              16
     Socket(s):                       1
     NUMA node(s):                    1
     Vendor ID:                       AuthenticAMD
     CPU family:                      23
     Model:                           113
     Model name:                      AMD Ryzen 9 3950X 16-Core Processor
     Stepping:                        0
     Frequency boost:                 disabled
     CPU MHz:                         3999.380
     CPU max MHz:                     4700.0000
     CPU min MHz:                     2200.0000
     BogoMIPS:                        7999.42
     Virtualization:                  AMD-V
     L1d cache:                       512 KiB
     L1i cache:                       512 KiB
     L2 cache:                        8 MiB
     L3 cache:                        64 MiB
     NUMA node0 CPU(s):               0-31
     Vulnerability Itlb multihit:     Not affected
     Vulnerability L1tf:              Not affected
     Vulnerability Mds:               Not affected
     Vulnerability Meltdown:          Not affected
     Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
     Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
     Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, STIBP conditional, RSB filling
     Vulnerability Srbds:             Not affected
     Vulnerability Tsx async abort:   Not affected
     Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd mba sev ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca

     ***CMake***
     /home/galahad/miniconda3/envs/ns1116/bin/cmake
     cmake version 3.18.4

     CMake suite maintained and supported by Kitware (kitware.com/cmake).

     ***g++***
     /usr/bin/g++
     g++ (Ubuntu 8.4.0-3ubuntu2) 8.4.0
     Copyright (C) 2018 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

     ***nvcc***
     /usr/local/cuda-11.1/bin/nvcc
     nvcc: NVIDIA (R) Cuda compiler driver
     Copyright (c) 2005-2020 NVIDIA Corporation
     Built on Tue_Sep_15_19:10:02_PDT_2020
     Cuda compilation tools, release 11.1, V11.1.74
     Build cuda_11.1.TC455_06.29069683_0

     ***Python***
     /home/galahad/miniconda3/envs/ns1116/bin/python
     Python 3.8.6

     ***Environment Variables***
     PATH                            : /usr/local/cuda-11.1/bin:/usr/local/cuda-11.1/NsightCompute-2020.3.2:/home/galahad/miniconda3/envs/ns1116/bin:/home/galahad/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
     LD_LIBRARY_PATH                 : /usr/local/cuda-11.1/lib64
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    : /home/galahad/miniconda3/envs/ns1116
     PYTHON_PATH                     :

     ***conda packages***
     /home/galahad/miniconda3/envs/ns1116/bin/conda
     # packages in environment at /home/galahad/miniconda3/envs/ns1116:
     #
     # Name                    Version                   Build  Channel
     _libgcc_mutex             0.1                 conda_forge    conda-forge
     _openmp_mutex             4.5                      1_llvm    conda-forge
     abseil-cpp                20200225.2           he1b5a44_2    conda-forge
     aiohttp                   3.7.2            py38h1e0a361_0    conda-forge
     alabaster                 0.7.12                     py_0    conda-forge
     appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
     argon2-cffi               20.1.0           py38h25fe258_2    conda-forge
     arrow-cpp                 1.0.1           py38h9444e92_15_cuda    conda-forge
     arrow-cpp-proc            2.0.0                      cuda    conda-forge
     asn1crypto                1.4.0              pyh9f0ad1d_0    conda-forge
     async-timeout             3.0.1                   py_1000    conda-forge
     async_generator           1.10                       py_0    conda-forge
     atk                       2.36.0               ha770c72_4    conda-forge
     atk-1.0                   2.36.0               h0d5b62e_4    conda-forge
     attrs                     20.3.0             pyhd3deb0d_0    conda-forge
     autoconf                  2.69            pl5320h36c2ea0_10    conda-forge
     automake                  1.16.2          pl5320ha770c72_3    conda-forge
     autopep8                  1.5.4              pyh9f0ad1d_0    conda-forge
     aws-c-common              0.4.59               h36c2ea0_1    conda-forge
     aws-c-event-stream        0.1.6                had2084c_6    conda-forge
     aws-checksums             0.1.10               h4e93380_0    conda-forge
     aws-sam-translator        1.27.0             pyh9f0ad1d_0    conda-forge
     aws-sdk-cpp               1.8.70               h57dc084_1    conda-forge
     aws-xray-sdk              0.95                       py_0    conda-forge
     babel                     2.9.0              pyhd3deb0d_0    conda-forge
     backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
     backports                 1.0                        py_2    conda-forge
     backports.functools_lru_cache 1.6.1                      py_0    conda-forge
     backports.tempfile        1.0                        py_0    conda-forge
     backports.weakref         1.0.post1       py38h32f6830_1002    conda-forge
     beautifulsoup4            4.9.3              pyhb0f4dca_0    conda-forge
     benchmark                 1.5.1                he1b5a44_2    conda-forge
     black                     19.10b0                    py_4    conda-forge
     blas                      2.17                   openblas    conda-forge
     bleach                    3.2.1              pyh9f0ad1d_0    conda-forge
     blinker                   1.4                        py_1    conda-forge
     bokeh                     2.2.3            py38h32f6830_0    conda-forge
     boost                     1.72.0           py38h1e42940_1    conda-forge
     boost-cpp                 1.72.0               h8e57a91_0    conda-forge
     boto                      2.49.0                     py_0    conda-forge
     boto3                     1.16.18            pyhd8ed1ab_0    conda-forge
     botocore                  1.19.18            pyhd8ed1ab_0    conda-forge
     brotli                    1.0.9                he1b5a44_3    conda-forge
     brotlipy                  0.7.0           py38h8df0ef7_1001    conda-forge
     bzip2                     1.0.8                h516909a_3    conda-forge
     c-ares                    1.16.1               h516909a_3    conda-forge
     ca-certificates           2020.11.8            ha878542_0    conda-forge
     cachetools                4.1.1                      py_0    conda-forge
     cairo                     1.16.0            hcf35c78_1003    conda-forge
     certifi                   2020.11.8        py38h578d9bd_0    conda-forge
     cffi                      1.14.3           py38h1bdcb99_1    conda-forge
     cfitsio                   3.470                h2e3daa1_7    conda-forge
     cfn-lint                  0.40.0           py38h578d9bd_0    conda-forge
     chardet                   3.0.4           py38h924ce5b_1008    conda-forge
     clang                     8.0.1                hc9558a2_2    conda-forge
     clang-tools               8.0.1                hc9558a2_2    conda-forge
     clangxx                   8.0.1                         2    conda-forge
     click                     7.1.2              pyh9f0ad1d_0    conda-forge
     click-plugins             1.1.1                      py_0    conda-forge
     cligj                     0.7.0                      py_0    conda-forge
     cloudpickle               1.6.0                      py_0    conda-forge
     cmake                     3.18.4               h1f3970d_0    conda-forge
     cmake-format              0.6.11             pyh9f0ad1d_0    conda-forge
     cmake_setuptools          0.1.3                      py_0    rapidsai
     cmarkgfm                  0.4.2            py38h1e0a361_3    conda-forge
     colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
     colorcet                  2.0.1                      py_0    conda-forge
     commonmark                0.9.1                      py_0    conda-forge
     conda                     4.8.3            py38h32f6830_2    conda-forge
     conda-build               3.20.3           py38h32f6830_0    conda-forge
     conda-package-handling    1.7.2            py38h8df0ef7_0    conda-forge
     conda-verify              3.1.1           py38h32f6830_1002    conda-forge
     cookies                   2.2.1                      py_0    conda-forge
     coverage                  5.3              py38h1e0a361_1    conda-forge
     cryptography              3.2.1            py38h7699a38_0    conda-forge
     cudatoolkit               11.0.221             h6bb024c_0    nvidia
     cudf                      0.17.0a201116   cuda_11.0_py38_g21b28ba7bb_262    rapidsai-nightly
     cudnn                     8.0.0                cuda11.0_0    nvidia
     cuml                      0.17.0a0+182.g2170217ba          pypi_0    pypi
     cupy                      7.8.0            py38hb7c6141_0    rapidsai
     curl                      7.71.1               he644dc0_8    conda-forge
     cycler                    0.10.0                     py_2    conda-forge
     cyrus-sasl                2.1.27               h063b49f_1    conda-forge
     cython                    0.29.21          py38h348cfbe_1    conda-forge
     cytoolz                   0.11.0           py38h1e0a361_1    conda-forge
     dask                      2.30.0+57.g3e264491          pypi_0    pypi
     dask-core                 2.30.0                     py_0    conda-forge
     dask-cuda                 0.17.0a201116           py38_42    rapidsai-nightly
     dask-cudf                 0.17.0a201116   py38_g21b28ba7bb_262    rapidsai-nightly
     dask-glm                  0.2.0                      py_1    conda-forge
     dask-labextension         3.0.0                      py_0    conda-forge
     dask-ml                   1.7.0                      py_0    conda-forge
     datashader                0.11.1             pyh9f0ad1d_0    conda-forge
     datashape                 0.5.4                      py_1    conda-forge
     dbus                      1.13.6               hfdff14a_1    conda-forge
     decorator                 4.4.2                      py_0    conda-forge
     defusedxml                0.6.0                      py_0    conda-forge
     distributed               2.31.0.dev0+31.g62eb3372          pypi_0    pypi
     dlpack                    0.3                  he1b5a44_1    conda-forge
     docker-py                 4.3.1            py38h32f6830_1    conda-forge
     docker-pycreds            0.4.0                      py_0    conda-forge
     docutils                  0.16             py38h924ce5b_2    conda-forge
     double-conversion         3.1.5                he1b5a44_2    conda-forge
     doxygen                   1.8.20               h0e019cf_0    conda-forge
     ecdsa                     0.16.1             pyhd8ed1ab_0    conda-forge
     entrypoints               0.3             pyhd8ed1ab_1003    conda-forge
     expat                     2.2.9                he1b5a44_2    conda-forge
     fa2                       0.3.5            py38h1e0a361_0    conda-forge
     faiss-proc                1.0.0                      cuda    rapidsai-nightly
     fancycompleter            0.9.1           py38h32f6830_1002    conda-forge
     fastavro                  1.1.0            py38h25fe258_0    conda-forge
     fastrlock                 0.5              py38h950e882_1    conda-forge
     feather-format            0.4.1              pyh9f0ad1d_0    conda-forge
     filelock                  3.0.12             pyh9f0ad1d_0    conda-forge
     filterpy                  1.4.5                      py_1    conda-forge
     fiona                     1.8.13           py38h033e0f6_1    conda-forge
     flake8                    3.8.4                      py_0    conda-forge
     flask                     1.1.2              pyh9f0ad1d_0    conda-forge
     flatbuffers               1.10.0            hf484d3e_1002    conda-forge
     font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
     font-ttf-inconsolata      2.001                hab24e00_0    conda-forge
     font-ttf-source-code-pro  2.030                hab24e00_0    conda-forge
     font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
     fontconfig                2.13.1            h86ecdb6_1001    conda-forge
     fonts-conda-ecosystem     1                             0    conda-forge
     fonts-conda-forge         1                             0    conda-forge
     freetype                  2.10.4               h7ca028e_0    conda-forge
     freexl                    1.0.5             h516909a_1002    conda-forge
     fribidi                   1.0.10               h36c2ea0_0    conda-forge
     fsspec                    0.8.4                      py_0    conda-forge
     future                    0.18.2           py38h578d9bd_2    conda-forge
     gcsfs                     0.7.1                      py_0    conda-forge
     gdal                      3.0.4           py38h172510d_10    conda-forge
     gdk-pixbuf                2.42.0               h0536704_0    conda-forge
     geopandas                 0.8.1                      py_0    conda-forge
     geos                      3.8.1                he1b5a44_0    conda-forge
     geotiff                   1.6.0                h05acad5_0    conda-forge
     gettext                   0.19.8.1          hf34092f_1004    conda-forge
     gflags                    2.2.2             he1b5a44_1004    conda-forge
     giflib                    5.2.1                h36c2ea0_2    conda-forge
     git                       2.29.2          pl5320h78be0e9_1    conda-forge
     glib                      2.66.2               h58526e2_0    conda-forge
     glob2                     0.7                        py_0    conda-forge
     glog                      0.4.0                h49b9bf7_3    conda-forge
     gmock                     1.10.0               ha770c72_4    conda-forge
     gmp                       6.2.0                h58526e2_4    conda-forge
     gobject-introspection     1.66.1           py38h4eacb9c_3    conda-forge
     google-auth               1.23.0             pyhd8ed1ab_0    conda-forge
     google-auth-oauthlib      0.4.2              pyhd8ed1ab_0    conda-forge
     graphite2                 1.3.13            h58526e2_1001    conda-forge
     graphviz                  2.42.3               h6939c30_2    conda-forge
     grpc-cpp                  1.33.2               h1870a98_1    conda-forge
     gtest                     1.10.0               h0efe328_4    conda-forge
     gtk2                      2.24.32              h586f36d_1    conda-forge
     gts                       0.7.6                h17b2bb4_1    conda-forge
     harfbuzz                  2.4.0                h9f30f68_3    conda-forge
     hdf4                      4.2.13            hf30be14_1003    conda-forge
     hdf5                      1.10.6          nompi_h54c07f9_1110    conda-forge
     heapdict                  1.0.1                      py_0    conda-forge
     holoviews                 1.13.5             pyh9f0ad1d_0    conda-forge
     httpretty                 1.0.2                      py_0    conda-forge
     hypothesis                5.41.2             pyhd8ed1ab_0    conda-forge
     icu                       64.2                 he1b5a44_1    conda-forge
     idna                      2.8                   py38_1000    conda-forge
     imagesize                 1.2.0                      py_0    conda-forge
     importlib-metadata        2.0.0                      py_1    conda-forge
     importlib_metadata        2.0.0                         1    conda-forge
     iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
     ipykernel                 5.3.4            py38h81c977d_1    conda-forge
     ipython                   7.15.0           py38h32f6830_0    conda-forge
     ipython_genutils          0.2.0                      py_1    conda-forge
     ipywidgets                7.5.1              pyh9f0ad1d_1    conda-forge
     isort                     5.0.9            py38h32f6830_0    conda-forge
     itsdangerous              1.1.0                      py_0    conda-forge
     jedi                      0.17.2           py38h578d9bd_1    conda-forge
     jeepney                   0.5.0              pyhd8ed1ab_0    conda-forge
     jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge
     jmespath                  0.10.0             pyh9f0ad1d_0    conda-forge
     joblib                    0.17.0                     py_0    conda-forge
     jpeg                      9d                   h36c2ea0_0    conda-forge
     json-c                    0.13.1            hbfbb72e_1002    conda-forge
     json5                     0.9.5              pyh9f0ad1d_0    conda-forge
     jsondiff                  1.1.2                      py_0    conda-forge
     jsonpatch                 1.24                       py_0    conda-forge
     jsonpickle                1.4.1              pyh9f0ad1d_0    conda-forge
     jsonpointer               2.0                        py_0    conda-forge
     jsonschema                3.2.0                      py_2    conda-forge
     junit-xml                 1.9                pyh9f0ad1d_0    conda-forge
     jupyter-server-proxy      1.5.0                      py_0    conda-forge
     jupyter_client            6.1.7                      py_0    conda-forge
     jupyter_core              4.6.3            py38h578d9bd_2    conda-forge
     jupyter_sphinx            0.3.1            py38h32f6830_1    conda-forge
     jupyterlab                2.1.5                      py_0    conda-forge
     jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
     jupyterlab_server         1.2.0                      py_0    conda-forge
     kealib                    1.4.13               h33137a7_1    conda-forge
     keyring                   21.5.0           py38h578d9bd_0    conda-forge
     kiwisolver                1.3.1            py38h82cb98a_0    conda-forge
     krb5                      1.17.1               hfafb76e_3    conda-forge
     lapack                    3.6.1                ha44fe06_2    conda-forge
     lcms2                     2.11                 hcbb858e_1    conda-forge
     ld_impl_linux-64          2.35.1               hed1e6ac_0    conda-forge
     libarchive                3.4.3                hf837322_0    conda-forge
     libblas                   3.8.0               17_openblas    conda-forge
     libcblas                  3.8.0               17_openblas    conda-forge
     libcudf                   0.17.0a201116   cuda11.0_g21b28ba7bb_262    rapidsai-nightly
     libcumlprims              0.17.0a201030   cuda11.0_g1fa28a5_8    rapidsai-nightly
     libcurl                   7.71.1               hcdd3856_8    conda-forge
     libcypher-parser          0.6.2                         1    rapidsai
     libdap4                   3.20.6               h1d1bd15_1    conda-forge
     libedit                   3.1.20191231         he28a2e2_2    conda-forge
     libev                     4.33                 h516909a_1    conda-forge
     libevent                  2.1.10               hcdb4288_3    conda-forge
     libfaiss                  1.6.3           h328c4c8_1_cuda    rapidsai
     libffi                    3.2.1             he1b5a44_1007    conda-forge
     libgcc-ng                 7.5.0               h5dbcf3e_17    conda-forge
     libgcrypt                 1.8.7                h36c2ea0_0    conda-forge
     libgdal                   3.0.4               he6a97d6_10    conda-forge
     libgfortran               3.0.0                         1    conda-forge
     libgfortran-ng            7.5.0               hae1eefd_17    conda-forge
     libgfortran4              7.5.0               hae1eefd_17    conda-forge
     libglib                   2.66.2               hbe7bbb4_0    conda-forge
     libgpg-error              1.39                 he1b5a44_0    conda-forge
     libgsasl                  1.8.0                         2    conda-forge
     libhwloc                  2.3.0                h3c4fd83_0    conda-forge
     libiconv                  1.16                 h516909a_0    conda-forge
     libkml                    1.3.0             hd79254b_1012    conda-forge
     liblapack                 3.8.0               17_openblas    conda-forge
     liblapacke                3.8.0               17_openblas    conda-forge
     liblief                   0.10.1               he1b5a44_2    conda-forge
     libllvm10                 10.0.1               he513fc3_3    conda-forge
     libllvm8                  8.0.1                hc9558a2_0    conda-forge
     libnetcdf                 4.7.4           nompi_hefab0ff_106    conda-forge
     libnghttp2                1.41.0               h8cfc5f6_2    conda-forge
     libntlm                   1.4               h516909a_1002    conda-forge
     libopenblas               0.3.10          pthreads_hb3c22a3_5    conda-forge
     libpng                    1.6.37               h21135ba_2    conda-forge
     libpq                     12.3                 h5513abc_2    conda-forge
     libprotobuf               3.13.0.1             h8b12597_0    conda-forge
     librdkafka                1.5.0                h40bdf00_0    conda-forge
     librmm                    0.17.0a201106   cuda11.0_gb1ac445_43    rapidsai-nightly
     libsodium                 1.0.18               h36c2ea0_1    conda-forge
     libspatialindex           1.9.3                he1b5a44_3    conda-forge
     libspatialite             4.3.0a            h2482549_1038    conda-forge
     libssh2                   1.9.0                hab1572f_5    conda-forge
     libstdcxx-ng              7.5.0               h2ae2ef3_17    conda-forge
     libthrift                 0.13.0               h5aa387f_6    conda-forge
     libtiff                   4.1.0                h4f3a223_6    conda-forge
     libtool                   2.4.6             h58526e2_1007    conda-forge
     libutf8proc               2.5.0                h516909a_2    conda-forge
     libuuid                   2.32.1            h14c3975_1000    conda-forge
     libuv                     1.34.0               h516909a_0    conda-forge
     libwebp                   1.1.0                h76fa15c_4    conda-forge
     libwebp-base              1.1.0                h36c2ea0_3    conda-forge
     libxcb                    1.13              h14c3975_1002    conda-forge
     libxml2                   2.9.10               hee79883_0    conda-forge
     lightgbm                  3.0.0            py38hadf7658_2    conda-forge
     llvm-openmp               11.0.0               hfc4b9b4_1    conda-forge
     llvmlite                  0.34.0           py38h4f45e52_2    conda-forge
     locket                    0.2.0                      py_2    conda-forge
     lz4-c                     1.9.2                he1b5a44_3    conda-forge
     lzo                       2.10              h516909a_1000    conda-forge
     m4                        1.4.18            h516909a_1001    conda-forge
     make                      4.3                  hd18ef5c_1    conda-forge
     markdown                  3.3.3              pyh9f0ad1d_0    conda-forge
     markupsafe                1.1.1            py38h8df0ef7_2    conda-forge
     matplotlib-base           3.3.3            py38h5c7f4ab_0    conda-forge
     mccabe                    0.6.1                      py_1    conda-forge
     mimesis                   4.0.0              pyh9f0ad1d_0    conda-forge
     mistune                   0.8.4           py38h25fe258_1002    conda-forge
     mock                      4.0.2            py38h32f6830_1    conda-forge
     more-itertools            8.6.0              pyhd8ed1ab_0    conda-forge
     moto                      1.3.14                     py_0    conda-forge
     msgpack-python            1.0.0            py38hbf85e49_2    conda-forge
     multidict                 4.7.5            py38h1e0a361_2    conda-forge
     multipledispatch          0.6.0                      py_0    conda-forge
     munch                     2.5.0                      py_0    conda-forge
     mypy                      0.782                      py_0    conda-forge
     mypy_extensions           0.4.3            py38h32f6830_2    conda-forge
     nbclient                  0.5.1                      py_0    conda-forge
     nbconvert                 6.0.7            py38h578d9bd_3    conda-forge
     nbformat                  5.0.8                      py_0    conda-forge
     nbsphinx                  0.7.1              pyh9f0ad1d_0    conda-forge
     nccl                      2.7.8.1            h4962215_100    nvidia
     ncurses                   6.2                  h58526e2_4    conda-forge
     nest-asyncio              1.4.3              pyhd8ed1ab_0    conda-forge
     networkx                  2.5                        py_0    conda-forge
     nltk                      3.4.4                      py_0    conda-forge
     nodejs                    13.13.0              hf5d1a2b_0    conda-forge
     notebook                  6.1.5            py38h578d9bd_0    conda-forge
     numba                     0.51.2           py38hc5bc63f_0    conda-forge
     numpy                     1.19.4           py38hf0fd68c_1    conda-forge
     numpydoc                  1.1.0                      py_1    conda-forge
     nvtx                      0.2.1            py38h1e0a361_2    conda-forge
     oauthlib                  3.0.1                      py_0    conda-forge
     olefile                   0.46               pyh9f0ad1d_1    conda-forge
     openjpeg                  2.3.1                h981e76c_3    conda-forge
     openssl                   1.1.1h               h516909a_0    conda-forge
     orc                       1.6.5                hd3605a7_0    conda-forge
     packaging                 20.4               pyh9f0ad1d_0    conda-forge
     pandas                    1.1.4            py38h0ef3d22_0    conda-forge
     pandoc                    1.19.2                        0    conda-forge
     pandocfilters             1.4.2                      py_1    conda-forge
     panel                     0.10.1             pyhd8ed1ab_1    conda-forge
     pango                     1.42.4               h7062337_4    conda-forge
     param                     1.10.0                     py_0    conda-forge
     parquet-cpp               1.5.1                         2    conda-forge
     parso                     0.7.1              pyh9f0ad1d_0    conda-forge
     partd                     1.1.0                      py_0    conda-forge
     patchelf                  0.11                 he1b5a44_0    conda-forge
     pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
     patsy                     0.5.1                      py_0    conda-forge
     pcre                      8.44                 he1b5a44_0    conda-forge
     pdbpp                     0.10.0                     py_0    conda-forge
     perl                      5.32.0               h36c2ea0_0    conda-forge
     pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
     pickleshare               0.7.5                   py_1003    conda-forge
     pillow                    8.0.1            py38h70fbd49_0    conda-forge
     pip                       20.2.4                     py_0    conda-forge
     pixman                    0.38.0            h516909a_1003    conda-forge
     pkg-config                0.29.2            h36c2ea0_1008    conda-forge
     pkginfo                   1.6.1              pyh9f0ad1d_0    conda-forge
     pluggy                    0.13.1           py38h924ce5b_3    conda-forge
     poppler                   0.87.0               h4190859_1    conda-forge
     poppler-data              0.4.10                        0    conda-forge
     postgresql                12.3                 h8573dbc_2    conda-forge
     proj                      7.0.0                h966b41f_5    conda-forge
     prometheus_client         0.9.0              pyhd3deb0d_0    conda-forge
     prompt-toolkit            3.0.8              pyha770c72_0    conda-forge
     protobuf                  3.13.0.1         py38h950e882_1    conda-forge
     psutil                    5.7.3            py38h8df0ef7_0    conda-forge
     pthread-stubs             0.4               h14c3975_1001    conda-forge
     ptyprocess                0.6.0                   py_1001    conda-forge
     py                        1.9.0              pyh9f0ad1d_0    conda-forge
     py-cpuinfo                7.0.0              pyh9f0ad1d_0    conda-forge
     py-lief                   0.10.1           py38h348cfbe_2    conda-forge
     pyarrow                   1.0.1           py38h3e2403a_15_cuda    conda-forge
     pyasn1                    0.4.8                      py_0    conda-forge
     pyasn1-modules            0.2.7                      py_0    conda-forge
     pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
     pycosat                   0.6.3           py38h8df0ef7_1005    conda-forge
     pycparser                 2.20               pyh9f0ad1d_2    conda-forge
     pyct                      0.4.6                      py_0    conda-forge
     pyct-core                 0.4.6                      py_0    conda-forge
     pydeck                    0.5.0              pyh9f0ad1d_0    conda-forge
     pyee                      7.0.4              pyh9f0ad1d_0    conda-forge
     pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
     pygments                  2.7.2                      py_0    conda-forge
     pyjwt                     1.7.1                      py_0    conda-forge
     pynvml                    8.0.4                      py_1    conda-forge
     pyopenssl                 19.1.0                     py_1    conda-forge
     pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
     pyppeteer                 0.2.2                      py_1    conda-forge
     pyproj                    2.6.1.post1      py38h7521cb9_0    conda-forge
     pyrepl                    0.9.0            py38h1e0a361_2    conda-forge
     pyrsistent                0.17.3           py38h25fe258_1    conda-forge
     pysocks                   1.7.1            py38h924ce5b_2    conda-forge
     pytest                    6.1.2            py38h578d9bd_0    conda-forge
     pytest-asyncio            0.12.0           py38h32f6830_2    conda-forge
     pytest-benchmark          3.2.3              pyh9f0ad1d_0    conda-forge
     pytest-cov                2.10.1             pyh9f0ad1d_0    conda-forge
     pytest-timeout            1.4.2              pyh9f0ad1d_0    conda-forge
     python                    3.8.6           h852b56e_0_cpython    conda-forge
     python-confluent-kafka    1.5.0            py38h1e0a361_0    conda-forge
     python-dateutil           2.8.1                      py_0    conda-forge
     python-jose               3.1.0              pyh9f0ad1d_0    conda-forge
     python-libarchive-c       2.9              py38h924ce5b_2    conda-forge
     python-louvain            0.13                       py_0    conda-forge
     python_abi                3.8                      1_cp38    conda-forge
     pytz                      2020.4             pyhd8ed1ab_0    conda-forge
     pyviz_comms               0.7.6              pyh9f0ad1d_0    conda-forge
     pyyaml                    5.3.1            py38h8df0ef7_1    conda-forge
     pyzmq                     20.0.0           py38h1d1b12f_1    conda-forge
     rapidjson                 1.1.0             he1b5a44_1002    conda-forge
     rapids-build-env          0.17.0a201116   cuda11.0_py38_g5aced79_102    rapidsai-nightly
     rapids-doc-env            0.17.0a201116   py38_g5aced79_102    rapidsai-nightly
     rapids-notebook-env       0.17.0a201116   cuda11.0_py38_g5aced79_102    rapidsai-nightly
     re2                       2020.11.01           h58526e2_0    conda-forge
     readline                  8.0                  he28a2e2_2    conda-forge
     readme_renderer           27.0               pyh9f0ad1d_0    conda-forge
     recommonmark              0.6.0                      py_0    conda-forge
     regex                     2020.11.13       py38h25fe258_0    conda-forge
     requests                  2.25.0             pyhd3deb0d_0    conda-forge
     requests-oauthlib         1.3.0              pyh9f0ad1d_0    conda-forge
     requests-toolbelt         0.9.1                      py_0    conda-forge
     responses                 0.12.1             pyhd3deb0d_0    conda-forge
     rfc3986                   1.4.0              pyh9f0ad1d_0    conda-forge
     rhash                     1.3.6             h516909a_1001    conda-forge
     ripgrep                   12.1.1               h516909a_1    conda-forge
     rmm                       0.17.0a201106   cuda_11.0_py38_gb1ac445_43    rapidsai-nightly
     rsa                       4.6                pyh9f0ad1d_0    conda-forge
     rtree                     0.9.4            py38h02d302b_2    conda-forge
     ruamel_yaml               0.15.80         py38h8df0ef7_1003    conda-forge
     s3fs                      0.4.2                      py_0    conda-forge
     s3transfer                0.3.3                      py_3    conda-forge
     scikit-learn              0.23.1           py38h3a94b23_0    conda-forge
     scipy                     1.5.1            py38h18bccfc_0    conda-forge
     seaborn                   0.11.0               ha770c72_1    conda-forge
     seaborn-base              0.11.0             pyhd8ed1ab_1    conda-forge
     secretstorage             3.2.0            py38h578d9bd_0    conda-forge
     send2trash                1.5.0                      py_0    conda-forge
     setuptools                49.6.0           py38h924ce5b_2    conda-forge
     shap                      0.37.0                   pypi_0    pypi
     shapely                   1.7.1            py38hc7361b7_1    conda-forge
     shellcheck                0.7.1                         0    conda-forge
     simpervisor               0.3                        py_1    conda-forge
     six                       1.15.0             pyh9f0ad1d_0    conda-forge
     slicer                    0.0.3                    pypi_0    pypi
     snakeviz                  2.1.0              pyh9f0ad1d_0    conda-forge
     snappy                    1.1.8                he1b5a44_3    conda-forge
     snowballstemmer           2.0.0                      py_0    conda-forge
     sortedcontainers          2.3.0              pyhd8ed1ab_0    conda-forge
     soupsieve                 2.0.1                      py_1    conda-forge
     spdlog                    1.7.0                hc9558a2_2    conda-forge
     sphinx                    3.3.1              pyhd8ed1ab_0    conda-forge
     sphinx-copybutton         0.3.1              pyhd8ed1ab_0    conda-forge
     sphinx-markdown-tables    0.0.15                   pypi_0    pypi
     sphinx_rtd_theme          0.5.0              pyh9f0ad1d_0    conda-forge
     sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
     sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
     sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
     sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
     sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
     sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
     sphinxcontrib-websupport  1.2.4              pyh9f0ad1d_0    conda-forge
     sqlite                    3.33.0               h4cf870e_1    conda-forge
     sshpubkeys                3.1.0                      py_0    conda-forge
     statsmodels               0.12.1           py38h0b5ebd8_1    conda-forge
     streamz                   0.6.1              pyhd3deb0d_1    conda-forge
     tbb                       2020.2               hc9558a2_0    conda-forge
     tblib                     1.6.0                      py_0    conda-forge
     terminado                 0.9.1            py38h32f6830_1    conda-forge
     testpath                  0.4.4                      py_0    conda-forge
     threadpoolctl             2.1.0              pyh5ca1d4c_0    conda-forge
     tiledb                    1.7.7                h8efa9f0_3    conda-forge
     tk                        8.6.10               hed695b0_1    conda-forge
     toml                      0.10.2             pyhd8ed1ab_0    conda-forge
     toolz                     0.11.1                     py_0    conda-forge
     tornado                   6.1              py38h25fe258_0    conda-forge
     tqdm                      4.52.0             pyhd3deb0d_0    conda-forge
     traitlets                 5.0.5                      py_0    conda-forge
     treelite                  0.93             py38hadf7658_3    conda-forge
     treelite-runtime          0.93                     pypi_0    pypi
     twine                     3.2.0            py38h32f6830_1    conda-forge
     typed-ast                 1.4.1            py38h25fe258_1    conda-forge
     typing-extensions         3.7.4.3                       0    conda-forge
     typing_extensions         3.7.4.3                    py_0    conda-forge
     tzcode                    2020a                h516909a_0    conda-forge
     ucx                       1.8.1+g6b29558       cuda11.0_0    rapidsai
     ucx-proc                  1.0.0                       gpu    rapidsai
     ucx-py                    0.17.0a201116   py38_g6b29558_17    rapidsai-nightly
     umap-learn                0.4.6            py38h32f6830_0    conda-forge
     urllib3                   1.25.11                    py_0    conda-forge
     wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
     webencodings              0.5.1                      py_1    conda-forge
     websocket-client          0.57.0           py38h32f6830_3    conda-forge
     websockets                8.1              py38h1e0a361_2    conda-forge
     werkzeug                  1.0.1              pyh9f0ad1d_0    conda-forge
     wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
     widgetsnbextension        3.5.1            py38h578d9bd_4    conda-forge
     wmctrl                    0.3             py38h32f6830_1002    conda-forge
     wrapt                     1.12.1           py38h1e0a361_1    conda-forge
     xarray                    0.16.1                     py_0    conda-forge
     xerces-c                  3.2.2             h8412b87_1004    conda-forge
     xmltodict                 0.12.0                     py_0    conda-forge
     xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
     xorg-libice               1.0.10               h516909a_0    conda-forge
     xorg-libsm                1.2.3             h84519dc_1000    conda-forge
     xorg-libx11               1.6.12               h516909a_0    conda-forge
     xorg-libxau               1.0.9                h14c3975_0    conda-forge
     xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
     xorg-libxext              1.3.4                h516909a_0    conda-forge
     xorg-libxpm               3.5.13               h516909a_0    conda-forge
     xorg-libxrender           0.9.10            h516909a_1002    conda-forge
     xorg-libxt                1.1.5             h516909a_1003    conda-forge
     xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
     xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
     xorg-xproto               7.0.31            h14c3975_1007    conda-forge
     xz                        5.2.5                h516909a_1    conda-forge
     yaml                      0.2.5                h516909a_0    conda-forge
     yarl                      1.6.3            py38h25fe258_0    conda-forge
     zeromq                    4.3.3                h58526e2_2    conda-forge
     zict                      2.0.0                      py_0    conda-forge
     zipp                      3.4.0                      py_0    conda-forge
     zlib                      1.2.11            h516909a_1010    conda-forge
     zstd                      1.4.5                h6597ccf_2    conda-forge

[github-actions[bot]]

This issue has been marked stale due to no recent activity in the past 30d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be marked rotten if there is no activity in the next 60d.