search

rapidsai / cugraph

cuGraph - RAPIDS Graph Analytics Library
https://docs.rapids.ai/api/cugraph/stable/
Apache License 2.0
1.69k stars 300 forks source link

Leiden Clustering produces too many clusters #3749

Closed Intron7 closed 10 months ago

Intron7 commented 1 year ago

Version

23.06 - 23.06.02

Which installation method(s) does this occur on?

Conda

Describe the bug.

Leiden Clustering produces over 1000 Clusters. This is in contrast to 23.04 where I got around 30-40 for the same test dataset. Louvain Clustering give me 23 clusters.

Minimum reproducible example

def leiden(
    adata: AnnData,
    resolution: float = 1.0,
    n_iterations: int = 100,
    use_weights: bool = True,
    neighbors_key: Optional[int] = None,
    key_added: str = "leiden",
) -> None:
    """
    Performs Leiden Clustering using cuGraph

    Parameters
    ----------
        adata :
            annData object with 'neighbors' field.

        resolution
            A parameter value controlling the coarseness of the clustering.
            Higher values lead to more clusters.

        n_iterations
            This controls the maximum number of levels/iterations of the Louvain algorithm.
            When specified the algorithm will terminate after no more than the specified number of iterations.
            No error occurs when the algorithm terminates early in this manner.

        use_weights
            If `True`, edge weights from the graph are used in the computation
            (placing more emphasis on stronger edges).

        neighbors_key
            If not specified, `leiden` looks at `.obsp['connectivities']` for neighbors connectivities
            If specified, `leiden` looks at `.obsp['neighbors_key_ connectivities']` for neighbors connectivities

        key_added
            `adata.obs` key under which to add the cluster labels.

    """
    # Adjacency graph

    if neighbors_key:
        adjacency = adata.obsp[neighbors_key + "_connectivities"]
    else:
        adjacency = adata.obsp["connectivities"]
    offsets = cudf.Series(adjacency.indptr)
    indices = cudf.Series(adjacency.indices)
    if use_weights:
        weights = cudf.Series(adjacency.data)
    else:
        weights = None

    g = Graph()

    g.from_cudf_adjlist(offsets, indices, weights)

    # Cluster
    leiden_parts, _ = culeiden(g, resolution=resolution, max_iter=n_iterations)

    # Format output
    groups = (
        leiden_parts.to_pandas().sort_values("vertex")[["partition"]].to_numpy().ravel()
    )

    adata.obs[key_added] = pd.Categorical(
        values=groups.astype("U"),
        categories=natsorted(map(str, np.unique(groups))),
    )
    # store information on the clustering parameters
    adata.uns["leiden"] = {}
    adata.uns["leiden"]["params"] = dict(
        resolution=resolution,
        n_iterations=n_iterations,
    )

Relevant log output

No response

Environment details

<details><summary>Click here to see environment details</summary><pre>

     **git***
     Not inside a git repository

     ***OS Information***
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=22.04
     DISTRIB_CODENAME=jammy
     DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS"
     PRETTY_NAME="Ubuntu 22.04.2 LTS"
     NAME="Ubuntu"
     VERSION_ID="22.04"
     VERSION="22.04.2 LTS (Jammy Jellyfish)"
     VERSION_CODENAME=jammy
     ID=ubuntu
     ID_LIKE=debian
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     UBUNTU_CODENAME=jammy
     Linux cl-rajewsky112 5.19.0-46-generic #47~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jun 21 15:35:31 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Thu Jul 27 16:27:52 2023
     +---------------------------------------------------------------------------------------+
     | NVIDIA-SMI 535.54.03              Driver Version: 535.54.03    CUDA Version: 12.2     |
     |-----------------------------------------+----------------------+----------------------+
     | GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
     |                                         |                      |               MIG M. |
     |=========================================+======================+======================|
     |   0  NVIDIA GeForce RTX 3090        On  | 00000000:21:00.0  On |                  N/A |
     |  0%   45C    P8              36W / 350W |   8866MiB / 24576MiB |      1%      Default |
     |                                         |                      |                  N/A |
     +-----------------------------------------+----------------------+----------------------+

     +---------------------------------------------------------------------------------------+
     | Processes:                                                                            |
     |  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
     |        ID   ID                                                             Usage      |
     |=======================================================================================|
     |    0   N/A  N/A      2296      G   /usr/lib/xorg/Xorg                          233MiB |
     |    0   N/A  N/A      2443      G   /usr/bin/gnome-shell                         84MiB |
     |    0   N/A  N/A     34689      G   ...,WinRetrieveSuggestionsOnlyOnDemand       40MiB |
     |    0   N/A  N/A     35675      G   ...1728265,13290406300585528125,262144      181MiB |
     |    0   N/A  N/A     67030      G   ...sion,SpareRendererForSitePerProcess       39MiB |
     |    0   N/A  N/A     68146      C   ...conda3/envs/anndata_test/bin/python      374MiB |
     |    0   N/A  N/A     69689      C   ...conda3/envs/anndata_test/bin/python      374MiB |
     +---------------------------------------------------------------------------------------+

     ***CPU***
     Architecture:                    x86_64
     CPU op-mode(s):                  32-bit, 64-bit
     Address sizes:                   43 bits physical, 48 bits virtual
     Byte Order:                      Little Endian
     CPU(s):                          48
     On-line CPU(s) list:             0-47
     Vendor ID:                       AuthenticAMD
     Model name:                      AMD Ryzen Threadripper 3960X 24-Core Processor
     CPU family:                      23
     Model:                           49
     Thread(s) per core:              2
     Core(s) per socket:              24
     Socket(s):                       1
     Stepping:                        0
     Frequency boost:                 enabled
     CPU max MHz:                     3800,0000
     CPU min MHz:                     2200,0000
     BogoMIPS:                        7585.95
     Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov succor smca sev sev_es
     Virtualization:                  AMD-V
     L1d cache:                       768 KiB (24 instances)
     L1i cache:                       768 KiB (24 instances)
     L2 cache:                        12 MiB (24 instances)
     L3 cache:                        128 MiB (8 instances)
     NUMA node(s):                    1
     NUMA node0 CPU(s):               0-47
     Vulnerability Itlb multihit:     Not affected
     Vulnerability L1tf:              Not affected
     Vulnerability Mds:               Not affected
     Vulnerability Meltdown:          Not affected
     Vulnerability Mmio stale data:   Not affected
     Vulnerability Retbleed:          Mitigation; untrained return thunk; SMT enabled with STIBP protection
     Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
     Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
     Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
     Vulnerability Srbds:             Not affected
     Vulnerability Tsx async abort:   Not affected

     ***CMake***

     ***g++***
     /usr/bin/g++
     g++ (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
     Copyright (C) 2021 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

     ***nvcc***
     /usr/local/cuda/bin/nvcc
     nvcc: NVIDIA (R) Cuda compiler driver
     Copyright (c) 2005-2023 NVIDIA Corporation
     Built on Tue_Jun_13_19:16:58_PDT_2023
     Cuda compilation tools, release 12.2, V12.2.91
     Build cuda_12.2.r12.2/compiler.32965470_0

     ***Python***
     /home/sdicks/miniconda3/envs/anndata_test/bin/python
     Python 3.10.12

     ***Environment Variables***
     PATH                            : /usr/local/cuda/bin:/home/sdicks/miniconda3/envs/anndata_test/bin:/home/sdicks/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/opt/puppetlabs/bin
     LD_LIBRARY_PATH                 : /usr/local/cuda/lib64:
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    : /home/sdicks/miniconda3/envs/anndata_test
     PYTHON_PATH                     :

     ***conda packages***
     conda is /home/sdicks/miniconda3/condabin/conda
     /home/sdicks/miniconda3/condabin/conda
     # packages in environment at /home/sdicks/miniconda3/envs/anndata_test:
     #
     # Name                    Version                   Build  Channel
     _libgcc_mutex             0.1                 conda_forge    conda-forge
     _openmp_mutex             4.5                       2_gnu    conda-forge
     aiohttp                   3.8.5           py310h2372a71_0    conda-forge
     aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
     anndata                   0.10.0.dev68+g0c4c0b0          pypi_0    pypi
     annotated-types           0.5.0                    pypi_0    pypi
     anyio                     3.7.1              pyhd8ed1ab_0    conda-forge
     aom                       3.5.0                h27087fc_0    conda-forge
     appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
     argon2-cffi               21.3.0             pyhd8ed1ab_0    conda-forge
     argon2-cffi-bindings      21.2.0          py310h5764c6d_3    conda-forge
     arpack                    3.7.0                hdefa2d7_2    conda-forge
     arrow                     1.2.3              pyhd8ed1ab_0    conda-forge
     asttokens                 2.2.1              pyhd8ed1ab_0    conda-forge
     async-lru                 2.0.3              pyhd8ed1ab_0    conda-forge
     async-timeout             4.0.2              pyhd8ed1ab_0    conda-forge
     attrs                     23.1.0             pyh71513ae_1    conda-forge
     aws-c-auth                0.7.0                hf8751d9_2    conda-forge
     aws-c-cal                 0.6.0                h93469e0_0    conda-forge
     aws-c-common              0.8.23               hd590300_0    conda-forge
     aws-c-compression         0.2.17               h862ab75_1    conda-forge
     aws-c-event-stream        0.3.1                h9599702_1    conda-forge
     aws-c-http                0.7.11               hbe98c3e_0    conda-forge
     aws-c-io                  0.13.28              h3870b5a_0    conda-forge
     aws-c-mqtt                0.8.14               h2e270ba_2    conda-forge
     aws-c-s3                  0.3.13               heb0bb06_2    conda-forge
     aws-c-sdkutils            0.1.11               h862ab75_1    conda-forge
     aws-checksums             0.1.16               h862ab75_1    conda-forge
     aws-crt-cpp               0.20.3               he9c0e7f_4    conda-forge
     aws-sdk-cpp               1.10.57             hbc2ea52_17    conda-forge
     babel                     2.12.1             pyhd8ed1ab_1    conda-forge
     backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
     backports                 1.0                pyhd8ed1ab_3    conda-forge
     backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
     beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
     bleach                    6.0.0              pyhd8ed1ab_0    conda-forge
     blosc                     1.21.4               h0f2a231_0    conda-forge
     bokeh                     2.4.3              pyhd8ed1ab_3    conda-forge
     boost-cpp                 1.78.0               h5adbc97_2    conda-forge
     branca                    0.6.0              pyhd8ed1ab_0    conda-forge
     brotli                    1.0.9                h166bdaf_9    conda-forge
     brotli-bin                1.0.9                h166bdaf_9    conda-forge
     brotlipy                  0.7.0           py310h5764c6d_1005    conda-forge
     brunsli                   0.1                  h9c3ff4c_0    conda-forge
     bzip2                     1.0.8                h7f98852_4    conda-forge
     c-ares                    1.19.1               hd590300_0    conda-forge
     c-blosc2                  2.10.0               hb4ffafa_0    conda-forge
     ca-certificates           2023.7.22            hbcca054_0    conda-forge
     cached-property           1.5.2                hd8ed1ab_1    conda-forge
     cached_property           1.5.2              pyha770c72_1    conda-forge
     cachetools                5.3.1              pyhd8ed1ab_0    conda-forge
     cairo                     1.16.0            ha61ee94_1014    conda-forge
     certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
     cffi                      1.15.1          py310h255011f_3    conda-forge
     cfitsio                   4.2.0                hd9d235c_0    conda-forge
     charls                    2.4.2                h59595ed_0    conda-forge
     charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
     click                     8.1.6           unix_pyh707e725_0    conda-forge
     click-plugins             1.1.1                      py_0    conda-forge
     cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
     cloudpickle               2.2.1              pyhd8ed1ab_0    conda-forge
     colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
     colorcet                  3.0.1              pyhd8ed1ab_0    conda-forge
     comm                      0.1.3              pyhd8ed1ab_0    conda-forge
     contourpy                 1.1.0           py310hd41b1e2_0    conda-forge
     cryptography              41.0.2          py310h75e40e8_0    conda-forge
     cubinlinker               0.3.0           py310hfdf336d_0    rapidsai
     cucim                     23.06.00        cuda11_py310_230607_gfdc657b_0    rapidsai
     cuda-cudart               12.2.53                       0    nvidia
     cuda-nvrtc                12.2.91                       0    nvidia
     cuda-profiler-api         11.8.86                       0    nvidia
     cuda-python               11.8.2          py310h3fd9d12_0    nvidia
     cuda-version              11.8                 h70ddcb2_2    conda-forge
     cudatoolkit               11.8.0              h37601d7_11    conda-forge
     cudf                      23.06.01        cuda11_py310_230629_g6a548b072d_0    rapidsai
     cudf_kafka                23.06.01        py310_230629_g6a548b072d_0    rapidsai
     cudnn                     8.8.0.121            h0800d71_1    conda-forge
     cugraph                   23.06.02        cuda11_py310_230613_gdb9d3c12_0    rapidsai
     cuml                      23.06.00        cuda11_py310_230607_ga381e03f2_0    rapidsai
     cupy                      12.1.0          py310h53f8385_1    conda-forge
     curl                      8.1.2                h409715c_0    conda-forge
     cusignal                  23.06.00        py39_230607_g22c7120_0    rapidsai
     cusparselt                0.2.0.1              h0800d71_1    conda-forge
     cuspatial                 23.06.00        py310_230607_g7b3284af_0    rapidsai
     custreamz                 23.06.01        py310_230629_g6a548b072d_0    rapidsai
     cutensor                  1.7.0.1              h12f7317_0    conda-forge
     cuxfilter                 23.06.00        py310_230607_g862c7d1_0    rapidsai
     cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
     cyrus-sasl                2.1.27               h9033bb2_6    conda-forge
     cytoolz                   0.12.2          py310h2372a71_0    conda-forge
     dask                      2023.3.2           pyhd8ed1ab_0    conda-forge
     dask-core                 2023.3.2           pyhd8ed1ab_0    conda-forge
     dask-cuda                 23.06.00        py310_230607_gfd3ab2d_0    rapidsai
     dask-cudf                 23.06.01        cuda11_py310_230629_g6a548b072d_0    rapidsai
     datashader                0.15.1             pyhd8ed1ab_0    conda-forge
     datashape                 0.5.4                      py_1    conda-forge
     dav1d                     1.2.1                hd590300_0    conda-forge
     debugpy                   1.6.7           py310heca2aa9_0    conda-forge
     decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
     decoupler                 1.4.0                    pypi_0    pypi
     defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
     distributed               2023.3.2.1         pyhd8ed1ab_0    conda-forge
     dlpack                    0.5                  h9c3ff4c_0    conda-forge
     docrep                    0.3.2                    pypi_0    pypi
     entrypoints               0.4                pyhd8ed1ab_0    conda-forge
     exceptiongroup            1.1.2              pyhd8ed1ab_0    conda-forge
     executing                 1.2.0              pyhd8ed1ab_0    conda-forge
     expat                     2.5.0                hcb278e6_1    conda-forge
     fa2                       0.3.5           py310h5764c6d_2    conda-forge
     fastrlock                 0.8             py310hd8f1fbe_3    conda-forge
     filelock                  3.12.2                   pypi_0    pypi
     fiona                     1.9.1           py310ha325b7b_0    conda-forge
     flit-core                 3.9.0              pyhd8ed1ab_0    conda-forge
     fmt                       9.1.0                h924138e_0    conda-forge
     folium                    0.14.0             pyhd8ed1ab_0    conda-forge
     font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
     font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
     font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
     font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
     fontconfig                2.14.2               h14ed4e7_0    conda-forge
     fonts-conda-ecosystem     1                             0    conda-forge
     fonts-conda-forge         1                             0    conda-forge
     fonttools                 4.41.1          py310h2372a71_0    conda-forge
     fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
     freetype                  2.12.1               hca18f0e_1    conda-forge
     freexl                    1.0.6                h166bdaf_1    conda-forge
     frozenlist                1.4.0           py310h2372a71_0    conda-forge
     fsspec                    2023.6.0           pyh1a96a4e_0    conda-forge
     gdal                      3.6.2           py310hc1b7723_3    conda-forge
     gdk-pixbuf                2.42.10              h05c8ddd_0    conda-forge
     gdown                     4.7.1                    pypi_0    pypi
     geopandas                 0.13.2             pyhd8ed1ab_1    conda-forge
     geopandas-base            0.13.2             pyha770c72_1    conda-forge
     geos                      3.11.1               h27087fc_0    conda-forge
     geotiff                   1.7.1                h7157cca_5    conda-forge
     gettext                   0.21.1               h27087fc_0    conda-forge
     gflags                    2.2.2             he1b5a44_1004    conda-forge
     giflib                    5.2.1                h0b41bf4_3    conda-forge
     glib                      2.76.4               hfc55251_0    conda-forge
     glib-tools                2.76.4               hfc55251_0    conda-forge
     glog                      0.6.0                h6f12383_0    conda-forge
     glpk                      5.0                  h445213a_0    conda-forge
     gmock                     1.13.0               ha770c72_1    conda-forge
     gmp                       6.2.1                h58526e2_0    conda-forge
     gtest                     1.13.0               h00ab1b0_1    conda-forge
     h5py                      3.9.0                    pypi_0    pypi
     harmonypy                 0.0.9                    pypi_0    pypi
     hdf4                      4.2.15               h9772cbc_5    conda-forge
     hdf5                      1.12.2          nompi_h4df4325_101    conda-forge
     holoviews                 1.15.4             pyhd8ed1ab_0    conda-forge
     icu                       70.1                 h27087fc_0    conda-forge
     idna                      3.4                pyhd8ed1ab_0    conda-forge
     igraph                    0.10.3               hb9ddf80_0    conda-forge
     imagecodecs               2023.1.23       py310ha3ed6a1_0    conda-forge
     imageio                   2.31.1             pyh24c5eb1_0    conda-forge
     importlib-metadata        6.8.0              pyha770c72_0    conda-forge
     importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
     importlib_resources       6.0.0              pyhd8ed1ab_1    conda-forge
     inflect                   7.0.0                    pypi_0    pypi
     ipykernel                 6.25.0             pyh71e2992_0    conda-forge
     ipython                   8.14.0             pyh41d4057_0    conda-forge
     ipywidgets                8.0.7              pyhd8ed1ab_0    conda-forge
     isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
     jbig                      2.1               h7f98852_2003    conda-forge
     jedi                      0.18.2             pyhd8ed1ab_0    conda-forge
     jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
     joblib                    1.3.0              pyhd8ed1ab_1    conda-forge
     jpeg                      9e                   h0b41bf4_3    conda-forge
     json-c                    0.16                 hc379101_0    conda-forge
     json5                     0.9.14             pyhd8ed1ab_0    conda-forge
     jsonpointer               2.0                        py_0    conda-forge
     jsonschema                4.18.4             pyhd8ed1ab_0    conda-forge
     jsonschema-specifications 2023.7.1           pyhd8ed1ab_0    conda-forge
     jsonschema-with-format-nongpl 4.18.4             pyhd8ed1ab_0    conda-forge
     jupyter-lsp               2.2.0              pyhd8ed1ab_0    conda-forge
     jupyter-server-proxy      4.0.0              pyhd8ed1ab_0    conda-forge
     jupyter_client            8.3.0              pyhd8ed1ab_0    conda-forge
     jupyter_core              5.3.1           py310hff52083_0    conda-forge
     jupyter_events            0.6.3              pyhd8ed1ab_1    conda-forge
     jupyter_server            2.7.0              pyhd8ed1ab_0    conda-forge
     jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
     jupyterlab                4.0.3              pyhd8ed1ab_0    conda-forge
     jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
     jupyterlab_server         2.24.0             pyhd8ed1ab_0    conda-forge
     jupyterlab_widgets        3.0.8              pyhd8ed1ab_0    conda-forge
     jxrlib                    1.1                  h7f98852_2    conda-forge
     kealib                    1.5.0                ha7026e8_0    conda-forge
     keyutils                  1.6.1                h166bdaf_0    conda-forge
     kiwisolver                1.4.4           py310hbf28c38_1    conda-forge
     krb5                      1.20.1               h81ceb04_0    conda-forge
     lazy_loader               0.2                pyhd8ed1ab_0    conda-forge
     lcms2                     2.15                 hfd0df8a_0    conda-forge
     ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
     leidenalg                 0.9.1           py310heca2aa9_0    conda-forge
     lerc                      4.0.0                h27087fc_0    conda-forge
     libabseil                 20230125.3      cxx17_h59595ed_0    conda-forge
     libaec                    1.0.6                hcb278e6_1    conda-forge
     libarrow                  11.0.0          hd2d78f0_31_cpu    conda-forge
     libavif                   0.11.1               h8182462_2    conda-forge
     libblas                   3.9.0           17_linux64_openblas    conda-forge
     libbrotlicommon           1.0.9                h166bdaf_9    conda-forge
     libbrotlidec              1.0.9                h166bdaf_9    conda-forge
     libbrotlienc              1.0.9                h166bdaf_9    conda-forge
     libcblas                  3.9.0           17_linux64_openblas    conda-forge
     libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
     libcublas                 11.11.3.6                     0    nvidia
     libcublas-dev             11.11.3.6                     0    nvidia
     libcucim                  23.06.00        cuda11_230607_gfdc657b_0    rapidsai
     libcudf                   23.06.01        cuda11_230629_g6a548b072d_0    rapidsai
     libcudf_kafka             23.06.01        230629_g6a548b072d_0    rapidsai
     libcufft                  10.9.0.58                     0    nvidia
     libcufile                 1.4.0.31                      0    nvidia
     libcufile-dev             1.4.0.31                      0    nvidia
     libcugraph                23.06.02        cuda11_230613_gdb9d3c12_0    rapidsai
     libcugraph_etl            23.06.02        cuda11_230613_gdb9d3c12_0    rapidsai
     libcugraphops             23.06.00        cuda11_230607_g77d012ac_0    nvidia
     libcuml                   23.06.00        cuda11_230607_ga381e03f2_0    rapidsai
     libcumlprims              23.06.00        cuda11_230607_g7081940_0    nvidia
     libcurand                 10.3.0.86                     0    nvidia
     libcurand-dev             10.3.0.86                     0    nvidia
     libcurl                   8.1.2                h409715c_0    conda-forge
     libcusolver               11.4.1.48                     0    nvidia
     libcusolver-dev           11.4.1.48                     0    nvidia
     libcusparse               11.7.5.86                     0    nvidia
     libcusparse-dev           11.7.5.86                     0    nvidia
     libcuspatial              23.06.00        cuda11_230607_g7b3284af_0    rapidsai
     libdeflate                1.17                 h0b41bf4_0    conda-forge
     libedit                   3.1.20191231         he28a2e2_2    conda-forge
     libev                     4.33                 h516909a_1    conda-forge
     libevent                  2.1.12               hf998b51_1    conda-forge
     libexpat                  2.5.0                hcb278e6_1    conda-forge
     libffi                    3.4.2                h7f98852_5    conda-forge
     libgcc-ng                 13.1.0               he5830b7_0    conda-forge
     libgdal                   3.6.2                h10cbb15_3    conda-forge
     libgfortran-ng            13.1.0               h69a702a_0    conda-forge
     libgfortran5              13.1.0               h15d22d2_0    conda-forge
     libglib                   2.76.4               hebfc3b9_0    conda-forge
     libgomp                   13.1.0               he5830b7_0    conda-forge
     libgoogle-cloud           2.12.0               hac9eb74_1    conda-forge
     libgrpc                   1.54.2               hb20ce57_2    conda-forge
     libhwloc                  2.9.1                hd6dc26d_0    conda-forge
     libiconv                  1.17                 h166bdaf_0    conda-forge
     libkml                    1.3.0             h37653c0_1015    conda-forge
     libkvikio                 23.06.00        cuda11_230607_gd3b823c_0    rapidsai
     liblapack                 3.9.0           17_linux64_openblas    conda-forge
     libllvm14                 14.0.6               hcd5def8_3    conda-forge
     libnetcdf                 4.8.1           nompi_h261ec11_106    conda-forge
     libnghttp2                1.52.0               h61bc06f_0    conda-forge
     libnsl                    2.0.0                h7f98852_0    conda-forge
     libntlm                   1.4               h7f98852_1002    conda-forge
     libnuma                   2.0.16               h0b41bf4_1    conda-forge
     libopenblas               0.3.23          pthreads_h80387f5_0    conda-forge
     libpng                    1.6.39               h753d276_0    conda-forge
     libpq                     15.2                 hb675445_0    conda-forge
     libprotobuf               3.21.12              h3eb15da_0    conda-forge
     libraft                   23.06.02        cuda11_230705_g7dd2f6dc_0    rapidsai
     libraft-headers           23.06.02        cuda11_230705_g7dd2f6dc_0    rapidsai
     libraft-headers-only      23.06.02        cuda11_230705_g7dd2f6dc_0    rapidsai
     librdkafka                1.9.2                ha5a0de0_2    conda-forge
     librmm                    23.06.00        cuda11_230607_gacaf3f5e_0    rapidsai
     librttopo                 1.1.0               ha49c73b_12    conda-forge
     libsodium                 1.0.18               h36c2ea0_1    conda-forge
     libspatialindex           1.9.3                h9c3ff4c_4    conda-forge
     libspatialite             5.0.1               h7c8129e_22    conda-forge
     libsqlite                 3.42.0               h2797004_0    conda-forge
     libssh2                   1.11.0               h0841786_0    conda-forge
     libstdcxx-ng              13.1.0               hfd8a6a1_0    conda-forge
     libthrift                 0.18.1               h8fd135c_2    conda-forge
     libtiff                   4.5.0                h6adf6a1_2    conda-forge
     libutf8proc               2.8.0                h166bdaf_0    conda-forge
     libuuid                   2.38.1               h0b41bf4_0    conda-forge
     libuv                     1.44.2               hd590300_1    conda-forge
     libwebp                   1.2.4                h1daa5a0_1    conda-forge
     libwebp-base              1.2.4                h166bdaf_0    conda-forge
     libxcb                    1.13              h7f98852_1004    conda-forge
     libxgboost                1.7.5dev.rapidsai23.06        cuda11_0    rapidsai
     libxml2                   2.10.3               hca2bb57_4    conda-forge
     libzip                    1.9.2                hc929e4a_1    conda-forge
     libzlib                   1.2.13               hd590300_5    conda-forge
     libzopfli                 1.0.3                h9c3ff4c_0    conda-forge
     llvmlite                  0.40.1          py310h1b8f574_0    conda-forge
     locket                    1.0.0              pyhd8ed1ab_0    conda-forge
     louvain                   0.8.0           py310hd8f1fbe_1    conda-forge
     lz4                       4.3.2           py310h0cfdcf0_0    conda-forge
     lz4-c                     1.9.4                hcb278e6_0    conda-forge
     mapclassify               2.5.0              pyhd8ed1ab_1    conda-forge
     markdown                  3.4.4              pyhd8ed1ab_0    conda-forge
     markupsafe                2.1.3           py310h2372a71_0    conda-forge
     matplotlib-base           3.7.2           py310hf38f957_0    conda-forge
     matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
     metis                     5.1.1                h59595ed_0    conda-forge
     mistune                   3.0.0              pyhd8ed1ab_0    conda-forge
     mpfr                      4.2.0                hb012696_0    conda-forge
     msgpack-python            1.0.5           py310hdf3cbec_0    conda-forge
     multidict                 6.0.4           py310h1fa729e_0    conda-forge
     multipledispatch          0.6.0                      py_0    conda-forge
     munch                     4.0.0              pyhd8ed1ab_0    conda-forge
     munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
     natsort                   8.4.0                    pypi_0    pypi
     nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
     nbconvert-core            7.7.3              pyhd8ed1ab_0    conda-forge
     nbformat                  5.9.1              pyhd8ed1ab_0    conda-forge
     nccl                      2.18.3.1             h12f7317_0    conda-forge
     ncurses                   6.4                  hcb278e6_0    conda-forge
     nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
     networkx                  3.1                pyhd8ed1ab_0    conda-forge
     nodejs                    18.15.0              h8d033a5_0    conda-forge
     notebook-shim             0.2.3              pyhd8ed1ab_0    conda-forge
     nspr                      4.35                 h27087fc_0    conda-forge
     nss                       3.89                 he45b914_0    conda-forge
     numba                     0.57.1          py310h0f6aa51_0    conda-forge
     numpy                     1.24.4          py310ha4c1d20_0    conda-forge
     nvtx                      0.2.5           py310h1fa729e_0    conda-forge
     omnipath                  1.0.7                    pypi_0    pypi
     openjpeg                  2.5.0                hfec8fc6_2    conda-forge
     openslide                 3.4.1                h7773abc_6    conda-forge
     openssl                   3.1.1                hd590300_1    conda-forge
     orc                       1.9.0                h2f23424_1    conda-forge
     overrides                 7.3.1              pyhd8ed1ab_0    conda-forge
     packaging                 23.1               pyhd8ed1ab_0    conda-forge
     pandas                    1.5.3           py310h9b08913_1    conda-forge
     pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
     panel                     0.14.1             pyhd8ed1ab_0    conda-forge
     param                     1.13.0             pyh1a96a4e_0    conda-forge
     parso                     0.8.3              pyhd8ed1ab_0    conda-forge
     partd                     1.4.0              pyhd8ed1ab_0    conda-forge
     patsy                     0.5.3                    pypi_0    pypi
     pcre2                     10.40                hc3806b6_0    conda-forge
     pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
     pickleshare               0.7.5                   py_1003    conda-forge
     pillow                    9.4.0           py310h023d228_1    conda-forge
     pip                       23.2.1             pyhd8ed1ab_0    conda-forge
     pixman                    0.40.0               h36c2ea0_0    conda-forge
     pkgutil-resolve-name      1.3.10             pyhd8ed1ab_0    conda-forge
     platformdirs              3.9.1              pyhd8ed1ab_0    conda-forge
     pooch                     1.7.0              pyha770c72_3    conda-forge
     poppler                   22.12.0              h091648b_1    conda-forge
     poppler-data              0.4.12               hd8ed1ab_0    conda-forge
     postgresql                15.2                 h3248436_0    conda-forge
     proj                      9.1.0                h8ffa02c_1    conda-forge
     prometheus_client         0.17.1             pyhd8ed1ab_0    conda-forge
     prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
     prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
     protobuf                  4.21.12         py310heca2aa9_0    conda-forge
     psutil                    5.9.5           py310h1fa729e_0    conda-forge
     pthread-stubs             0.4               h36c2ea0_1001    conda-forge
     ptxcompiler               0.8.1           py310h01a121a_0    conda-forge
     ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
     pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
     py-xgboost                1.7.5dev.rapidsai23.06  cuda11_py310_0    rapidsai
     pyarrow                   11.0.0          py310he6bfd7f_31_cpu    conda-forge
     pycparser                 2.21               pyhd8ed1ab_0    conda-forge
     pyct                      0.4.6                      py_0    conda-forge
     pyct-core                 0.4.6                      py_0    conda-forge
     pydantic                  2.1.1                    pypi_0    pypi
     pydantic-core             2.4.0                    pypi_0    pypi
     pydeck                    0.5.0              pyh9f0ad1d_0    conda-forge
     pyee                      8.1.0              pyhd8ed1ab_0    conda-forge
     pygments                  2.15.1             pyhd8ed1ab_0    conda-forge
     pylibcugraph              23.06.02        cuda11_py310_230613_gdb9d3c12_0    rapidsai
     pylibraft                 23.06.02        cuda11_py310_230705_g7dd2f6dc_0    rapidsai
     pynndescent               0.5.10                   pypi_0    pypi
     pynvml                    11.4.1             pyhd8ed1ab_0    conda-forge
     pyopenssl                 23.2.0             pyhd8ed1ab_1    conda-forge
     pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
     pyppeteer                 1.0.2              pyhd8ed1ab_0    conda-forge
     pyproj                    3.4.0           py310hb1338dc_2    conda-forge
     pysocks                   1.7.1              pyha2e5f31_6    conda-forge
     python                    3.10.12         hd12c33a_0_cpython    conda-forge
     python-confluent-kafka    1.9.2           py310h5764c6d_2    conda-forge
     python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
     python-fastjsonschema     2.18.0             pyhd8ed1ab_0    conda-forge
     python-igraph             0.10.3          py310hc734324_0    conda-forge
     python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
     python_abi                3.10                    3_cp310    conda-forge
     pytz                      2023.3             pyhd8ed1ab_0    conda-forge
     pyviz_comms               2.3.2              pyhd8ed1ab_0    conda-forge
     pywavelets                1.4.1           py310h0a54255_0    conda-forge
     pyyaml                    6.0             py310h5764c6d_5    conda-forge
     pyzmq                     25.1.0          py310h5bbb5d0_0    conda-forge
     raft-dask                 23.06.02        cuda11_py310_230705_g7dd2f6dc_0    rapidsai
     rapids                    23.06.02        cuda11_py310_230613_g9b052fc_0    rapidsai
     rapids-singlecell         0.7.2                    pypi_0    pypi
     rapids-xgboost            23.06.02        cuda11_py310_230613_g9b052fc_0    rapidsai
     rdma-core                 28.9                 h59595ed_1    conda-forge
     re2                       2023.03.02           h8c504da_0    conda-forge
     readline                  8.2                  h8228510_1    conda-forge
     referencing               0.30.0             pyhd8ed1ab_0    conda-forge
     requests                  2.31.0             pyhd8ed1ab_0    conda-forge
     rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
     rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
     rmm                       23.06.00        cuda11_py310_230607_gacaf3f5e_0    rapidsai
     rpds-py                   0.9.2           py310hcb5633a_0    conda-forge
     rtree                     1.0.1           py310hbdcdc62_2    conda-forge
     s2n                       1.3.46               h06160fa_0    conda-forge
     scanpy                    1.9.3                    pypi_0    pypi
     scikit-image              0.20.0          py310h9b08913_1    conda-forge
     scikit-learn              1.3.0           py310hf7d194e_0    conda-forge
     scikit-misc               0.3.0                    pypi_0    pypi
     scipy                     1.11.1          py310ha4c1d20_0    conda-forge
     seaborn                   0.12.2                   pypi_0    pypi
     send2trash                1.8.2              pyh41d4057_0    conda-forge
     session-info              1.0.0                    pypi_0    pypi
     setuptools                68.0.0             pyhd8ed1ab_0    conda-forge
     shapely                   2.0.1           py310h8b84c32_0    conda-forge
     simpervisor               1.0.0              pyhd8ed1ab_0    conda-forge
     six                       1.16.0             pyh6c4a22f_0    conda-forge
     snappy                    1.1.10               h9fff704_0    conda-forge
     sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
     sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
     soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
     spdlog                    1.11.0               h9b3ece8_1    conda-forge
     sqlite                    3.42.0               h2c6b66d_0    conda-forge
     stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
     statsmodels               0.14.0                   pypi_0    pypi
     stdlib-list               0.9.0                    pypi_0    pypi
     streamz                   0.6.4              pyh6c4a22f_0    conda-forge
     suitesparse               5.10.1               h9e50725_1    conda-forge
     tbb                       2021.9.0             hf52228f_0    conda-forge
     tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
     terminado                 0.17.1             pyh41d4057_0    conda-forge
     texttable                 1.6.7              pyhd8ed1ab_0    conda-forge
     threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
     tifffile                  2023.7.18          pyhd8ed1ab_0    conda-forge
     tiledb                    2.13.2               hd532e3d_0    conda-forge
     tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
     tk                        8.6.12               h27826a3_0    conda-forge
     tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
     toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
     tornado                   6.3.2           py310h2372a71_0    conda-forge
     tqdm                      4.65.0             pyhd8ed1ab_1    conda-forge
     traitlets                 5.9.0              pyhd8ed1ab_0    conda-forge
     treelite                  3.2.0           py310h1be96d9_0    conda-forge
     treelite-runtime          3.2.0                    pypi_0    pypi
     typing-extensions         4.7.1                hd8ed1ab_0    conda-forge
     typing_extensions         4.7.1              pyha770c72_0    conda-forge
     typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
     tzcode                    2023c                h0b41bf4_0    conda-forge
     tzdata                    2023c                h71feb2d_0    conda-forge
     ucx                       1.14.1               h4a2ce2d_2    conda-forge
     ucx-proc                  1.0.0                       gpu    rapidsai
     ucx-py                    0.32.00         py310_230607_gded9ea2_0    rapidsai
     umap-learn                0.5.3                    pypi_0    pypi
     unicodedata2              15.0.0          py310h5764c6d_0    conda-forge
     uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
     urllib3                   1.26.15            pyhd8ed1ab_0    conda-forge
     wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
     webcolors                 1.13               pyhd8ed1ab_0    conda-forge
     webencodings              0.5.1                      py_1    conda-forge
     websocket-client          1.6.1              pyhd8ed1ab_0    conda-forge
     websockets                10.4            py310h5764c6d_1    conda-forge
     wget                      3.2                      pypi_0    pypi
     wheel                     0.41.0             pyhd8ed1ab_0    conda-forge
     widgetsnbextension        4.0.8              pyhd8ed1ab_0    conda-forge
     wrapt                     1.15.0                   pypi_0    pypi
     xarray                    2023.7.0           pyhd8ed1ab_0    conda-forge
     xerces-c                  3.2.4                h55805fa_1    conda-forge
     xgboost                   1.7.5dev.rapidsai23.06  cuda11_py310_0    rapidsai
     xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
     xorg-libice               1.1.1                hd590300_0    conda-forge
     xorg-libsm                1.2.4                h7391055_0    conda-forge
     xorg-libx11               1.8.4                h0b41bf4_0    conda-forge
     xorg-libxau               1.0.11               hd590300_0    conda-forge
     xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
     xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
     xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
     xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
     xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
     xorg-xproto               7.0.31            h7f98852_1007    conda-forge
     xyzservices               2023.7.0           pyhd8ed1ab_0    conda-forge
     xz                        5.2.6                h166bdaf_0    conda-forge
     yaml                      0.2.5                h7f98852_2    conda-forge
     yarl                      1.9.2           py310h2372a71_0    conda-forge
     zeromq                    4.3.4                h9c3ff4c_1    conda-forge
     zfp                       1.0.0                h27087fc_3    conda-forge
     zict                      3.0.0              pyhd8ed1ab_0    conda-forge
     zipp                      3.16.2             pyhd8ed1ab_0    conda-forge
     zlib                      1.2.13               hd590300_5    conda-forge
     zlib-ng                   2.0.7                h0b41bf4_0    conda-forge
     zstd                      1.5.2                hfc55251_7    conda-forge

</pre></details>

Other/Misc.

This happens on 4 tested systems with both 3090 and A100s

Code of Conduct

naimnv commented 1 year ago

@Intron7 I don't think it's a bug. If the criteria to form Leiden clusters don't fulfill then the nodes are not merged to form aggregated clusters and we end up with a lot more clusters compared to Louvain algorithm.

The way the Leiden algorithm works - the nodes in a graph move to form temporary clusters (similar to modularity optimization phase in Louvain algorithm) that maximize the modularity. Afterwards, in the nodes within each such temporary clusters are checked if they are strongly connected (according to Leiden algorithm). If not, then the nodes in a temporary clusters are not merged to from aggregated clusters.

One can change the thresholds so that relatively weakly-connected nodes keep forming Leiden clusters. If you have specific input graph, we can look into it. cc @ChuckHastings

Intron7 commented 1 year ago

I know that Leiden and Louvain produce different clusterings. But when the number of Leiden clusters goes up from 50 to over 1000 from 23.04 to 23.06 I think there might be some unintentional changes. In my testing it looks like the issue has to do with max_iter. But that's just a hunch.

Intron7 commented 1 year ago

umap

naimnv commented 1 year ago

Thank you for sharing nice plots. Could you try to run with smaller resolution? max_iter indicates maximum number graph aggregation steps. If a graph doesn't change, say after 5 iterations, then the algorithm would terminate.

Intron7 commented 1 year ago

umap

Here is the plot you asked for. The original Plot was taken at 0.6 resolution. now you can see for Leiden the resolution scaling and that it behaves very erratically.

Intron7 commented 1 year ago

umapmaxiter

Here you can see that max_iter doesnt change the plot. My hunch is that the weights dont get updated and they stay the same so the algorithm stops prematurly

naimnv commented 1 year ago

Thank you for sharing additional plots. Is your dataset public? If so, we would like to run it on our end to figure out a bit more on it. If the algorithm doesn't change after 1st iteration, it indicates no Leiden refinement is happening.

Intron7 commented 1 year ago

https://github.com/Intron7/rapids_singlecell/blob/main/notebooks/demo_gpu-seuratv3.ipynb

It's this notebook. I can also create the anndata object needed (with all the preprocessing done) and upload it to googledrive and provide the link

naimnv commented 1 year ago

It would be great if could upload kindly upload the data (that are input the Leiden algorithm) on google drive. I would like to run it locally.

Intron7 commented 1 year ago

https://drive.google.com/file/d/1HfCyeSvEh8RPeMyObTVCdLpeduYCdYU4/view?usp=drive_link

johnhickey22 commented 1 year ago

I also have the same issue with my data that is separate. Solution for leiden clustering would be super helpful!

ChuckHastings commented 1 year ago

@johnhickey22 - which version of cugraph are you using?

johnhickey22 commented 1 year ago

@ChuckHastings - I am using 23.06.02 - let me know if you need anything else.

naimnv commented 1 year ago

@Intron7 I was trying to create graph from cugraph.h5ad.
I wonder, would it be possible to create smaller possible dataset ( perhaps in the vicinity of 50 nodes) out of the data you sent where we would still see too manly leiden cluster compared to louvain?

Intron7 commented 1 year ago

You can already see this behaviour with scanpy.datasets.pbmc68k_reduced()dataset. It's 8 clusters for louvain and 54 for leiden clustering. with resolution=0.6 Edit: Here is a test example

import rapids_singlecell as rsc
import scanpy as sc
adata = sc.datasets.pbmc68k_reduced()
rsc.tl.leiden(adata, resolution=0.6)
rsc.tl.louvain(adata, resolution=0.6)
sc.tl.leiden(adata, resolution=0.6, key_added="cpu_leiden")
sc.pl.umap(adata, color = ["louvain","leiden","cpu_leiden"])

The CPU version of Leiden gives me 9 Clusters.

The Bug still exists in Rapids-23.08

johnhickey22 commented 1 year ago

Any update here - was it solved in another thread? Just wanted to check in, in case I missed something. Would love to integrate this into my analysis pipeline and just waiting for this clustering issue to be solved. Thanks!

naimnv commented 1 year ago

Hi, We are working on it. Will keep you updated. Thanks for your interest.

naimnv commented 11 months ago

Hi, We have pushed new changes to branch-23.12 that would fix the issues.

Intron7 commented 10 months ago

Looks good to me now

naimnv commented 10 months ago

image

@Intron7 Thank you for reporting.