Segfault with n_jobs > 17 during indexing

[redst4r]

Hi, I ran into this weird bug: pynndescent works fine when using a smaller number of n_jobs (anything up to n_jobs=17 works) , but above, it just throws a segmentation fault. Note that I'm running this on a 32-CPU server, so n_jobs=18 is nowhere near the physical number of CPUs.

from pynndescent import NNDescent
import numpy as np
x =np.random.randn(10_000, 20)

index = NNDescent(x, verbose=True, n_jobs=16) # works fine
index = NNDescent(x, verbose=True, n_jobs=17) # works fine
index = NNDescent(x, verbose=True, n_jobs=18) # --> segfault

# Thu Sep 23 03:05:20 2021 Building RP forest with 23 trees
# Thu Sep 23 03:05:21 2021 NN descent for 13 iterations
#    1  /  17
# [1]    2730 segmentation fault  ipython

The size of the input data doesn't seem to matter much, I get the same breakdown at n_jobs=18 with 100k instead of 10k datapoints (or 200 instead of 20 features).

---

Package Versions

python                    3.7.7                hcff3b4d_5  
numba                     0.52.0           py37hdc94413_0    conda-forge
pynndescent               0.5.4              pyh6c4a22f_0    conda-forge

EDIT: This seems to be specific to the conda-forge version, cloning the current commit and doing a pip install -e . allows the above code to run without the segfault. No idea what the problem is though.

Even weirder: Cloning, checking out 0.5.4 (which is the commit the conda-forge version is based on) and doing a pip install -e . also DOES NOT produce the seqfault. Must be something in that conda-forge package...