BUG: scipy.Spatial.KDTree Process finished with exit code -1073741571 (0xC00000FD)

samFarrellDay commented 2 years ago

Describe your issue.

KDTree hits this error when it encounters too many duplicates. Without rounding, the reproduceable example works fine. I am running on Windows 10, 16GB memory, Python version 3.9.6.

Reproducing Code Example

from scipy.spatial import KDTree
import numpy as np
random_state = np.random.RandomState(1)
candidate_preds = random_state.uniform(-10,7,size=(294392,1))
candidate_preds = 1/(1+np.exp(-candidate_preds))
candidate_preds = candidate_preds.round(4)
kd_tree = KDTree(candidate_preds, leafsize=100)

Error message

Process finished with exit code -1073741571 (0xC00000FD)

SciPy/NumPy/Python version information

1.7.0 1.21.1 sys.version_info(major=3, minor=9, micro=6, releaselevel='final', serial=0)

tupui commented 2 years ago

Hi @samFarrellDay, thanks for the report. On macOS I cannot reproduce. I would say you could be hitting the recursion limit (0xC00000FD stands for "stack overflow").

Did you try with other parameters on the KDTree or could you try to change the recursion limit of Python (with import sys;sys.setrecursionlimit(1500), don't put something too high. Check the current value with print(sys.getrecursionlimit()))? Or you can play on the stack size, see here.

samFarrellDay commented 2 years ago

I did try different recursion limits to no avail. Changing leaf size also does not help. However, setting balanced_tree=False fixes the issue.

scipy / scipy