Closed ljwolf closed 3 years ago
FWIW, Here is the implementation on the K Function in spaghetti
. The original formulation predates the pointpats
submodule and I am currently working on refactoring for correctness and speedups.
I think that implementation is exactly what I'm suggesting above.
Indeed, we might want to put the functional forms in esda
and then, if useful for these custom point pattern classes in R^d or a network domain, we use them directly from `esda.
Also, got a pretty speed in works by performing step checking in a list comprehension.
Running
Kenv
on the data as in #51, I exhaust 16GB of ram & peg my CPU at n=419.I believe this is because of the memory pressure exerted in
kdtree.query_pairs
https://github.com/pysal/pointpats/blob/bceba7f70deb2a27ac1cf08fe5c7013926714431/pointpats/distance_statistics.py#L486
All told, this is a very inefficient way to compute the K function. Each iteration has to re-discover the neighbors from every previous iteration. At the maximum distance, you have to repeatedly re-discover nearly the entire dataset. Doing this for simulations means this is done for thousands of simulations.
The documentation for
scipy.spatial.KDTree
explicitly warns against using it for this purpose:A better algorithm would:
(distances < d).sum(axis=1)
using numpy vectorization & broadcasting) and multiply by 2 (?) to capture pairs