KristofferC / NearestNeighbors.jl

High performance nearest neighbor data structures (KDTree and BallTree) and algorithms for Julia.
Other
413 stars 65 forks source link

Dynamic trees #44

Open bkalpert opened 7 years ago

bkalpert commented 7 years ago

This package has excellent performance for queries on a static set of points. For streaming data, with points added and earliest points removed to limit storage size, can similar performance be achieved through an extension of the implementation?

Procopiuc, Agarwal, Arge, and Vitter, "Bkd-tree: A dynamic scalable kd-tree" (2003), seem to observe that this is possible, by avoiding memory I/O problems of earlier methods. Is there interest in testing their ideas with an implementation?

KristofferC commented 7 years ago

Right now, the staticness of trees is used quite extensively for performance optimizations. I think that dynamic tress would require a whole new approach to the data structures used, so most of it would have to be written from scratch. It would of course be nice to have dynamic trees in the package but it is significant work. I will not be working on this but will of course review PRs that adds this feature in a way that is consistent with the current "API".

bkalpert commented 7 years ago

It is quite apparent that a great deal of optimization work is present in the static implementation, but also that significant extensions will be needed for the proposed capability. I will experiment, initially with a primitive streaming implementation, to assess the magnitude and feasibility of the task. Thank you for being open to the possibility.

clu8 commented 7 years ago

Thanks @KristofferC for your great work on this package. I am wondering if you or anyone else has any recommendations for a Julia package to use for dynamic k-d trees (with the assumption that all data points will be uniform within some bounds [0,1]^k)? I'm not able to find any existing package. Much appreciated!