JuliaDynamics / Associations.jl

Algorithms for quantifying associations, independence testing and causal inference from data.
https://juliadynamics.github.io/Associations.jl/stable/
Other
148 stars 12 forks source link

A general reminder that speed-ups are possible for many methods #344

Open kahaaga opened 11 months ago

kahaaga commented 11 months ago

As mentioned in #341, independence testing can be very slow. The slow runtime is mostly due to intrinsic properties of the methods. However For example, in #341 it is mentioned that

For example, only with a timeseries of length 3,000, the LocalPermutationTest with 50 shuffles takes quite a while.

The reason this is so slow is because LocalPermutationTest uses repeated nearest neighbor searches. In general, there is a huge potential for speed-ups both here and in ComplexityMeasures.jl for the kNN-based methods. Potential avenues for improvements are approximate kNN searches, and implementing in-place neighbor searches, so that we can re-use memory when re-searching for neighbors for surrogate realizations.