opendistro-for-elasticsearch / k-NN

🆕 A machine learning plugin which supports an approximate k-NN search algorithm for Open Distro.
https://opendistro.github.io/
Apache License 2.0
277 stars 55 forks source link

Warmup API causes duplicate graphs to be loaded into memory #255

Closed jmazanec15 closed 3 years ago

jmazanec15 commented 3 years ago

For background, the Plugin uses a cache to store references to the HNSW graphs used to perform k-NN search. The key for this cache is the path to the hnsw file.

On systems that use symlinks for the data path, search will load the graphs into the cache with the key being the absolute path to the data, not the symlink path.

However, warmup will use the symlink path.

This leads the warmup api to load duplicate graphs into the cache.

The fix will be to make the paths the warmup api the same as the paths used in search.