shatejas
commented
1 month ago | To reproduce easily cohere 1m dataset was used for benchmarking for the below table | number of index segments | force merge time (minutes) | force merge segments |
|---|---|---|---|
| 2.17 code | 75 | 15.68263 | 3 |
| https://github.com/opensearch-project/k-NN/pull/2133 code | 92 | 12.51252 | 3 |
| 2.16 code (minor code change to mimic it) | 88 | 15.68209 | 3 |
Estimated bottlenecks
KNNVectorValues Creation
KNNVectorValues are created 3 times currently, we cannot reuse the same object as there is no way we could reset the iterator and putting effort into logic for resetting the iterator might not result in latency improvements
- Computing totalLiveDocs
- Training for quantization
- Building index
Currently we are creating KNNVectorValues when quantization is not needed. Exp 2 in the above table shows some improvement in force merge time
TotalLiveDocs computes
There is a linear time complexity to compute total live docs. TotalLiveDocs value is currently needed to
- Mean calculations during quantization training
- Memory allocation computations while building graph for HNSW
Flush case
For flush we can avoid this calculation as there are no deleted docs involved and we can rely on KNNVectorValues or vectors in the field to give us the right result for totalLiveDocs
Merge case
Merge involves removing deleted docs, While merging the segments the deleted docs aren’t considered. To do that current code path is using APIs in MergedVectorValues to have an iterator that can iterate while skipping the deleted docs. The APIs here does not give an iterator which considered deleted docs in its size count. As a result even KNNVectorValues cannot return the right result as it relies on the iterator provided by the MergedVectorValues to compute total live docs
navneet1v
jmazanec15
What is the bug?
After the switch to
NativeEngines990KnnVectorsWriterwe saw force merge latencies increased approximately by 20%&_a=(description:'',filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'333e2950-edec-11ed-a453-51f8c622cf9a',key:query,negate:!f,type:custom,value:'%7B%22bool%22:%7B%22minimum_should_match%22:1,%22should%22:%5B%7B%22match_phrase%22:%7B%22user-tags.cluster-config%22:%22arm64-r6g.4xlarge-3-data-6-shards-1-replica-nmslib-cohere-10m%22%7D%7D,%7B%22match_phrase%22:%7B%22meta.tag_cluster-config%22:%22arm64-r6g.4xlarge-3-data-6-shards-1-replica-nmslib-cohere-10m%22%7D%7D%5D%7D%7D'),query:(bool:(minimum_should_match:1,should:!((match_phrase:(user-tags.cluster-config:arm64-r6g.4xlarge-3-data-6-shards-1-replica-nmslib-cohere-10m)),(match_phrase:(meta.tag_cluster-config:arm64-r6g.4xlarge-3-data-6-shards-1-replica-nmslib-cohere-10m))))))),fullScreenMode:!f,options:(hidePanelTitles:!f,useMargins:!t),query:(language:kuery,query:''),timeRestore:!f,title:'%5BVector%20Search%5D%20%5BNmslib%5D%201OM%20Cohere%20768D',viewMode:view)) in nightly runsThe increase has been consistent
How can one reproduce the bug? Running a benchmark for force-merge against 2.17 vs 2.16 with cohere 10m dataset takes 2000seconds (~30 mins) more
What is the expected behavior? It should take the same time or less
What is your host/environment? Nightlies dashboard&_a=(description:'',filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'333e2950-edec-11ed-a453-51f8c622cf9a',key:query,negate:!f,type:custom,value:'%7B%22bool%22:%7B%22minimum_should_match%22:1,%22should%22:%5B%7B%22match_phrase%22:%7B%22user-tags.cluster-config%22:%22arm64-r6g.4xlarge-3-data-6-shards-1-replica-nmslib-cohere-10m%22%7D%7D,%7B%22match_phrase%22:%7B%22meta.tag_cluster-config%22:%22arm64-r6g.4xlarge-3-data-6-shards-1-replica-nmslib-cohere-10m%22%7D%7D%5D%7D%7D'),query:(bool:(minimum_should_match:1,should:!((match_phrase:(user-tags.cluster-config:arm64-r6g.4xlarge-3-data-6-shards-1-replica-nmslib-cohere-10m)),(match_phrase:(meta.tag_cluster-config:arm64-r6g.4xlarge-3-data-6-shards-1-replica-nmslib-cohere-10m))))))),fullScreenMode:!f,options:(hidePanelTitles:!f,useMargins:!t),query:(language:kuery,query:''),timeRestore:!f,title:'%5BVector%20Search%5D%20%5BNmslib%5D%201OM%20Cohere%20768D',viewMode:view))
Do you have any screenshots? NA
Do you have any additional context? NA