kakao / n2

TOROS N2 - lightweight approximate Nearest Neighbor library which runs fast even with large datasets
Apache License 2.0
569 stars 70 forks source link

multiple indices #40

Open ronsoohyeong opened 3 years ago

ronsoohyeong commented 3 years ago

First of all, thanks for sharing your great work as open source.

When I tried to use n2, I found the current n2 v0.1.7 only assumes there is only one index. For example: ` num_indices =14

indices =[] for idx in range(num_indices): indices.append( HnswIndex(f, "L2"))

num_examples = 1000 for i in range(num_examples): for idx in range(num_indices): v = [random.gauss(0, 1) for z in range(f)] res = indices[idx].add_data(v)

for idx in range(num_indices): indices[idx].build()

k = 30 for i in range(num_examples): for idx in range(num_indices): neighbor_ids = indices[idx].search_by_id(i, k, include_distances =True) `

Since each index is added with 1000 examples each, the returned ids are supposed to be smaller than 1000 or limited to multiples of 14, but neither was the case. I am wondering if the support for multiple independent indices is in the future plan.

BR.

gony-noreply commented 3 years ago

I found the current n2 v0.1.7 only assumes there is only one index.

No, N2 does not assume that.

the returned ids are supposed to be smaller than 1000 or limited to multiples of 14, but neither was the case.

The Ids returned by search_by_id in your case must be smaller than 1000. If not, tell us how to reproduce it(sample executable and reproducible code)