search

milvus-io / milvus-sdk-node

The Official Mivus node.js sdk(client)
https://milvus.io
Apache License 2.0
123 stars 37 forks source link

Search with iterator #286

Open BeAllAround opened 6 months ago

BeAllAround commented 6 months ago

Describe the feature:

It doesn't seem to be possible to "search with iterator" as documented in:

https://milvus.io/docs/v2.3.x/with_iterators.md#Search-with-iterator

Describe a specific use case for the feature:

avoiding: invalid max query result window, (offset+limit) should be in range [1, 16384] ERROR

shanghaikid commented 6 months ago

Do you really need this feature? if so, I will add it later.

BeAllAround commented 6 months ago

@shanghaikid well, if there is a workaround where I can return more than 16384 items ( if a collection contains so ) with the milvusClient.search, do let me know. Thank you very much for your consideration.

shanghaikid commented 6 months ago

@shanghaikid well, if there is a workaround where I can return more than 16384 items ( if a collection contains so ) with the milvusClient.search, do let me know. Thank you very much for your consideration.

I see, I will implement this feature.

BeAllAround commented 6 months ago

@shanghaikid Awesome - thanks. If you could republish @zilliz/milvus2-sdk-node via npm then as well, please. That'd be great. I am using package managers to install it.

shanghaikid commented 6 months ago

@shanghaikid Awesome - thanks. If you could republish @zilliz/milvus2-sdk-node via npm then as well, please. That'd be great. I am using package managers to install it.

Of course, but it takes time. stay tuned.

enricrp-rea commented 5 months ago

Hi, What is the status of the issue? It would be nice to has the iterator for search and query

shanghaikid commented 5 months ago

Hi, What is the status of the issue? It would be nice to has the iterator for search and query

sorry, it's on hold,I've finished the query iterator, but based on my testing, the search iterator is not accurate, I am waiting for milvus team 's response.

shanghaikid commented 5 months ago

can you give me your case, I can do some extra testing specifically for your case, like what's your schema, which metric do you like to use?

enricrp-rea commented 5 months ago

I need to get all primary keys from all the entities. So right now I am iterating through pagination until there are no more results left. However, the right solution would be to have the iterator

shanghaikid commented 5 months ago

I need to get all primary keys from all the entities. So right now I am iterating through pagination until there are no more results left. However, the right solution would be to have the iterator

so all you need is query iterator, not search right? the query iterator is ready, but search is not good.

enricrp-rea commented 5 months ago

Yes, in my case i only need a query iterator. Thx

shanghaikid commented 5 months ago

Yes, in my case i only need a query iterator. Thx

ok, I will release one version tomorrow.

shanghaikid commented 5 months ago

Yes, in my case i only need a query iterator. Thx

Hi , v2.4.2 is released. please check here to view the usage method:

https://github.com/milvus-io/milvus-sdk-node/tree/v2.4.2?tab=readme-ov-file#whats-new-in-v242

enricrp-rea commented 5 months ago

Hey thx for the quick reply and update. Just one last thing, I have seen that the limit is required but the documentation sais it is optional. Thx!!