search

StackExchange / NRediSearch

Other
31 stars 9 forks source link

Aggregate limit greater than 10000 in nredisearch #27

Closed Ekotenggara closed 1 year ago

Ekotenggara commented 2 years ago

Hi everyone,

Is it possible to set limi greater than 10000 in nredisearch aggregate? My aggregate can grow to be more than 1 million, can nredisearch to this? Can I remove the limit forever? I have redis installed in local docker in windows.

image

Thank you

Ekotenggara commented 2 years ago

I have figured out by using FT.CONFIG SET MAXSEARCHRESULTS -1 FT.CONFIG SET MAXAGGREGATERESULTS -1 in the cli will enable me to get higher limit. However, is there any way to not specify the limit at all (since I am not sure how many the results are)? At the moment, I will still only get10 results if I don't specify the limit. Thank you.

slorello89 commented 2 years ago

Hi @Ekotenggara - This limit is something that exists in the configuration so you're right, you can either use an FT.CONFIG - or add a configuration flag when you are loading RediSearch to remove those configuration settings.

The accurate, but somewhat disappointing, answer to your second question is that no, you cannot specify no-limit. There are reasons behind this as searches and aggregations can become a pretty accurate foot-gun when you start enumerating massive data sets given that Redis is single-threaded and needs to service those commands one at a time. Your best bet is to use the cursor-API with very large page sizes to paginate massive aggregations.

Ekotenggara commented 1 year ago

Thank you so much for your help :)