Open noonenene opened 2 years ago
Hi! I believe there are currently a few ways you can do this depending on your data / needs.
If you use all
you can add a batch_size
cust = Customer.find(Customer.plan == "plan1").all(batch_size=1000000)
You can use an offset to actually get the next 10k
cust = Customer.find(Customer.plan == "plan1", offset=10000).all()
You can also change the page size and limit. In the source code, unless those options are specified they use a default page size of 1,000 - I think even with the .all
cust = Customer.find(Customer.plan == "plan1", limit=1000000,page_size=1000000).all()
to get 1000000 results at once.
There is a page
function but my understanding is that it will only give you a single "page" of results where you specify the offset and how many results you want. But I believe you could use it similarly to the all
cust = Customer.find(Customer.plan == "plan1").page(offset=0,limit=1000000)
That's a relatively new function and @dvora-h might be able to correct my understanding or add clarity.
Thank you for replaying.
cust = Customer.find(Customer.plan == "plan1").all(batch_size=1000000)
redis.exceptions.ResponseError: LIMIT exceeds maximum of 10000
, offset=10000
TypeError: find() got an unexpected keyword argument 'offset'
Page function not published yet to PyPi.
@noonenene You may be hitting a RediSearch limit. See the https://redis.io/docs/stack/search/configuring/#maxsearchresults setting. You can change this by running:
FT.CONFIG SET MAXSEARCHRESULTS -1
Can you try setting this and the re-run your query?
Hi,
I am using
JsonModel
.I apologize for posting this noob question here. I am really stuck.
cust = Customer.find(Customer.plan == "plan1").all()
return only 10k. How do i get the next 10k?