MeilliSearchTimeOutError: timeout 5000ms exceeded on process

[davidho8087]

Hi guys. I got this error while creating 150K documents for indexing.

[MeilliSearchTimeOutError ]: timeout of 5000ms has exceeded on process 201 when waiting for pending update to resolve.

May I know how to solve this issue? What settings / attributes and filename should I look into for code changes / customization?

Thanks.

[bidoubiwa]

Hello @davidho8087 Unfortunately when the dataset is to big my function doesn't handle well the long waiting times. A quick solution should be to improve the timeout. Meanwhile, your documents should still be indexed correctly. Is it the case?

[davidho8087]

Hi @bidoubiwa

Thanks for the reply, yes, indeed indexed correctly. however it is quick slow to perform indexing. Waiting time is about 1.5 to 3 seconds per indexing.

[bidoubiwa]

Hey @davidho8087 The indexation time is something our team is working on. It is planned for the next months to make indexation faster!

[bidoubiwa]

Hey @davidho8087 Just a small update after talking with a team member. A reason why the indexation is very slow is that the documents are sent one by one. How did you add the documents? Did you click on the checkbox or did you add the documents one by one into strapi using their own API? Knowing how it happens might help improve the script if possible

For the faster indexation, it will be because a auto-batching system is going to be implemented. It automatically batch the small tasks to avoid having to process multiple task of small sizes in a separate task.  You can learn more about it here.

[davidho8087]

Hey @davidho8087 Just a small update after talking with a team member. A reason why the indexation is very slow is that the documents are sent one by one. How did you add the documents? Did you click on the checkbox or did you add the documents one by one into strapi using their own API? Knowing how it happens might help improve the script if possible

For the faster indexation, it will be because a auto-batching system is going to be implemented. It automatically batch the small tasks to avoid having to process multiple task of small sizes in a separate task.  You can learn more about it here.

[davidho8087]

Hi @bidoubiwa Thanks for prompt response. Here is my finding and hope your team make this plugin amazing.

1) Real Time POST Request ( 10 documents per sec for 40K documents ) I did click the checkbox with listener ( Active ) from zero records and performed a stress test by pumping Post Request of 10 documents per seconds.

FYI , Here is the gap result.

Beginning

Ended.

2) Bulk Indexing with 100K documents in dbase

• Indexing automatically stopped around 60K ++ even I tried a few attempts

  Here is the result . Beginning..

Ended

[bidoubiwa]

This is very intersting! I will have to try with a bigger dataset. Thanks a lot for raising this issue