Algolia: Response not successful: Received status code 429

[bauhem]

Hello, I have a problem with rate limitations and Algolia. Netlify gives me the following error: Algolia: Response not successful: Received status code 429. Do you have any tips on how we can avoid this using your plugins in Gridsome?

Thanks

[u12206050]

Hei, What does your config look like? Do you have unique id for each item you index and a data changed that you can use to check what items have changed?

Sounds to me that you might just have to many items that you want to index. How many are you dealing with?

[bauhem]

[bauhem]

Is that related to this, because my index would have more than 1000 items.

?

[bauhem]

And the Netlify (or Vercel) error

[bauhem]

I was wondering if your plugins could throttle or send in batch or any other solution.

https://support.algolia.com/hc/en-us/articles/4406975251089-Is-there-a-rate-limit-/#what-happens-when-the-rate-limit-is-reached

[u12206050]

How many items do you have? It might be that if you have never indexed to Algolia before, that to just get it running try running the build process a few times so indexing can happen, and after a while only the changes should start triggering.

However, if you have an insane amount of documents typically 100000+ then it could also be that the partial update is the issue when it is trying to fetch a diff.

[u12206050]

Nevermind all that.

Just realised the error is being thrown here

      const result = await graphql(query);
      if (result.errors) {
        throw new Error(`failed to index to Algolia: ${result.errors}`);
      }

Meaning while trying to execute your graphql query, so might be your server that is rate limiting.

Also the fact that in your log output it says items in collection 0 means that there is nothing returned from your transformer either.

Add a console log within your transformer to check it is being called with the data you expect.

[bauhem]

Ok, i'll follow your advice about get it running, and running again until only the updated one are sent. Thanks for your time.

[u12206050]

Do you mind sharing your config. How often are you building, and do you use an id for the items you index so you don't have to reindex if nothing has changed?

On Mon, 28 Mar 2022, 19:35 Guillaume Gosselin, @.***> wrote:

Hello, I have a problem with rate limitations and Algolia. Netlify gives me the following error: Algolia: Response not successful: Received status code 429. Do you have any tips on how we can avoid this using your plugins in Gridsome?

Thanks

— Reply to this email directly, view it on GitHub https://github.com/u12206050/gridsome-plugin-algolia/issues/13, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSVMWSF7DWETMR5H4WA7STVCHUW3ANCNFSM5R33V7HQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>