[Bug]: 429 Too Many Requests

lugfug commented 1 year ago

What happened?

I'm using the latest version of the Quivr docker deployment as of 09-22-2023.

It's been working this morning. But now all of a sudden, when I ask it a question, there is NO response and the logs show a status/error code of 429 too many requests.

2023-09-22 13:34:30 backend-core | INFO: 192.168.65.1:24054 - "POST /chat/8bfd06b0-2094-4796-bcb6-12ccd3e34f8e/question/stream?brain_id=951b5abe-d0b9-47a7-aa67-0cab1aef3d03 HTTP/1.1" 429 Too Many Requests

So Is this a fault of the docker application stack or of an API?

What the heck is happening?

Relevant log output

2023-09-22 13:34:28 backend-core  | INFO:     192.168.65.1:24055 - "GET /user HTTP/1.1" 200 OK
2023-09-22 13:34:28 backend-core  | INFO:     192.168.65.1:24054 - "GET /user/identity HTTP/1.1" 200 OK
2023-09-22 13:34:29 backend-core  | 2023-09-22 20:34:29,539:INFO - Streaming request for gpt-3.5-turbo
2023-09-22 13:34:30 backend-core  | 2023-09-22 20:34:30,024 [INFO] models.user_usage: User user@gmail.com request count updated to 65
2023-09-22 13:34:30 backend-core  | INFO:     192.168.65.1:24054 - "POST /chat/8bfd06b0-2094-4796-bcb6-12ccd3e34f8e/question/stream?brain_id=951b5abe-d0b9-47a7-aa67-0cab1aef3d03 HTTP/1.1" 429 Too Many Requests
2023-09-22 13:34:30 backend-core  | INFO:     192.168.65.1:24057 - "OPTIONS /brains/951b5abe-d0b9-47a7-aa67-0cab1aef3d03/ HTTP/1.1" 200 OK
2023-09-22 13:34:30 backend-core  | 2023-09-22 20:34:30,511 [INFO] repository.brain.get_default_user_brain: Default brain response: 8db9f98e-77c4-4019-a259-96c01da908fb
2023-09-22 13:34:30 backend-core  | 2023-09-22 20:34:30,512 [INFO] repository.brain.get_default_user_brain: Default brain id: 8db9f98e-77c4-4019-a259-96c01da908fb
2023-09-22 13:34:43 backend-core  | INFO:     192.168.65.1:24054 - "GET /user HTTP/1.1" 200 OK
2023-09-22 13:34:43 backend-core  | INFO:     192.168.65.1:24055 - "GET /user/identity HTTP/1.1" 200 OK
2023-09-22 13:34:43 backend-core  | INFO:     192.168.65.1:24058 - "GET /brains/951b5abe-d0b9-47a7-aa67-0cab1aef3d03/ HTTP/1.1" 200 OK
2023-09-22 13:35:37 backend-core  | 2023-09-22 20:35:37,830 [INFO] repository.brain.get_default_user_brain: Default brain response: 8db9f98e-77c4-4019-a259-96c01da908fb
2023-09-22 13:35:37 backend-core  | 2023-09-22 20:35:37,832 [INFO] repository.brain.get_default_user_brain: Default brain id: 8db9f98e-77c4-4019-a259-96c01da908fb
2023-09-22 13:35:52 backend-core  | INFO:     192.168.65.1:24060 - "GET /user HTTP/1.1" 200 OK
2023-09-22 13:35:52 backend-core  | INFO:     192.168.65.1:24059 - "GET /user/identity HTTP/1.1" 200 OK

Twitter / LinkedIn details

No response

dosubot[bot] commented 1 year ago

🤖

Hello,

The "429 Too Many Requests" error you're encountering is typically associated with rate limiting, which is a common protective measure used by APIs to prevent abuse. This error is returned when a user sends too many requests in a given amount of time.

From the logs you've provided, it seems like the Quivr application is making too many requests to an API within a short period, causing the API server to respond with a 429 error. This is not directly related to the Docker application stack.

To resolve this, you might need to check the Quivr application's code or the API's documentation to understand their rate limiting policies and adjust your requests accordingly. If the issue is with the Quivr application, you might need to implement some form of rate limiting or request throttling. If the issue is with the API, you might need to slow down your requests or request an increased rate limit from the API provider.

In the Quivr application, the rate limit is configured in the api.js file under the rest object. The defaultLimit is set to 25 and the maxLimit is set to 100. This means that by default, a user can make 25 requests, but this can be increased up to a maximum of 100 requests. The withCount property set to true indicates that the count of requests is maintained.

Here is the configuration:

module.exports = {
  rest: {
    defaultLimit: 25,
    maxLimit: 100,
    withCount: true,
  },
};

You can find this in the cms/quivr/config/api.js file.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Best, Dosu

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

lugfug commented 1 year ago

WHAT API?

lugfug commented 1 year ago

I'm using the DEFAULT settings from the Docker Compose file. !

lugfug commented 1 year ago

It appears that you need an Open API Key for each Brain, aka its required. Is this documented? Once I added an API Key for each brain (in addition to the API key required to setup Quivr), then things started working without erroring with 429 too many requests.

I found a mention of this issue and a potential fix here https://github.com/StanGirard/quivr/issues/965 A user should not be allowed to do more than 50 requests per day through Quivr unless they have their own OpenAI API Key or the brain has used has an OpenAI Api Key

StanGirard commented 1 year ago

Go in the user_settings and increase the max brain questions :)

lugfug commented 1 year ago

Hi @StanGirard Man I would LOVE to follow your instructions. Could you please kindly be a little more specific? Where is user_settings supposed to be located...?

EDIT: Ohh you mean in supabase... go into the user_settings table and change the value of max_requests_number.

Man that took me a while to figure out.

I made the change, seems to be working as expected now. I also added an OpenAI API key for each brain, so maybe that is also mitigating the issues I was experiencing.

QuivrHQ / quivr