search

zulip / python-zulip-api

Python library for the Zulip API.
https://zulip.com/api/
Apache License 2.0
350 stars 352 forks source link

Handle `RATE_LIMIT_HIT` (Rate limit exceeded) errors #773

Open i-ky opened 1 year ago

i-ky commented 1 year ago

It would be nice if this library handled RATE_LIMIT_HIT errors automatically by sleeping for retry-after seconds and retrying the request. Otherwise this logic needs to be implemented in every user application.

i-ky commented 1 year ago

@zulipbot add "enhancement" "help wanted"

Fahdmoh01 commented 9 months ago

@zulipbot claim

zulipbot commented 9 months ago

Welcome to Zulip, @Fahdmoh01! We just sent you an invite to collaborate on this repository at https://github.com/zulip/python-zulip-api/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:

As you work on this issue, you'll also want to refer to the Zulip code contribution guide, as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!

Fahdmoh01 commented 9 months ago

@i-ky does the Rate limit has to do with the message_limit?

i-ky commented 9 months ago

@i-ky does the Rate limit has to do with the message_limit?

Sorry, I am not familiar with Zulip's internals. What is message_limit you are talking about?

Fahdmoh01 commented 9 months ago

@i-ky does the Rate limit has to do with the message_limit?

Sorry, I am not familiar with Zulip's internals. What is message_limit you are talking about?

message_limit is a variable inside the Ratelimit class of the zulip python api. where do I go to ask for further clarification for this issue?

i-ky commented 9 months ago

I can only clarify what I propose to be changed from a python-zulip-api user's perspective, how it is to be changed from python-zulip-api perspective is a different matter.

timabbott commented 9 months ago

This need to be done as an option -- some clients might very much not want to have sleeps of this form be part of what might happen.

I suppose there's also a question of whether we want the semantics to be to automatically retry the operation that failed due to the rate limit, or something else.

We also need to be cautious in implementing documenting this sort of thing to make sure we don't end up with bugged clients hitting the server exactly as often as the rate limits allow - that can generate a lot more useless load than something that doesn't retry. Probably this is not too hard to manage.

zulipbot commented 8 months ago

@Fahdmoh01 You have been unassigned from this issue because you have not made any updates for over 14 days. Please feel free to reclaim the issue if you decide to pick up again. Thanks!

GeekGawd commented 7 months ago

@timabbott Hi, is this issue still up for grabs and is it something that I can start working on...

grwt commented 7 months ago

@timabbott claim

quinnah commented 6 months ago

@zulipbot claim

zulipbot commented 6 months ago

Welcome to Zulip, @quinnah! We just sent you an invite to collaborate on this repository at https://github.com/zulip/python-zulip-api/invitations. Please accept this invite in order to claim this issue and begin a fun, rewarding experience contributing to Zulip!

Here's some tips to get you off to a good start:

As you work on this issue, you'll also want to refer to the Zulip code contribution guide, as well as the rest of the developer documentation on that site.

See you on the other side (that is, the pull request side)!

codewithnick commented 5 months ago

how do i reproduce this?

codewithnick commented 5 months ago

i tried sending a lot of messages in a loop but it seems to handle it well already

image
i-ky commented 5 months ago

@codewithnick, try a different API endpoint (e.g. I originally suffered from it while uploading custom emojis). I would assume that Zulip is quite tolerant to bursts of messages sent, but may use stricter limits for other stuff.

Or try harder using multiprocessing/threading/asyncio :wink:

codewithnick commented 5 months ago

tried using custom emojis it all looks the same

image

I think it is use less to make it any faster even if it fails on multithreads, because then you are basically abusing the api

i-ky commented 5 months ago

@codewithnick, you can also try lowering limits for your Zulip server.

codewithnick commented 5 months ago

@codewithnick, you can also try lowering limits for your Zulip server.

is there any way to do it? pl lmk

i-ky commented 5 months ago

@codewithnick, you can also try lowering limits for your Zulip server.

is there any way to do it? pl lmk

I believe this is the relevant piece of documentation.