WeblateOrg / weblate

Web based localization tool with tight version control integration.
https://weblate.org/
GNU General Public License v3.0
4.6k stars 1.02k forks source link

Weblate throttling #6128

Closed ZloyChert closed 3 years ago

ZloyChert commented 3 years ago

Describe the issue Weblate API throttling works unpredictable. No throttling options are rewritten, I have default 5000 per hour for authorized user. After random number of requests (2-5), Weblate returns Too Many Requests. In headers of the response I got:

I already tried Requests on different endpoints:

One of them - /exports/stats/xxx/xxx - works fine, other return 429

I read all parts about rate limiting, only one of them is about api rate limiting. And there was nothing useful about why I'm getting 429 after 5-10 requests within 5-10 minutes.

github-actions[bot] commented 3 years ago

This issue looks more like a support question than an issue. We strive to answer these reasonably fast, but purchasing the support subscription is not only more responsible and faster for your business but also makes Weblate stronger. In case your question is already answered, making a donation is the right way to say thank you!

nijel commented 3 years ago

The /exports/ endpoint are not covered by the API endpoints (and do not accept API like authentication).

For the /api/ are you sure all your requests are authenticated? In case they are not, you are having way lower limits and these requests would fail earlier...

ZloyChert commented 3 years ago

Authenticated requests are requests with Authorization header, right? I have authorization header. I've tried several formats - "Authorization: Token XXXXXXXXX" and "Authorization: XXXXXXXXX". I still don't know what is the right way. In examples "Token" word exists, but in description of authorization nothing about it. I've thought about unauthorized requests, because 429 have appeared too fast. And the result message of 429 request is "Request was throttled. Expected available in 1895 seconds", Yesterday the number of seconds was 60k+ for sure, it is close to 24 hours - like in unauthorized requests. How can I make sure I am authorized? Maybe something in responses can help it.

ZloyChert commented 3 years ago

I have tried again different combinations of header. Now it works.

The problem was in header. I've tried just token and token with word Token, but I've forgot to add space between token and word "Token"

Thanks

github-actions[bot] commented 3 years ago

The issue you have reported is now resolved. If you don’t feel it’s right, please follow its labels to get a clue for further steps.

nijel commented 3 years ago

See https://docs.weblate.org/en/latest/api.html#authentication-examples

nijel commented 3 years ago

...or use wlc which does that properly for you.

ZloyChert commented 3 years ago

By the way, I'm getting 429 again, with proper authentication. And "Retry-After" 5000sec. Everything was fine for a 3 hours, now it is throttled again.

And I really think, that throttling is not kind of feature, that should be turned on by default.

nijel commented 3 years ago

I think you are still doing some unauthenticated requests (the retry after is longer that the rate limit window for authenticated users). If you really need more than 5000 requests per hour, you can adjust your configuration. I think this is reasonable default, for example it matches what GitHub does.

ZloyChert commented 3 years ago

Yes, I agree. And I've investigated this. I think, I've found the issue. It's trailing slash in url. Urls without slash are redirected (via 301), headers are not copied. Seems so.

github-actions[bot] commented 3 years ago

The issue you have reported is now resolved. If you don’t feel it’s right, please follow its labels to get a clue for further steps.