urschrei / pyzotero

Pyzotero: a Python client for the Zotero API
https://pyzotero.readthedocs.org
Other
908 stars 99 forks source link

Is rate limiting handled? #98

Closed LucaMarconato closed 5 years ago

LucaMarconato commented 5 years ago

From this excerpt from the Zotero Web API Documentation, I see that a client should handle rate limiting.

Does PyZotero handle rate limiting by checking for the Backoff: <seconds> HTTP header, and the 429 Too Many Requests return code and the Retry-After: <seconds>?

I am asking this because I use a script which accesses Zotero API indirectly through PyZotero and when running the script I invoke approximately 100-200 requests sequentially. I am afraid that if the limiting rate requests are not handled my ip could be blacklisted.

urschrei commented 5 years ago

The new header-based rate-limiting functionality isn't yet handled by Pyzotero, though it will respect HTTP 429 codes and set progressively-higher backoff durations.

LucaMarconato commented 5 years ago

Thank you. So I can just make all the call I need and the library will automatically make sure that it will not look like I am trying to overload the server, right?

urschrei commented 5 years ago

I don't know whether the API is still returning 429 for too many requests – please test it to make sure. In general, I would err on the side of caution if you think you might be making excessive requests – I haven't extensively tested the rate-limiting functionality.

LucaMarconato commented 5 years ago

Thank you for the information 😊

Since I had to write some extra code for integrating Zotero I ended up writing a simple interface to extract the information that I need directly from the SQLite database stored in ~/Zotero. In my case, this leads to better performance at the price of much more limited APIs. It would be extremely cool if your library handled also the offline case.