search

urschrei / pyzotero

Pyzotero: a Python client for the Zotero API
https://pyzotero.readthedocs.org
Other
910 stars 100 forks source link

API Limits? #139

Closed pjmattingly closed 3 years ago

pjmattingly commented 3 years ago

I was planning to use pyzotero for some Zotero automation, but I didn't see a mention on polite limits on queries. Are you aware of any such limits? Currently my automation does a query every ~1 minute or so, but this could increase in time.

Thank you.

urschrei commented 3 years ago

Pyzotero tries to do the right thing by introducing successively longer backoff periods before retrying an API call if it receives a 429 ("you are being rate-limited") response. I don't have any information about hourly / daily limits; the zotero-dev Google group is probably the best place to ask if you're concerned.

pjmattingly commented 3 years ago

Fair enough.

Can these responses be piped to the caller? That way I could include some handling as well. More specifically, I might have some other calls to the Zotero occurring that I could limit if I knew they had requested or required some limiting.

urschrei commented 3 years ago

Sorry, I just checked and it's slightly more complex: any response (successful or otherwise) can return a backoff header, which the client handles by sleeping for the requested number of seconds before making subsequent calls. If the client receives a 429 response with a backoff header, it'll sleep and then retry that call until it succeeds or it receives a 429 response without a backoff header, which will raise a TooManyRetries error which you can catch and handle as you see fit.

Essentially, the client implements the behaviour that the API v3 docs ask for, and you can't easily make it behave more conservatively than that -- in practice, that shouldn't be a problem, but the Zotero team is helpful and responsive on the group if you're concerned about rate-limiting.

There are retrieval approaches that take advantage of the sync functionality provided by the API, and you may be able to exploit those to reduce calls or bandwidth, depending on your use case, but again, that's probably better discussed on the group.

pjmattingly commented 3 years ago

Okay, great. I'll keep an eye out for those exceptions then.

Thanks for the additional details.

Take care.

On Wed, Jun 2, 2021 at 5:54 PM Stephan Hügel @.***> wrote:

Sorry, I just checked and it's slightly more complex: any response (successful or otherwise) can return a backoff header, which the client handles by sleeping for the requested number of seconds before making subsequent calls. If the client receives a 429 response with a backoff header, it'll sleep and then retry that call until it succeeds or it receives a 429 response without a backoff header, which will raise a TooManyRetries error which you can catch and handle as you see fit.

Essentially, the client implements the behaviour that the API v3 docs ask for, and you can't easily make it behave more conservatively than that -- in practice, that shouldn't be a problem, but the Zotero team is helpful and responsive on the group if you're concerned about rate-limiting.

There are retrieval approaches that take advantage of the sync functionality provided by the API, and you may be able to exploit those to reduce calls or bandwidth, depending on your use case, but again, that's probably better discussed on the group.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/urschrei/pyzotero/issues/139#issuecomment-853455643, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABP7MZ5NNIKDUHHKUQZGZYLTQ3ABTANCNFSM457T37DQ .