search

cambialens / lens-api-doc

10 stars 5 forks source link

Error 429, Scholarly API post #49

Closed chmras closed 3 years ago

chmras commented 3 years ago

I get a error back when using scroll option: "The remote server returned an error: (429) ." Unfortunately, i don't get more info, than above.

When i copy out the exact same query, it works fine in PostMan;

What could be the culprit of this?

(i get it at, whats equal to, document number 3000 to 3100, with a size of 100.)

rosharma9 commented 3 years ago

You should see the response like this if you are getting 429.

{
    "reference": "rate_limited",
    "message": "Too many requests. Allowed 'X' per minute.",
    "code": 429
}

You also get the following response header x-rate-limit-retry-after-seconds with numeric value representing the number seconds you need to wait before making another request. In the sample code, you can see this logic following this comment: If rate-limited, wait for n seconds and proceed the same scroll id.

To check if you are rate limited on number of allowed request or check your usage, you can use following endpoint: https://api.lens.org/subscriptions/scholarly_api/usage?token=YOUR_ACCESS_TOKEN

I see you mentioned you requested 100 per request. If you are downloading 5000 records, you can set size up-to 1000 for faster download and less chance to get rate limited.