pybliometrics-dev / pybliometrics

Python-based API-Wrapper to access Scopus
https://pybliometrics.readthedocs.io/en/stable/
Other
401 stars 124 forks source link

Lookup quota reset time after depletion? #324

Closed Skojar closed 4 months ago

Skojar commented 4 months ago

I naturally wasn't expecting to use up my API keys, but I did. Now I think it's been a week but I'm still getting 429 errors.

I'd like to check my reset time, but calling the API directly I don't get the headers and creating an instance of one of the pybliometrics classes appears to require a call, which gets rejected, so the object isn't created.

How can I use .get_key_reset_time() if I can't create an instance except unless the search is cached, in which case .get_key_reset_time() is None?

It would be super helpful if pybliometerics somehow remembered my reset times for me.

Skojar commented 4 months ago

Different issue, but I discovered that my "it's been a week why haven't my keys reset' was because I let the jupyter notebook sit open over the reset, and apparently the keys weren't being retried. I guess that makes sense, but it didn't occur to me.

Michael-E-Rose commented 4 months ago

I guess the API actually returns a header even when it retursn 429 otherwise, but I cannot confirm. The header contains the information that .get_key_reset_time() parses.

If you have some time, any your key is still over quota, would you try this out for me? All you need to do is to edit https://github.com/pybliometrics-dev/pybliometrics/blob/master/pybliometrics/scopus/utils/get_content.py#L114 and write print(resp.headers).

It would be super helpful if pybliometerics somehow remembered my reset times for me.

This would be too complex. The API reset is per key and API, not just per key.

Skojar commented 4 months ago

I don't have anymore over-quota keys at this time, but I did try to call the API directly from powershell and I got the error but not the headers.

If I exhaust a key anytime soon I will run that test.

This would be too complex. The API reset is per key and API, not just per key.

Ah. I was hoping keeping track of key: timestamp pairs might work.

As I said, the problem resolved itself when I started a new notebook; the index was out of range at line 65, in get_content header = {'X-ELS-APIKey': KEYS[0], and I kept coming back and trying again thinking the library was cycling through keys in the config until it got one to work. Obviously that's not the case.

Michael-E-Rose commented 4 months ago

As I said, the problem resolved itself when I started a new notebook; the index was out of range at line 65, in get_content header = {'X-ELS-APIKey': KEYS[0], and I kept coming back and trying again thinking the library was cycling through keys in the config until it got one to work. Obviously that's not the case.

Ah, now I get it. The keys are loaded once on script startup, and once depleted, they are removed from an internal list. This way pybliometrics can raise a final 429 error.