Value Error due to headers format change/

[Ankush-Chander]

Hi @fabiobatalha,

It seems like crossref api has made some modifications in header format.

{'date': 'Mon, 26 Jul 2021 11:41:59 GMT', 'content-type': 'application/json', 'transfer-encoding': 'chunked', 'access-control-allow-origin': '*', 'access-control-allow-headers': 'X-Requested-With', 'vary': 'Accept-Encoding', 'content-encoding': 'gzip', 'server': 'Jetty(9.4.40.v20210413)', 'x-ratelimit-limit': '50', 'x-ratelimit-interval': '1s', 'x-rate-limit-limit': '50, 50', 'x-rate-limit-interval': '1s, 1s', 'permissions-policy': 'interest-cohort=()', 'connection': 'close'}

'x-rate-limit-limit': '50, 50', 'x-rate-limit-interval': '1s, 1s',

Running below code

from crossref.restful import Works
works = Works()
w1 = works.query('zika').sample(20)
for item in w1:
    print(item["title"])

is giving following error:

Traceback (most recent call last):
  File "/home/ankush/.config/JetBrains/PyCharm2021.1/scratches/crossref_scratch.py", line 6, in <module>
    for item in w1:
  File "/media/ankush/ContinentalGroun/workplace/open_source/crossrefapi/crossref/restful.py", line 264, in __iter__
    result = self.do_http_request(
  File "/media/ankush/ContinentalGroun/workplace/open_source/crossrefapi/crossref/restful.py", line 80, in do_http_request
    self._update_rate_limits(result.headers)
  File "/media/ankush/ContinentalGroun/workplace/open_source/crossrefapi/crossref/restful.py", line 43, in _update_rate_limits
    self.rate_limits['X-Rate-Limit-Limit'] = int(headers.get('X-Rate-Limit-Limit', 50))
ValueError: invalid literal for int() with base 10: '50, 50'

[fabiobatalha]

Hi @Ankush-Chander

It seems to be a bug in their API, they are moving to their new version, but this new version should not break compatibility with the previous version.

https://github.com/CrossRef/rest-api-doc#rate-limits

I will include a default value in case of exception.

[fabiobatalha]

Hello @ppolischuk

We noticed an error in the library that is related to the value being retrieved in the header of the Crossref API. It seems the attributes X-Rate-Limit-Limit and X-Rate-Limit-Interval changed their behavior.

Previously:

X-Rate-Limit-Limit: 50 X-Rate-Limit-Interval: 1s

Now:

X-Rate-Limit-Limit: 50, 50 X-Rate-Limit-Interval: 1s, 1s

Is there any reason for this change?

[fabiobatalha]

Related Issue in Crossref API repository : https://gitlab.com/crossref/issues/-/issues/1389

[fabiobatalha]

Hi @Ankush-Chander;

I have just pushed the workaround to master, can you check if it is working.

[Ankush-Chander]

Thanks @fabiobatalha Workaround works as expected. I am keeping the issue open till we hear from our friends at crossref.