Closed anermina closed 1 year ago
@anermina can you please update relaton-bipm (e.g. gem install relaton-bipm
or bundle update
)?
IOP has recently implemented anti-DDoS measures which blocks machine access.
The gem now implements detection for IOP's blocking of bulk access, which will now ask you to clear the online captcha.
@andrew2net can you also document the Anti-DDOS detection in the README? Thanks.
I actually get different output when machine access is blocked due to anti-DDoS measures:
[relaton-bipm] This source employs anti-DDoS measures that unfortunately affects automated requests.
[relaton-bipm] Please visit this link in your browser to resolve the CAPTCHA, then retry: https://iopscience.iop.org/article/10.1088/0026-1394/6/3/001
But for these references, I am still getting this kind of output after performing gem install relaton-bipm
and bundle update
:
[relaton-bipm] ("BIPM Metrologia 43 2 S78") fetching...
403 => Net::HTTPForbidden for https://iopscience.iop.org/article/10.1088/0026-1394/43/2/S16 -- unhandled response
@anermina you're right, I get the same problem!
[relaton-bipm] ("BIPM Metrologia 43 2 S78") fetching...
403 => Net::HTTPForbidden for https://iopscience.iop.org/article/10.1088/0026-1394/43/2/S16 -- unhandled response
I am able to open the link in my browser as well.
@ronald in a browser the HTTP request also returns 403 error however the page is displayed correctly. I've made the 403 allowed HTTP code in the relaton-bipm. Maybe we should let BIPM know that some pages have the error.
Fixed in version 1.13.6
Thanks @andrew2net, that is clearly broken on the IOP side. I will let BIPM know.
I've reported to BIPM.
This is a bug report for IOP’s publication of Metrologia citation information. When accessing some Metrologia articles via the browser, the IOP site occasionally returns the HTTP code “403”, which means “Forbidden”.
For example, this link, which goes to Metrologia Vol 47 No 5: https://iopscience.iop.org/article/10.1088/0026-1394/47/5/012
Gives me this response (refresh a few times and it will show up), as you can see the status code is “403". A “403” code means that the server has rejected the request and a properly operating server would return a “Access forbidden" page. Somehow, the IOP site still returns content, but the response has technically failed.
This issue does not happen 100% of the time but quite frequently. I suspect this is after IOP added a “DDoS” protection service in front of their servers, which prevents any sort of machine access (which we have to do to render citations…).
In any case, this issue is not related to our machine-accesses, this already happens with browser access.
Several Metrologia references cannot be fetched due to an error "no access to online site".
Examples:
Output of the
relaton fetch
command:Note: I can reach the site https://iopscience.iop.org/article/10.1088/0026-1394/47/5/R01 regularly using my browser.