search

mloesch / sickle

Sickle: OAI-PMH for Humans
Other
106 stars 42 forks source link

UA string? #24

Closed axfelix closed 6 years ago

axfelix commented 6 years ago

Hi,

Does Sickle send a UA string by default, and if not, is it configurable?

I'm having trouble hitting https://spectrum.library.concordia.ca/cgi/oai2?verb=ListRecords&metadataPrefix=oai_dc&set=74797065733D64617461736574 from Sickle when it works fine in a browser and curious about that.

mloesch commented 6 years ago

I'm getting an SSL error with this URL which looks like a broken SSL certificate and is not related to the user agent.

SSLError: HTTPSConnectionPool(host='spectrum.library.concordia.ca', port=443): Max retries exceeded with url: /cgi/oai2?verb=ListRecords&metadataPrefix=oai_dc (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),)

In any way, Sickle allows you to pass through arguments to the requests library. You could turn off SSL verification like this:

sickle = Sickle('https://spectrum.library.concordia.ca/cgi/oai2', verify=False)

This actually worked for me.

You can set any headers including the user agent like this:

headers = {
    'User-Agent': 'My User Agent'
}
sickle = Sickle('https://spectrum.library.concordia.ca/cgi/oai2', headers=headers)
axfelix commented 6 years ago

Thanks!