search

hsolbrig / PyShEx

ShEx interpreter for ShEx 2.0
Creative Commons Zero v1.0 Universal
24 stars 9 forks source link

Need ability to add a User-Agent to SPARQL query requests #62

Open hsolbrig opened 4 years ago

hsolbrig commented 4 years ago

Wikidata has put a set of query limits into place, including the ability to recognize and respond to a HTTP 429 response accompanied by a RETRY-AFTER parameter. It also requires compliance with the USER-AGENT Policy, failure to do as much will result in being "blocked completely".

ross-spencer commented 4 years ago

@hsolbrig thanks for the changes in https://github.com/hsolbrig/PyShEx/commit/2617cd0b816c1aae67490ccdc540c61f1e8f7804.

I was receiving 429 responses from Wikidata today. I was trying an example like in: https://github.com/hsolbrig/PyShEx/blob/master/notebooks/SPARQLEndpoints.ipynb. If it look at the agent, something like:

sparql_query = SPARQLQuery(endpoint, sparql)
print(sparql_query.endpoint.agent)

Then I am seeing the sparqlwrapper agent return, e.g. sparqlwrapper 1.8.5 (rdflib.github.io/sparqlwrapper). I am wondering if you're seeing the same as me when importing PyShex as a library?