WolfgangFahl / ConferenceCorpus

ScientificEventCorpus
Apache License 2.0
1 stars 2 forks source link

Wikidata Query Service Timeout handling #44

Open WolfgangFahl opened 2 years ago

WolfgangFahl commented 2 years ago

The behavior in case of a Wikidata Query Service timeout is:

 EndPointInternalError('EndPointInternalError: endpoint returned code 500 and response. \n\nResponse:\nb\'SPARQL-QUERY: queryStr=PREFIX pq: <http://www.wikidata.org/prop/qualifier/>\\nPREFIX p: <http://www.wikidata.org/prop/>\\nPREFIX wd: <http://www.wikidata.org/entity/>\\nPREFIX wdt: <http://www.wikidata.org/prop/direct/>\\nPREFIX wikibase: <http://wikiba.se/ontology#>\\nPREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>\\nSELECT DISTINCT\

Leading to a traceback:

Traceback (most recent call last):
  File "/home/wf/.local/lib/python3.8/site-packages/SPARQLWrapper/Wrapper.py", line 1073, in _query
    response = urlopener(request)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: Internal Server Error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wf/.local/lib/python3.8/site-packages/corpus/ccupdate.py", line 164, in main
    updater.updateDataSource(f"{lookupId} cache", args.sample)
  File "/home/wf/.local/lib/python3.8/site-packages/corpus/ccupdate.py", line 33, in updateDataSource
    lookup.load(forceUpdate=True,showProgress=True)
  File "/home/wf/.local/lib/python3.8/site-packages/corpus/lookup.py", line 175, in load
    self.eventCorpus.loadAll(forceUpdate=forceUpdate,showProgress=showProgress)
  File "/home/wf/.local/lib/python3.8/site-packages/corpus/eventcorpus.py", line 99, in loadAll
    eventDataSource.load(forceUpdate=forceUpdate,showProgress=showProgress)
  File "/home/wf/.local/lib/python3.8/site-packages/corpus/eventcorpus.py", line 49, in load
    self.eventManager.fromCache(force=forceUpdate)
  File "/home/wf/.local/lib/python3.8/site-packages/lodstorage/entity.py", line 211, in fromCache
    listOfDicts=getListOfDicts()
  File "/home/wf/.local/lib/python3.8/site-packages/corpus/event.py", line 471, in getLoDfromEndpoint
    listOfDicts=sparql.queryAsListOfDicts(query)
  File "/home/wf/.local/lib/python3.8/site-packages/lodstorage/sparql.py", line 285, in queryAsListOfDicts
    records=self.query(queryString)
  File "/home/wf/.local/lib/python3.8/site-packages/lodstorage/sparql.py", line 259, in query
    queryResult=self.rawQuery(queryString,method=method)
  File "/home/wf/.local/lib/python3.8/site-packages/lodstorage/sparql.py", line 56, in rawQuery
    queryResult = self.sparql.query()
  File "/home/wf/.local/lib/python3.8/site-packages/SPARQLWrapper/SmartWrapper.py", line 312, in query
    res = super(SPARQLWrapper2, self).query()
  File "/home/wf/.local/lib/python3.8/site-packages/SPARQLWrapper/Wrapper.py", line 1107, in query
    return QueryResult(self._query())
  File "/home/wf/.local/lib/python3.8/site-packages/SPARQLWrapper/Wrapper.py", line 1085, in _query
    raise EndPointInternalError(e.read())
SPARQLWrapper.SPARQLExceptions.EndPointInternalError: EndPointInternalError: endpoint returned code 500 and response. 
WolfgangFahl commented 2 years ago

getLoDfromEndpoint() is the function which needs to be able to handle this.