siznax / wptools

Wikipedia tools (for Humans): easily extract data from Wikipedia, Wikidata, and other MediaWikis
MIT License
574 stars 78 forks source link

AttributeError: 'NoneType' object has no attribute 'get' #38

Closed lisongx closed 8 years ago

lisongx commented 8 years ago
>>> wptools.page(u'松下奈緒',  lang='zh', variant='zh-cn').get_wikidata()
松下奈緒 (zh/zh-cn)
{
  lang: zh
  title: 松下奈緒
  variant: zh-cn
}
www.wikidata.org (wikidata) 松下奈緒
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/var/dae/apps/fm/venv/src/wptools/wptools/core.py", line 778, in get_wikidata
    self._set_wikidata()
  File "/var/dae/apps/fm/venv/src/wptools/wptools/core.py", line 475, in _set_wikidata
    self._marshal_claims(item.get('claims'))
  File "/var/dae/apps/fm/venv/src/wptools/wptools/core.py", line 444, in _marshal_claims
    self.props = self._wikidata_props(query_claims)
  File "/var/dae/apps/fm/venv/src/wptools/wptools/core.py", line 421, in _wikidata_props
    snak = prop.get('mainsnak').get('datavalue').get('value')
AttributeError: 'NoneType' object has no attribute 'get'
lisongx commented 8 years ago

seems that it went error when process this prop
{u'type': u'statement', u'mainsnak': {u'datatype': u'time', u'property': u'P576', u'snaktype': u'novalue'}, u'id': u'Q483718$9da5583a-45e8-4268-2770-41d999bd53da', u'rank': u'normal'}

so there is no datavalue here, see also in wikidata page https://www.wikidata.org/wiki/Q483718

screen shot 2016-11-03 at 5 45 26 pm

so I think it make sense to return a None in this case?

siznax commented 8 years ago

The offending item appears to be https://www.wikidata.org/wiki/Q1041708. Maybe you were looking at the wrong tab? 😉

The problematic property for this item was P19 ("place of birth"), which is hard to see without inspecting the API response directly (page.cache['wikidata']['response']). P19 is simply missing a "datavalue" member, normally found inside "mainsnak"!

"P19": [
    {
        "mainsnak": {
            "snaktype": "somevalue",
            "property": "P19",
            "datatype": "wikibase-item"
        },
        "type": "statement",
        "id": "q1041708$b4fe2176-475f-74f0-537a-9c9af65cc684",
        "rank": "normal"
    }
],
siznax commented 8 years ago

Thanks so much for the fix @seansay!