search

osm-quality / wikibrain

Stores knowledge and data necessary to properly use links from OpenStreetMap to Wikipedia, Wikidata and Wikimedia Commons.
MIT License
4 stars 1 forks source link

Exception when multiple brand wikidata is used #4

Closed KasperFranz closed 5 months ago

KasperFranz commented 6 months ago

Exception

'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/Users/kasper/dev/OSM-wikipedia-tag-validator/script.py", line 281, in update_problem_for_entry
    reported = issue_detector.get_the_most_important_problem_generic(tags, location, object_type, object_description)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/wikibrain/wikimedia_link_issue_reporter.py", line 233, in get_the_most_important_problem_generic
    something_reportable = self.freely_reorderable_issue_reports(object_description, location, tags)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/wikibrain/wikimedia_link_issue_reporter.py", line 443, in freely_reorderable_issue_reports
    something_reportable = self.check_is_object_brand_is_existing(tags)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/wikibrain/wikimedia_link_issue_reporter.py", line 669, in check_is_object_brand_is_existing
    if self.is_brand_nonexisting(present_wikidata_id):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/wikibrain/wikimedia_link_issue_reporter.py", line 643, in is_brand_nonexisting
    no_longer_existing = wikimedia_connection.get_property_from_wikidata(present_wikidata_id, 'P576')
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/wikimedia_connection/wikimedia_connection.py", line 245, in get_property_from_wikidata
    return wikidata['entities'][wikidata_id]['claims'][property]
           ~~~~~~~~^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

After a bit more digging, I found it is caused by https://www.openstreetmap.org/node/9410011527.

I can not find documentation if brand:wikidata can contain multiple wikidata brands (the wiki specifies that you can have multiple brands, which is why I assume it should be possible)
It is used 100 times in Europe: https://overpass-turbo.eu/s/1FNT

matkoniecz commented 6 months ago

1) exception should not happen with any data - valid or not, so this report is welcome

2) PR fixing it or PR adding test where it currently crashes on synthetic data would be great

3) it is probably potentially valid tagging and should not be reported as bogus

Note that this crashes in processing data from Wikidata - so probably just brand:wikidata with one of this brands will crash it

KasperFranz commented 5 months ago
  1. exception should not happen with any data - valid or not, so this report is welcome
  2. PR fixing it or PR adding test where it currently crashes on synthetic data would be great
  3. it is probably potentially valid tagging and should not be reported as bogus

Note that this crashes in processing data from Wikidata - so probably just brand:wikidata with one of this brands will crash it

When looking in the cache, it is trying to load Q53268;Q6746;Q27597;Q40966;Q30113.wikidata_entity.txt, so it is not splitting it.

I am not too familiar with wikibrain yet so will have a look now and see if I can resolve it