Error in update_wikidata should not crash the bot

[simon04]

Fixes #139.

[konstin]

I'm afraid that's not a solution; We can't just silently swallow all errors. What needs to be done is a check for the specific error from #139 as specifically as possible, which also logs the offending entries so we can work on fixing the spam filter and/or talk to projects about their websites doing something bad

[simon04]

The errors aren't swallowed, but logged one line above.

A failure on one Wikidata item/project probably shouldn't crash the bot all together, when it could resume with the next one.

I'd like to offer my help in maintaining and running this bot. Besides this issue, what would be the most relevant issues/features to look into?

On Fri, 30 Sep 2022, 08:50 konstin, @.***> wrote:

I'm afraid that's not a solution; We can't just silently swallow all errors. What needs to be done is a check for the specific error from #139 https://github.com/konstin/github-wikidata-bot/issues/139 as specifically as possible, which also logs the offending entries so we can work on fixing the spam filter and/or talk to projects about their websites doing something bad

— Reply to this email directly, view it on GitHub https://github.com/konstin/github-wikidata-bot/pull/156#issuecomment-1263177085, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF7A3QOTWGV473ZPHCJHA3WA2ETDANCNFSM6AAAAAAQZFNXTE . You are receiving this because you authored the thread.Message ID: @.***>

[konstin]

no, except Exception and just continuing isn't appropriate error handling.

A failure on one Wikidata item/project probably shouldn't crash the bot all together, when it could resume with the next one.

There shouldn't be any random failures on single items! We want to know when something fails, because it most likely means a bug, and i don't remember any spurious failures prior to the spam filter thing

I'd like to offer my help in maintaining and running this bot. Besides this issue, what would be the most relevant issues/features to look into?

What needs to be done is:

• Figure out the exact reason for #139 and fix it properly
• Fix #109
• Fix CI
• Update dependencies
• Reactivate cron job

[simon04]

I cannot reproduce the bug on Q107387130 mentioned on https://www.wikidata.org/wiki/Special:Log/spamblacklist/Github-wiki-bot since apparently the website of the GitHub repository has been changed from anti-captcha.com to antigate.com

> python main.py --filter antigate
# Querying Projects
11757 projects were found by the sparql query
1 projects remained after filtering
1 projects were found
# Processing projects
## antigate: http://www.wikidata.org/entity/Q107387130
There are 3 stable releases
Added '1.3.1'
Added '1.3.4'
Added '1.4.0'
Setting preferred rank for 1.4.0
# Finished successfully

[konstin]

I tried doing a new run, but now i get different crash:

$ .venv/bin/python ./main.py
[...]
## Q109616408: http://www.wikidata.org/entity/Q109616408
There are 23 stable releases
Setting normal rank for 1.3.1
Password for user Github-wiki-bot on wikidata:wikidata (no characters will be shown): 
Failed to update http://www.wikidata.org/entity/Q109616408: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "./main.py", line 844, in <module>
    main()                                     
  File "./main.py", line 838, in main
    raise e                                    
  File "./main.py", line 835, in main
    update_wikidata(properties, edit_group_hash)
  File "./main.py", line 722, in update_wikidata
    set_claim_rank(claim, latest_version, release, edit_group_hash)
  File "./main.py", line 587, in set_claim_rank 
    claim.changeRank("normal", summary=get_summary(edit_group_hash))
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/page/__init__.py", line 4666, in changeRank
    return self.repo.save_claim(self, **kwargs) 
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/site/_decorators.py", line 92, in callee
    return fn(self, *args, **kwargs)
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/site/_datasite.py", line 427, in save_claim
    'token': self.tokens['edit'],
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/site/_tokenwallet.py", line 58, in __getitem__
    key = self.site.validate_tokens([key])[0]
  File "github-wikidata-bot/.venv/lib/python3.8/site-packages/pywikibot/site/_apisite.py", line 1309, in validate_tokens
    types_wiki_old = self._paraminfo.parameter('query+info',
TypeError: 'NoneType' object is not subscriptable

No idea where that is coming from the password is correct. I tried updating pywikibot and all dependencies respectively but that didn't help either