Open JN-Jones opened 3 weeks ago
Perhaps the solution would be to make login()
atomic.
It may be next month that I'm able to address this and the other issues – if you want to take a shot at fixing them, please feel free to raise PRs.
(As an aside, no need to call both exists
and text
– just read()
will do the work of both, and is also quite faster as it uses action=query instead of action=parse.)
It may be next month that I'm able to address this and the other issues – if you want to take a shot at fixing them, please feel free to raise PRs.
Currently I've to get the bot running, but afterwards I plan to take a deeper look at your library and address some of the issues I've created.
(As an aside, no need to call both exists and text – just read() will do the work of both, and is also quite faster as it uses action=query instead of action=parse.)
There's some additional logic happening in my framework (mainly hooking up some additional functions for common tasks in our wiki), esp for getting the text. For now I've switched to read
when working with multiple articles as I don't need the additional functionality here.
I currently have a task running, which has around 40k of articles which are looped using
bot.batchOperation
. Each loop callsexists
andtext
. However after around 5k items it seems the wiki instance kills my session (still discussing setting up OAuth with the administration, currently username/password login is used). Withassert: 'bot'
it correctly attempts a re-login, however with the parallel requests the login then fails because the different callbacks all try to get a token and then attempt to login with potentially now invalid tokens (iegetToken (1) => getToken (2) => login (1)
will fail because the first token was invalidated when requesting the second one).Probably related to #38