siddharthvp / mwn

A JavaScript & TypeScript MediaWiki bot framework for Node.js
https://mwn.toolforge.org
GNU Lesser General Public License v3.0
45 stars 9 forks source link

Re-login doesn't work in batchOperations #83

Open JN-Jones opened 3 weeks ago

JN-Jones commented 3 weeks ago

I currently have a task running, which has around 40k of articles which are looped using bot.batchOperation. Each loop calls exists and text. However after around 5k items it seems the wiki instance kills my session (still discussing setting up OAuth with the administration, currently username/password login is used). With assert: 'bot' it correctly attempts a re-login, however with the parallel requests the login then fails because the different callbacks all try to get a token and then attempt to login with potentially now invalid tokens (ie getToken (1) => getToken (2) => login (1) will fail because the first token was invalidated when requesting the second one).

Probably related to #38

siddharthvp commented 3 weeks ago

Perhaps the solution would be to make login() atomic.

It may be next month that I'm able to address this and the other issues – if you want to take a shot at fixing them, please feel free to raise PRs.

(As an aside, no need to call both exists and text – just read() will do the work of both, and is also quite faster as it uses action=query instead of action=parse.)

JN-Jones commented 3 weeks ago

It may be next month that I'm able to address this and the other issues – if you want to take a shot at fixing them, please feel free to raise PRs.

Currently I've to get the bot running, but afterwards I plan to take a deeper look at your library and address some of the issues I've created.

(As an aside, no need to call both exists and text – just read() will do the work of both, and is also quite faster as it uses action=query instead of action=parse.)

There's some additional logic happening in my framework (mainly hooking up some additional functions for common tasks in our wiki), esp for getting the text. For now I've switched to read when working with multiple articles as I don't need the additional functionality here.