fix: rework RemoveQuests to run in bulks

[lenisko]

Rework RemoveQuests to run in bulks of 500.

Working

INFO 2023-12-27 22:37:49 RemoveQuests for fence inside ... bbox took 47.302689ms INFO 2023-12-27 22:37:49 ClearQuest: Removed quests from 530 pokestops

[jfberry]

Right general idea; I think the batch cleaner can also apply to the quest expiry which also runs; but this will tell us if you avoid the context deadline issue

[lenisko]

Hm... yeah, might be worth... but doesn't look like it caused issues so far.

$ grep "Cleanup of quest" golbat.log  | grep -v "0 quests"
INFO 2023-12-23 00:40:26 DB - Cleanup of quest table took 3.382054017s (312909 quests)
INFO 2023-12-23 01:41:23 DB - Cleanup of quest table took 1.346596634s (6087 quests)
INFO 2023-12-24 00:03:26 DB - Cleanup of quest table took 3.472740942s (321434 quests)
INFO 2023-12-24 01:04:24 DB - Cleanup of quest table took 1.472024557s (6192 quests)
ERRO 2023-12-24 02:05:23 DB - Cleanup of quest table error Error 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
INFO 2023-12-24 02:05:23 DB - Cleanup of quest table took 570.674975ms (1 quests)
INFO 2023-12-25 00:27:26 DB - Cleanup of quest table took 3.442564913s (321481 quests)
INFO 2023-12-25 01:28:24 DB - Cleanup of quest table took 1.433195464s (6155 quests)
INFO 2023-12-26 00:51:26 DB - Cleanup of quest table took 3.441973807s (321396 quests)
INFO 2023-12-26 01:52:24 DB - Cleanup of quest table took 1.389011447s (6166 quests)
INFO 2023-12-26 02:53:22 DB - Cleanup of quest table took 94.011848ms (3 quests)
INFO 2023-12-27 00:14:25 DB - Cleanup of quest table took 3.368248017s (321597 quests)
INFO 2023-12-27 01:15:24 DB - Cleanup of quest table took 1.382255949s (6153 quests)

[jfberry]

I actually wonder whether it should do the cleanup on expiry because looking at those times there is a risk that it will try a cleanup while a quest is running in the same area if the quest run is at 00:00 rather than 01:00... ReactMap could still use the calculated expiry time for not showing the quest even if we don't clear it

[lenisko]

Confirmed working and solved my issues.

As for expire cleanup - I would keep it but add few changes to StartQuestExpiry to run each *:59:59, set a lock (checked on quests), perform cleanup and disable lock. But other PR.

[jfberry]

lgtm