CM has many operations that can require heavy reddit API usage or can exhibit high usage when executed many times in close succession across the entire system. This became apparent due recent reddit issues that were temporarily addressed with #52
For instance, in isolation a heartbeat operation does not appear to cause heavy API usage:
iterate through subreddits for a bot
make 1 api call to wiki page to check for updates and update config accordingly
if subreddit state is valid and polling is stopped, restart it (no immediate api usage, activities pulled 30 seconds after start)
if subreddit state is valid and queue is paused/stopped, restart it
However there are unrealized consequences of this operation when multiple bots are performing their heartbeat at the same time and when there are many subreddits for a bot because...
almost simultaneous api calls per bot to get wiki page so 6 bots = 6x api calls/second~ish
multiple subreddits enter valid state and polling is restarted means many activities ingested at the same time
EX 5 bots, 1 subreddit each enters valid state with polling => 5 api calls to modqueue simultaneous => 5 activities processing at the same time when polling ingests
While individual bots/subreddits have concurrent queue worker limits there is no global limit so more bots = more concurrent workers all making api calls for their individual processing activities when queue is restarted
The result is that, while api usage may be reasonable for each activity/subreddit/bot in isolation, cumulatively CM can be slamming the api hard is infrequent but heavy spikes.
Currently the solution to this is many instances of await sleep() sprinkled throughout bot/manager startup and heartbeat operations in order to help stagger queue/polling (re)starts and wiki checks. This isn't tenable as a solution long-term though since it requires tweaking via operator configuration (by setting stagger time per bot).
A better solution to this comes in two forms:
use higher-level queues to moderate timing of high-impact operations
use snoowrap's requestDelay to enforce minimum delays between all api calls
Higher-level Queues
Regardless of snoowrap usage the higher-level queues will be useful for offsetting polling and heartbeat intervals to try to "smooth" out timing of queuing consumption and activity ingestion without needing user interaction in operator config to control timing.
Could also allow operator to configure concurrency limits or delays from a singular value AND/OR set all bot concurrency limits (global limit).
requestDelay
This is low-level enforcement of api usage "smoothing" but comes at the cost of performance. Additionally, as-is, its too blunt of a tool. IE Don't want to have delays during activity processing but do want delays during heartbeat. Or delay only during certain activities...since the delay is set at the api client level can't control this in different areas without also cloning/instantiating new client for each area.
Will need to tinker further with this concept to see if the delay can be controlled more precisely...
CM has many operations that can require heavy reddit API usage or can exhibit high usage when executed many times in close succession across the entire system. This became apparent due recent reddit issues that were temporarily addressed with #52
For instance, in isolation a heartbeat operation does not appear to cause heavy API usage:
However there are unrealized consequences of this operation when multiple bots are performing their heartbeat at the same time and when there are many subreddits for a bot because...
The result is that, while api usage may be reasonable for each activity/subreddit/bot in isolation, cumulatively CM can be slamming the api hard is infrequent but heavy spikes.
Currently the solution to this is many instances of
await sleep()sprinkled throughout bot/manager startup and heartbeat operations in order to help stagger queue/polling (re)starts and wiki checks. This isn't tenable as a solution long-term though since it requires tweaking via operator configuration (by setting stagger time per bot).A better solution to this comes in two forms:
requestDelayto enforce minimum delays between all api callsHigher-level Queues
Regardless of snoowrap usage the higher-level queues will be useful for offsetting polling and heartbeat intervals to try to "smooth" out timing of queuing consumption and activity ingestion without needing user interaction in operator config to control timing.
Could also allow operator to configure concurrency limits or delays from a singular value AND/OR set all bot concurrency limits (global limit).
requestDelay
This is low-level enforcement of api usage "smoothing" but comes at the cost of performance. Additionally, as-is, its too blunt of a tool. IE Don't want to have delays during activity processing but do want delays during heartbeat. Or delay only during certain activities...since the delay is set at the api client level can't control this in different areas without also cloning/instantiating new client for each area.
Will need to tinker further with this concept to see if the delay can be controlled more precisely...