is there a way for the list of participating subs to be periodically refreshed?

[username-is-required]

i’m just asking as ModCoord seem to be updating their megathreads a few times a day with more subs (with the updated list now including the largest sub so far, r/funny). from what i can see, at the moment the only way would be to kill the node process hosting the site and re-run it again.

if you’d like i can try and bodge together a pr that uses setInterval to refresh the list a few times a day? (maybe 4?)

alternatively, given how (relatively) little the megathreads will need reloading, i’d understand if you just wanted to keep things how they are & just rerun the node file every now and then.

[Tanza3D]

that sounds like it could be helpful! should be able to just re-emit the subreddits array to the clients and hopefully it'd be stable at the moment every few hours i go and manually restart the service, not too much of a bump but it updates the list. i'll do this right now for the time being

[username-is-required]

@Tanza3D what i was thinking was to emit loading to trigger the “Server Reloading” message on the clients (rather than just a sudden change in subreddits). a ‘reloading’ message would hopefully indicate that - well, something’s being reloaded - in this case the list of participating subs.

if you’re manually restarting it every few hours what do you think about timing it for every 3 hours? i can’t imagine it would be that much more server-intensive (compared to the 4000+ requests every time updateStatus runs, updating the sub lists should be a walk in the park lol)

i’ll get doing a pr now, i’ll put it in draft mode until i’ve finished. and please tell me if something i code doesn’t look right!

[username-is-required]

also @Tanza3D are you able to check the logs to see if the auto-updating broke again before you restarted the process just now? i’ve noticed the number of subreddits listed as going dark jumped by about 40-ish after the restart just now

[Tanza3D]

to your first reply, there's no need to hide the list meanwhilst i don't feel, we can just pause it while we get all the data collected, and then send the new list and the client should reset its html and load in all the new ones properly

for your second one, i'll take a look now

[Tanza3D]

somehow hitting enter to post my comment closed the issue! thanks, github

[Tanza3D]

seems like it dropped here

[Tanza3D]

i think your pr actually fixes this one already, from a while back, so we're good i think but im pushing another change just in case :3

[username-is-required]

i’m actually having trouble trying to think of the best way to implement the automatic list refreshing.

it’d be relatively easy to set the list to recreate itself every 3 hours - the code currently at the bottom of main.js could change to something like this, with maybe a few minor modifications to createList() as well:

async function run () {
    await createList();
    updateStatus();
}

(async () => {
    // create the list and start the updating
    run();
    // every 3 hours, repeat creating the list & restart the updating
    setInterval(run, 10800000);
})();

to me the hard part is trying to figure out how to control what’s going on with updateStatus() at the time we want to refresh the list.

presumably, when the list is refreshing, we don’t want updateStatus to be running. we can easily restart it after the list’s been refreshed. but then, how do we stop it running before we make a new call to createList?

the only idea that comes to mind is a sort of two-way boolean flag — e.g. the run function might set a variable called doNotUpdate to true (which could be picked up on the first line of updateStatus and cause it to return straightaway). but then, how would the run function know that that point’s been reached (and that the updateStatus function has stopped executing)? well, upon finding doNotUpdate equal to true, updateStatus could set a second boolean flag - kinda like an acknowledgment. this boolean flag would need to be being watched within the run function (perhaps on a setTimeout to check the variable several times a second), in order to know when it’s safe to proceed to call createList.

and i just think, there must be a better way of doing it than this, but i can’t for the life of me figure out what it is.

[Tanza3D]

presumably, when the list is refreshing, we don’t want updateStatus to be running. we can easily restart it after the list’s been refreshed. but then, how do we stop it running before we make a new call to createList?

my dumb solution would be to have a bool saying "scrapUpdate", and set that to true, which makes all the http requests return and do nothing. then have a bool you can pass into the updateStatus() function called "newUpdate" which bypasses that bool when true and sets scrapUpdate to false when finished

[username-is-required]

i edited the script in my forked repo to (hopefully successfully) automatically refresh the subreddit list (username-is-required#6), however there’s slight difference in the codebase from when i modified my fork to use setInterval, Promise.all, etc.

it was slightly easier to implement there because the script awaits the result of updateStatus before calling it again, so i could (a) add a flag that would be set to true every X hours, and (b) tell the function that repeatedly runs updateStatus to check for that flag before each run and recreate the list if it’s set to true.

i might try and give a pr to this repo another go, but i’m just conscious that when i tried yesterday i was finding it hard to wrap my head around how to implement it a little bit, so i can’t promise that i’ll be able to submit a pr unfortunately.

if it would be helpful, please feel free to copy any of the code in the fork for modification/use in this repo

[username-is-required]

actually i’ve just had an idea

if the current workaround is just restarting main.js every so often, you could probably create a bash script with the sole purpose of periodically killing and restarting the node process

maybe something like:

#!/bin/bash

while true
do
  node main.js &
  PID=$!

  sleep 10800

  kill $PID
done

[Tanza3D]

i use a systemctl service on the serverside, so if more than anything i'd use a crontab to run systemctl restart reddark every hour or so, but my worry is it'd restart right on the beginning of an hour which would probably be the time subreddits start going dark tomorrow

[rhysperry111]

Any update on this? (the stream has had the same number for a while now, so I came over here to check how it works internally)