username-is-required / reddark

reddark, but it's in realtime (a fork)
https://reddark.io/
GNU Affero General Public License v3.0
11 stars 1 forks source link

bug: the auto—updating system seems to sometimes…just stop #117

Open username-is-required opened 1 year ago

username-is-required commented 1 year ago

last night it seemed to happen during reddit’s downtime — the program logged that the requests had failed and would be retried, but then…. no further logging after that except for ‘number of connections’ type logs. i chalked it up to the downtime, and manually redeployed, which got it started again.

but then i wake up this morning to find it happened again — and with no logged request errors beforehand? from the look of the logs, it just started a check……and never finished it? or errored? or have the https request time out?

IMG_9422

(note: all log times are in utc)

i am very confused. i have manually redeployed again just now to get it back up and running, but this is probably a bigger issue than i thought and i’ve got no idea where the code could be hanging.

username-is-required commented 1 year ago

these are the only logged request errors for the app since the manual deployment last night (and before the manual deployment i’ve just done this morning). the last logged error was several request cycles before the one on which the app froze.

IMG_9423

username-is-required commented 1 year ago

i can’t think of anywhere it would hang except somewhere inside requests.js, which makes things more difficult as i’m not that familiar with the https module. i’m thinking i might just change up requests.js to use something like axios instead

username-is-required commented 1 year ago

i added in a request.end() call to requests.js because apparently it should have one of those. have deployed and i guess we will see if it randomly hangs again (though given the seemingly random nature of the request hanging, annoyingly, it’ll be pretty difficult to tell if the issue is fixed or not just by the script not hanging on a request).

username-is-required commented 1 year ago

okay i’ve added in a dead-man’s switch in #119, this should kill the process if it detects a request that has hung for 10 minutes (and then i believe digitalocean should restart it). also configured it for @username-is-required-bot to post a comment to this issue if/when that happens (because why not)

(edit: see now i hope b949f3b hasn’t fixed this because i really want to see my bot making a comment in this issue)

username-is-required-bot commented 1 year ago

2023-06-20T07:40:36.151Z [ALERT] Reddark: request hung for 10 minutes (exiting process)


this comment was made by a bot, beep boop

username-is-required commented 1 year ago

well, at least this confirms there’s still a problem lol (and that my dead man’s switch code works)

over the last few days i prioritised tracking the johnoliverification™ (#124) over this (as it doesn’t happen that often and now has a workaround), however i’ll make it my next priority to resolve - hopefully fixing the promise antipattern (#122 issue, #123 pr) should reveal what the issue is.

username-is-required-bot commented 1 year ago

2023-06-21T01:12:35.904Z [ALERT] Reddark: request hung for 10 minutes (exiting process)


this comment was made by a bot, beep boop

username-is-required-bot commented 1 year ago

2023-06-21T01:12:36.486Z [ALERT] Reddark: request hung for 10 minutes (exiting process)


this comment was made by a bot, beep boop

username-is-required-bot commented 1 year ago

2023-06-21T01:24:23.275Z [ALERT] Reddark: request hung for 10 minutes (exiting process)


this comment was made by a bot, beep boop

username-is-required-bot commented 1 year ago

2023-06-28T01:35:44.554Z [ALERT] Reddark: request hung for 10 minutes (exiting process)


this comment was made by a bot, beep boop

username-is-required commented 1 year ago

eh i guess im just sorta letting this occasionally break and restart for right now at least

username-is-required commented 1 year ago

this is still occurring even after getting rid of the antipattern. damnit.