Closed mkalam-alami closed 3 years ago
Happened again on Oct 5th 22:35 UTC, it locked the full server despite the lower CPU priority given to the Node process
Apache access logs haven't yielded much, here's the last few requested URLs before the 22:35 crash :
[05/Oct/2020:20:43:46 +0000] "GET /post/1380/i-get-the-feeling-well-all-be-seasick-by-the-end-of-the-rating-phase/ HTTP/1.1"
[05/Oct/2020:20:43:47 +0000] "GET /scorespace-alakajam HTTP/1.1"
[05/Oct/2020:20:43:47 +0000] "GET /post/1380/i-get-the-feeling-well-all-be-seasick-by-the-end-of-the-rating-phase HTTP/1.1"
[05/Oct/2020:20:43:47 +0000] "GET /posts?p=10 HTTP/1.1"
[05/Oct/2020:20:43:48 +0000] "GET /robots.txt HTTP/1.1"
[05/Oct/2020:21:13:36 +0000] "GET /robots.txt HTTP/1.1"
[05/Oct/2020:21:13:36 +0000] "GET /robots.txt HTTP/1.1"
[05/Oct/2020:22:19:28 +0000] "GET /robots.txt HTTP/1.1"
[05/Oct/2020:22:19:28 +0000] "GET /robots.txt HTTP/1.1"
The few Node server logs in that timeframe show nothing special, except a 500 error 40mn before the freeze (now fixed with 380563b).
I have set up per-request server logging just in case it could help, but the crash seems to be more subtle than just something route-specific.
With my jsx-pistols lib remaining the main suspect, I have optimized its behavior in production mode (instead of a transpiler, it basically becomes a pass-through that only loads the compiled templates using NodeJS' require()). It should reduce its use of resources a lot. I have good hopes it will help.
Assuming fixed 🤞
A 100% CPU issue making the server inoperable has been encountered multiple times before and during the 10th Alakajam. I suspect this is related to templating, as the biggest update introduced since the previous jam is the JSX templating system.