search

jmhertlein / comfydns

The world's most comfortable implementation of RFC 1035.
GNU Affero General Public License v3.0
8 stars 0 forks source link

shore up "lost" request logic #36

Closed jmhertlein closed 2 years ago

jmhertlein commented 2 years ago
  1. Install unhandled exception handlers to make sure I'm getting all my logs
  2. Consider adding object lifetime hooks (maybe the new Cleaner API?), and set some state in Request objects so that they mark when an answer has gone out for them, and then in the hook, if the request hasn't been answered, mark that request as lost in prometheus (increment a new requests_lost metric)
  3. Go through and put extra guardrail logic in to see if we can get better coverage of exception handling: there's small bits of logic that I think might be able to throw exceptions that won't get caught, like when we're setting up callbacks and such.
jmhertlein commented 2 years ago

I think, instead of tracking lifespan, I'm instead going to build a really good story around logging "in" and "out" INFO prints such that I could match metrics timestamps and postprocess logs to look for requests that came IN but don't have a matching OUT.

jmhertlein commented 2 years ago

done in 66ab435e027ae33a3828c7cbcd76f60f6b31d22a