search

iT3E / tnwks-ops

A mono repository for my home infrastructure and Kubernetes cluster which adheres to Infrastructure as Code (IaC) and GitOps practices where possible
https://iT3E.github.io/tnwks-ops
MIT License
1 stars 0 forks source link

bug: pod-gateway crashing randomly #115

Open iT3E opened 1 year ago

iT3E commented 1 year ago

Crashes cause outage of any containers using pod-gateway, requiring (afaik) a restart of the pod before it can recognize the "new" pod-gateway that relaunched.

iT3E commented 1 year ago

kubectl describe and log show no faults. Pod restart hasn't happened for 4+ hours, and no change was made, other than all containers (4 total 🤔) not being able to access the restarted, "new" container.

downloads-gateway-pod-gateway-85dd5cbb76-gmwf6           3/3     Running   4 (4h35m ago)   10h
iT3E commented 1 year ago

trying e012ca06a36bdde2632cb2d1e67c6a7b6a2f5674 because maybe this is a resource constraint issue somewhere? radarr and sonarr don't need to use the VPN either way.

iT3E commented 1 year ago

https://github.com/qdm12/gluetun/issues/641 - symptoms due to this bug

My hunch is that the cause of the symptoms (container restarting) is due to wireguard failures upstream. I'm attempting to change the VPN to somewhere geographically closer. This won't fix the bug above, but may reduce occurrences of the symptoms.

iT3E commented 1 year ago 
downloads-gateway-pod-gateway-5b5989fbc8-ft586           2/2     Running   0               27h

looking like the hunch was correct. No pod restarts after changing VPN locations, along with no VPN rebuilds in the logs.

iT3E commented 1 year ago

0a3d855684e8b859c1eb4dcfea166e42ade4fc43