Describe the bug
One shot jobs have no timeout and no error checking. If there is an issue where the pod cannot come online, CodeAmp waits on it forever. The result is that a kubernetes worker is stuck and must be destroyed. Unfortunately there is no current way of knowing which job is on which worker so it is a dangerous operation and is ideally done when no other deploys are ongoing (or all are stuck)
Describe the bug One shot jobs have no timeout and no error checking. If there is an issue where the pod cannot come online, CodeAmp waits on it forever. The result is that a kubernetes worker is stuck and must be destroyed. Unfortunately there is no current way of knowing which job is on which worker so it is a dangerous operation and is ideally done when no other deploys are ongoing (or all are stuck)