rbarry82
commented
1 year ago @sanchezfdezjavier was this when removing the application or destroying a model?
There's no Python traceback in the log, which makes me suspect it's the same Juju bug (we haven't seen it on 3.x in a while, but late 2.9 series saw it semi-frequently when churning models) where juju.worker.caasunitterminationworker sends a SIGTERM while a hook is running. Operator Framework doesn't have a signal handler, so it immediately shuts down and cleans up, returns a non-zero exit code to the hook runner, and Juju believes that it's failed.
There is/was a Launchpad issue about this, but I can never find it when I want it. @jameinel does this ring a bell? @benhoyt?
Not that OF necessarily needs to, but in our KubernetesComputeResourcesPatch, we set one up for a limited window of time which will allow Juju to happily continue on even if changing the PodSpec (first unit, usually) hits the scheduling window on the k8s reconciler loop and the Juju dispatch loop.
I can't think of many meaningful scenarios where executing charm code would receive a SIGTERM|SIGKILL|SIGHUP and actually want to exit 1, but I'm sure there are some.
benhoyt
sanchezfdezjavier
Bug Description
The hook
update-statusfails intermittently after deployment. We are experiencing this issue in our CI pipeline in a not frequently active project, after a while, there was a change that triggered the CI andprometheus-k8sfailed to deploy successfully. Although we've had successful runs, we're not able to reproduce the bug consistently.Here's a sample failed run
To Reproduce
Deploy
prometheus-k8sand wait for it to failEnvironment
We've experienced the issue using the
latest/stableand2.9.38-ubuntu-amd64in two different environments:charmed-kubernetes/actions-operator@mainaction.Relevant log output
Additional context
No response